Merged commit includes the following changes:

517007585  by Waymo Research:

    Fix test errors.

--
516987144  by Waymo Research:

    Fix a typo.

--
516984381  by Waymo Research:

    Update license headers.

--
516984214  by Waymo Research:

    Internal change.

--
516975261  by Waymo Research:

    Fix Docker/PIP build.

--
516973092  by Waymo Research:

    Update readme

--
516947665  by Waymo Research:

    Adopt WOD matcher which is consistent with other WOPD tasks.

--
516939776  by Waymo Research:

    Add python API for open dataset perception object matcher.

--

PiperOrigin-RevId: 517007585

README.md

@@ -10,22 +10,22 @@ This major update includes supporting code to four challenges at waymo.com/open,
 v2.0.0 of the Perception Dataset
  - Introduced the dataset in modular format, enabling users to selectively download only the components they need.
  - Includes all features in v1.4.2 of the Perception Dataset except maps.
- - Added a [tutorial](waymo-open-dataset/tutorial/tutorial_v2.ipynb) and supporting code.
+ - Added a [tutorial](tutorial/tutorial_v2.ipynb) and supporting code.
 
 v1.4.2 of the Perception Dataset
  - For the 2D video panoptic segmentation labels, added a mask to indicate the number of cameras covering each pixel.
  - Added 3D map data as polylines or polygons.
 
 v1.2.0 of the Motion Dataset
- - Added Lidar data for the training set (first 1s of each 9s windows), and the corresponding [tutorial](waymo_open_dataset/tutorial/tutorial_womd_lidar.ipynb) and supporting code.
+ - Added Lidar data for the training set (first 1s of each 9s windows), and the corresponding [tutorial](tutorial/tutorial_womd_lidar.ipynb) and supporting code.
  - Added driveway entrances to the map data. Adjusted some road edge boundary height estimates.
  - Increased the max number of map points in tf_examples to 30k and reduced sampling to 1.0m to increase map coverage, so the coverage equalizes that of the dataset in scenario proto format. Added conversion code from scenario proto format to tf_examples format.
 
 Added supporting code for the four 2023 Waymo Open Dataset Challenges
- - Sim Agents Challenge, with a [tutorial](waymo_open_dataset/tutorial/tutorial_sim_agents.ipynb)
- - Pose Estimation Challenge, with a [tutorial](waymo_open_dataset/tutorial/tutorial_keypoints.ipynb)
- - 2D Video Panoptic Segmentation Challenge, with a [tutorial](waymo_open_dataset/tutorial/tutorial_2d_pvps.ipynb)
- - Motion Prediction Challenge, with a [tutorial](waymo_open_dataset/tutorial/tutorial_motion.ipynb)
+ - Sim Agents Challenge, with a [tutorial](tutorial/tutorial_sim_agents.ipynb)
+ - Pose Estimation Challenge, with a [tutorial](tutorial/tutorial_keypoints.ipynb)
+ - 2D Video Panoptic Segmentation Challenge, with a [tutorial](tutorial/tutorial_2d_pvps.ipynb)
+ - Motion Prediction Challenge, with a [tutorial](tutorial/tutorial_motion.ipynb)
 
 ## December 2022 Update
 We released v1.4.1 of the Perception dataset.
@@ -36,8 +36,8 @@ We released v1.4.0 of the Perception dataset.
  - Added 2D video panoptic segmentation labels and supporting code.
 
 ## May 2022 Update (part 2)
- - Released a [tutorial](waymo-open-dataset/tutorial/tutorial_camera_only.ipynb) for the 3D Camera-Only Detection Challenge.
- - Added support for computing 3D-LET-APL in Python metrics ops. See `Compute Metrics` in the [tutorial](waymo-open-dataset/tutorial/tutorial_camera_only.ipynb).
+ - Released a [tutorial](tutorial/tutorial_camera_only.ipynb) for the 3D Camera-Only Detection Challenge.
+ - Added support for computing 3D-LET-APL in Python metrics ops. See `Compute Metrics` in the [tutorial](tutorial/tutorial_camera_only.ipynb).
  - Fixed a bug in the metrics implementation for the Occupancy and Flow Challenge.
 
 ## May 2022 Update
@@ -53,7 +53,7 @@ We released v1.3.1 of the Perception dataset to support the 2022 Challenges and
  - Added z-axis speed and acceleration in [lidar label metadata](waymo_open_dataset/label.proto#L53-L60).
  - Fixed some inconsistencies in `projected_lidar_labels` in [dataset.proto](waymo_open_dataset/dataset.proto).
  - Updated the default configuration for the Occupancy and Flow Challenge, switching from aggregate waypoints to [subsampled waypoints](waymo_open_dataset/protos/occupancy_flow_metrics.proto#L38-L55).
- - Updated the [tutorial](waymo-open-dataset/tutorial/tutorial_3d_semseg.ipynb) for 3D Semantic Segmentation Challenge with more detailed instructions.
+ - Updated the [tutorial](tutorial/tutorial_3d_semseg.ipynb) for 3D Semantic Segmentation Challenge with more detailed instructions.
 
 ## March 2022 Update
 

configure.sh

@@ -1,5 +1,5 @@
 #!/bin/bash
-# Copyright 2019 The Waymo Open Dataset Authors. All Rights Reserved.
+# Copyright 2019 The Waymo Open Dataset Authors.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.

pip_pkg_scripts/build.Dockerfile

@@ -38,7 +38,7 @@ RUN for python in python3.7 python3.8 python3.9; do \
       $python get-pip.py && \
       $python -m pip install --upgrade pip setuptools auditwheel && \
       $python -m pip install --upgrade grpcio>=1.24.3; \
-      $python -m pip install --upgrade matplotlib plotly scikit-image immutabledict scipy absl-py pandas==1.4 pyarrow dask; \
+      $python -m pip install --upgrade matplotlib plotly scikit-image immutabledict scipy sklearn absl-py pandas==1.4 numpy pyarrow dask[array]; \
       $python -m pip install --upgrade tensorflow==${TF_VERSION} OpenEXR==1.3.2 tensorflow_graphics; \
     done
 

pip_pkg_scripts/build.sh

@@ -1,5 +1,5 @@
 #!/bin/bash
-# Copyright 2019 The Waymo Open Dataset Authors. All Rights Reserved.
+# Copyright 2019 The Waymo Open Dataset Authors.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.

pip_pkg_scripts/build_pip_pkg.sh

@@ -1,5 +1,5 @@
 #!/usr/bin/env bash
-# Copyright 2019 The Waymo Open Dataset Authors. All Rights Reserved.
+# Copyright 2019 The Waymo Open Dataset Authors.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.

pip_pkg_scripts/docker_build_all.sh

@@ -1,5 +1,5 @@
 #!/bin/bash
-# Copyright 2021 The Waymo Open Dataset Authors. All Rights Reserved.
+# Copyright 2021 The Waymo Open Dataset Authors.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.

pip_pkg_scripts/setup.py.in

@@ -1,4 +1,4 @@
-# Copyright 2019 The Waymo Open Dataset Authors. All Rights Reserved.
+# Copyright 2019 The Waymo Open Dataset Authors.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -23,6 +23,7 @@ REQUIRED_PACKAGES = [
     'tensorflow==' + '.'.join('TF_VERSION'.split('-')),
     'plotly>=5.5.0',
     'matplotlib>=3.2.2',
+    'sklearn',
     'scikit-image>=0.17.2',
     'immutabledict>=2.0.0',
     'scipy>=1.9.3',

third_party/auditwheel_wrapper.py

@@ -1,4 +1,4 @@
-# Copyright 2022 The Waymo Open Dataset Authors. All Rights Reserved.
+# Copyright 2022 The Waymo Open Dataset Authors.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.

third_party/camera/camera_model.cc

@@ -108,8 +108,8 @@ double GetPixelTimestamp(
 // In normalized camera, undistorts point coordinates via iteration.
 void IterateUndistortion(const CameraCalibration& calibration, double u_nd,
                          double v_nd, double* u_n, double* v_n) {
- /* CHECK(u_n); */
- /* CHECK(v_n); */
+  CHECK(u_n);
+  CHECK(v_n);
   const double f_u = calibration.intrinsic(0);
   const double f_v = calibration.intrinsic(1);
   const double k1 = calibration.intrinsic(4);
@@ -125,8 +125,8 @@ void IterateUndistortion(const CameraCalibration& calibration, double u_nd,
   u = u_nd;
   v = v_nd;
 
- /* CHECK_GT(f_u, 0.0); */
- /* CHECK_GT(f_v, 0.0); */
+  CHECK_GT(f_u, 0.0);
+  CHECK_GT(f_v, 0.0);
 
   // Minimum required squared delta before terminating. Note that it is set in
   // normalized camera coordinates at a fraction of a pixel^2. The threshold
@@ -473,9 +473,9 @@ void CameraModel::ImageToWorld(double u_d, double v_d, double depth, double* x,
 void CameraModel::ImageToWorldGlobalShutter(double u_d, double v_d,
                                             double depth, double* x, double* y,
                                             double* z) const {
- /* CHECK(x); */
- /* CHECK(y); */
- /* CHECK(z); */
+  CHECK(x);
+  CHECK(y);
+  CHECK(z);
   double u_n = 0.0, v_n = 0.0;
   ImageToDirection(u_d, v_d, &u_n, &v_n);
   const Eigen::Vector3d wp = global_shutter_state_->n_tfm_cam0 *
@@ -488,10 +488,10 @@ void CameraModel::ImageToWorldGlobalShutter(double u_d, double v_d,
 void CameraModel::ImageToVehicleGlobalShutter(double u_d, double v_d,
                                               double depth, double* x,
                                               double* y, double* z) const {
- /* CHECK(x); */
- /* CHECK(y); */
- /* CHECK(z); */
- /* CHECK(global_shutter_state_) << "Please call PrepareProjection() first."; */
+  CHECK(x);
+  CHECK(y);
+  CHECK(z);
+  CHECK(global_shutter_state_) << "Please call PrepareProjection() first.";
   double u_n = 0.0, v_n = 0.0;
   ImageToDirection(u_d, v_d, &u_n, &v_n);
   const Eigen::Vector3d wp = global_shutter_state_->vehicle_tfm_cam *
@@ -542,8 +542,8 @@ bool CameraModel::WorldToImageWithDepthGlobalShutter(double x, double y,
                                                      bool check_image_bounds,
                                                      double* u_d, double* v_d,
                                                      double* depth) const {
- /* CHECK(u_d); */
- /* CHECK(v_d); */
+  CHECK(u_d);
+  CHECK(v_d);
   const Eigen::Vector3d cp =
       global_shutter_state_->cam_tfm_n * Eigen::Vector3d(x, y, z);
   return CameraToImageWithDepth(cp(0), cp(1), cp(2), check_image_bounds, u_d,
@@ -624,9 +624,9 @@ bool CameraModel::ComputeDepthResidualAndJacobian(
     double* jacobian) const {
   // The jacobian is allowed to be a nullptr.
   // The depth is allowed to be a nullptr.
- /* CHECK(normalized_coord); */
- /* CHECK(residual); */
- /* CHECK(rolling_shutter_state_); */
+  CHECK(normalized_coord);
+  CHECK(residual);
+  CHECK(rolling_shutter_state_);
   const RollingShutterState& rolling_shutter_state = *rolling_shutter_state_;
 
   const Eigen::Matrix3d cam_dcm_n = rolling_shutter_state.cam0_dcm_n +

third_party/camera/ops/camera_model_ops.cc

@@ -66,7 +66,7 @@ DataType GetTensorflowType() {
   if (std::is_same<absl::remove_const_t<T>, float>::value) {
     return DT_FLOAT;
   }
- /* CHECK(false) << "Unsupported type."; */
+  CHECK(false) << "Unsupported type.";
 }
 
 // Parse input tensors to protos.
@@ -76,25 +76,25 @@ void ParseInput(const Input& input, co::CameraCalibration* calibration_ptr,
   auto& calibration = *calibration_ptr;
   auto& image = *image_ptr;
 
- /* CHECK_EQ(input.extrinsic->dim_size(0), 4); */
- /* CHECK_EQ(input.extrinsic->dim_size(1), 4); */
+  CHECK_EQ(input.extrinsic->dim_size(0), 4);
+  CHECK_EQ(input.extrinsic->dim_size(1), 4);
   for (int i = 0; i < 4; ++i) {
     for (int j = 0; j < 4; ++j) {
       calibration.mutable_extrinsic()->add_transform(
           input.extrinsic->matrix<T>()(i, j));
     }
   }
- /* CHECK_EQ(input.intrinsic->dim_size(0), kIntrinsicLen); */
+  CHECK_EQ(input.intrinsic->dim_size(0), kIntrinsicLen);
   for (int i = 0; i < kIntrinsicLen; ++i) {
     calibration.add_intrinsic(input.intrinsic->vec<T>()(i));
   }
- /* CHECK_EQ(input.metadata->dim_size(0), kMetadataLen); */
+  CHECK_EQ(input.metadata->dim_size(0), kMetadataLen);
   calibration.set_width(input.metadata->vec<int32>()(0));
   calibration.set_height(input.metadata->vec<int32>()(1));
   calibration.set_rolling_shutter_direction(
       static_cast<co::CameraCalibration::RollingShutterReadOutDirection>(
           input.metadata->vec<int32>()(2)));
- /* CHECK_EQ(input.camera_image_metadata->dim_size(0), kCameraImageMedataLen); */
+  CHECK_EQ(input.camera_image_metadata->dim_size(0), kCameraImageMedataLen);
   int idx = 0;
   const auto& cim = input.camera_image_metadata->vec<T>();
   for (; idx < 16; ++idx) {
@@ -138,7 +138,7 @@ class WorldToImageOp : public OpKernel {
 
     const int num_points = input.input_coordinate->dim_size(0);
     const int out_channel = 3 + return_depth_;
-   /* CHECK_EQ(3, input.input_coordinate->dim_size(1)); */
+    CHECK_EQ(3, input.input_coordinate->dim_size(1));
     Tensor image_coordinates(GetTensorflowType<T>(), {num_points, out_channel});
     for (int i = 0; i < num_points; ++i) {
       double u_d = 0.0;
@@ -191,7 +191,7 @@ class ImageToWorldOp final : public OpKernel {
     model.PrepareProjection(image);
 
     const int num_points = input.input_coordinate->dim_size(0);
-   /* CHECK_EQ(3, input.input_coordinate->dim_size(1)); */
+    CHECK_EQ(3, input.input_coordinate->dim_size(1));
     Tensor global_coordinates(GetTensorflowType<T>(), {num_points, 3});
     for (int i = 0; i < num_points; ++i) {
       double x = 0.0;

third_party/camera_segmentation/camera_segmentation_metrics.py

@@ -1,4 +1,4 @@
-# Copyright 2023 The Waymo Open Dataset Authors. All Rights Reserved.
+# Copyright 2023 The Waymo Open Dataset Authors.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -19,8 +19,8 @@
 
 import numpy as np
 
-from deeplab2.deeplab.deeplab2.data import dataset
-from deeplab2.deeplab.deeplab2.evaluation import segmentation_and_tracking_quality
+from deeplab2..data import dataset
+from deeplab2..evaluation import segmentation_and_tracking_quality
 from waymo_open_dataset.protos import camera_segmentation_metrics_pb2
 
 

third_party/camera_segmentation/camera_segmentation_metrics_test.py

@@ -1,4 +1,4 @@
-# Copyright 2023 The Waymo Open Dataset Authors. All Rights Reserved.
+# Copyright 2023 The Waymo Open Dataset Authors.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.

third_party/sim_agents_metrics/estimators.py

@@ -1,4 +1,4 @@
-# Copyright 2023 The Waymo Open Dataset Authors. All Rights Reserved.
+# Copyright 2023 The Waymo Open Dataset Authors.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.

third_party/sim_agents_metrics/estimators_test.py

@@ -1,4 +1,4 @@
-# Copyright 2023 The Waymo Open Dataset Authors. All Rights Reserved.
+# Copyright 2023 The Waymo Open Dataset Authors.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.

third_party/sim_agents_metrics/interaction_features.py

@@ -1,4 +1,4 @@
-# Copyright 2023 The Waymo Open Dataset Authors. All Rights Reserved.
+# Copyright 2023 The Waymo Open Dataset Authors.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.

third_party/sim_agents_metrics/interaction_features_test.py

@@ -1,4 +1,4 @@
-# Copyright 2023 The Waymo Open Dataset Authors. All Rights Reserved.
+# Copyright 2023 The Waymo Open Dataset Authors.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.

third_party/sim_agents_metrics/map_metric_features.py

@@ -1,4 +1,4 @@
-# Copyright 2023 The Waymo Open Dataset Authors. All Rights Reserved.
+# Copyright 2023 The Waymo Open Dataset Authors.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.

third_party/sim_agents_metrics/map_metric_features_test.py

@@ -1,4 +1,4 @@
-# Copyright 2023 The Waymo Open Dataset Authors. All Rights Reserved.
+# Copyright 2023 The Waymo Open Dataset Authors.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.

third_party/sim_agents_metrics/metric_features.py

@@ -1,4 +1,4 @@
-# Copyright 2023 The Waymo Open Dataset Authors. All Rights Reserved.
+# Copyright 2023 The Waymo Open Dataset Authors.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.

third_party/sim_agents_metrics/metric_features_test.py

@@ -1,4 +1,4 @@
-# Copyright 2023 The Waymo Open Dataset Authors. All Rights Reserved.
+# Copyright 2023 The Waymo Open Dataset Authors.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.

third_party/sim_agents_metrics/metrics.py

@@ -1,4 +1,4 @@
-# Copyright 2023 The Waymo Open Dataset Authors. All Rights Reserved.
+# Copyright 2023 The Waymo Open Dataset Authors.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.

third_party/sim_agents_metrics/metrics_test.py

@@ -1,4 +1,4 @@
-# Copyright 2023 The Waymo Open Dataset Authors. All Rights Reserved.
+# Copyright 2023 The Waymo Open Dataset Authors.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.

third_party/sim_agents_metrics/trajectory_features.py

@@ -1,4 +1,4 @@
-# Copyright 2023 The Waymo Open Dataset Authors. All Rights Reserved.
+# Copyright 2023 The Waymo Open Dataset Authors.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.

third_party/sim_agents_metrics/trajectory_features_test.py

@@ -1,4 +1,4 @@
-# Copyright 2023 The Waymo Open Dataset Authors. All Rights Reserved.
+# Copyright 2023 The Waymo Open Dataset Authors.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.

third_party/toolchains/preconfig/ubuntu16.04/gcc7_manylinux2010-nvcc-cuda10.0/clang/bin/crosstool_wrapper_driver_is_not_gcc

@@ -1,5 +1,5 @@
 #!/usr/bin/env python
-# Copyright 2015 The TensorFlow Authors. All Rights Reserved.
+# Copyright 2015 The TensorFlow Authors.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.

third_party/toolchains/preconfig/ubuntu16.04/gcc7_manylinux2010-nvcc-cuda10.0/windows/msvc_wrapper_for_nvcc.py

@@ -1,5 +1,5 @@
 #!/usr/bin/env python
-# Copyright 2015 The TensorFlow Authors. All Rights Reserved.
+# Copyright 2015 The TensorFlow Authors.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.

tutorial/jupyter_kernel.py

@@ -1,4 +1,4 @@
-# Copyright 2019 The Waymo Open Dataset Authors. All Rights Reserved.
+# Copyright 2019 The Waymo Open Dataset Authors.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.

waymo_open_dataset/common/status.cc

@@ -1,4 +1,4 @@
-/* Copyright 2021 The Waymo Open Dataset Authors. All Rights Reserved.
+/* Copyright 2021 The Waymo Open Dataset Authors.
 
 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.

waymo_open_dataset/common/status.h

@@ -1,4 +1,4 @@
-/* Copyright 2021 The Waymo Open Dataset Authors. All Rights Reserved.
+/* Copyright 2021 The Waymo Open Dataset Authors.
 
 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.

waymo_open_dataset/data_conversion/BUILD

@@ -28,6 +28,7 @@ cc_library(
         "@com_google_absl//absl/status",
         "@com_google_absl//absl/status:statusor",
         "@com_google_absl//absl/strings",
+        "@com_google_glog//:glog",
         "@com_google_protobuf//:protobuf_lite",
         "@local_config_tf//:libtensorflow_framework",
         "@local_config_tf//:tf_header_lib",