Skip to content

feat(transformations): support custom reference timestamp in motion (de)compensation#146

Merged
janickm merged 1 commit into
NVIDIA:mainfrom
janickm:compensator-reference-time-v2
Jun 10, 2026
Merged

feat(transformations): support custom reference timestamp in motion (de)compensation#146
janickm merged 1 commit into
NVIDIA:mainfrom
janickm:compensator-reference-time-v2

Conversation

@janickm

@janickm janickm commented Jun 10, 2026

Copy link
Copy Markdown
Collaborator

Summary

Add an optional reference_timestamp_us parameter to MotionCompensator.motion_compensate_points and motion_decompensate_points, so callers can choose which in-frame timestamp the points are motion-compensated to / from.

It defaults to frame_end_timestamp_us, preserving the existing end-of-frame behavior used by the KITTI / nuScenes / Waymo converters.

Motivation

Different datasets motion-compensate lidar sweeps to different in-frame references. The API previously hard-coded the end-of-frame reference. Argoverse 2, for example, compensates each sweep to the sweep reference timestamp, which is the start of the sweep (offset_ns runs forward from it). Without an explicit reference, decompensating such data with the end-of-frame assumption applies the full intra-sweep ego motion (~1 m) as a systematic error.

This makes the compensation reference an explicit, optional parameter so converters can express their dataset's convention directly.

Changes

  • motion_compensate_points / motion_decompensate_points: new reference_timestamp_us: Optional[int] = None (defaults to frame_end_timestamp_us), asserted to lie within [frame_start, frame_end].
  • motion_decompensate_points is reimplemented as the exact inverse of motion_compensate_points: per point it evaluates the sensor pose at the point's own timestamp relative to the reference sensor pose (T_sensor_pointtime_world @ T_world_sensor_reftime) instead of linearly interpolating the end->start relative transform. This generalizes to any reference and is at least as accurate.
  • MotionCompensationResult: fields renamed from the (now misleading) xyz_s_sensorend / xyz_e_sensorend to xyz_s_reftime / xyz_e_reftime, and the struct now carries reference_timestamp_us unconditionally.
  • Internal transforms use the consistent T_<source>[_time]_<target>[_time] snake_case convention (no camelCase, no stray sensorend / sensorRef).
  • All callers updated: ncore/impl/data/v4/compat.py, the nuScenes and Waymo converters, structured_lidar_model.py, and the ncore_project_pc_to_img / ncore_evaluate_lidar_model tools.

Testing

bazel test //ncore/impl/common:test_3_11 \
            //ncore/impl/data/v4:pytest_compat_3_11 \
            //ncore/impl/sensors:pytest_lidar_3_11

New tests:

  • test_idempotence_custom_reference_timestamp -- compensate to a non-default reference (frame start) and decompensate back recovers the original points.
  • test_default_reference_matches_explicit_end_of_frame -- omitting reference_timestamp_us equals passing frame_end explicitly.

The existing test_idempotence still passes, as do the compat and lidar sensor tests (which consume the renamed result fields). All of //ncore/... and the affected //tools/... packages build and type-lint cleanly.


This unblocks the Argoverse 2 converter (separate PR #145), which will call motion_decompensate_points(..., reference_timestamp_us=sweep_ts).

@copy-pr-bot

copy-pr-bot Bot commented Jun 10, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@janickm

janickm commented Jun 10, 2026

Copy link
Copy Markdown
Collaborator Author

This supersedes janickm#1 and addresses all the review feedback there:

  • xyz_sensorend param -> renamed to xyz_reftime in motion_decompensate_points.
  • MotionCompensationResult "sensorend" fields -> renamed to xyz_s_reftime / xyz_e_reftime, and the struct now carries reference_timestamp_us unconditionally.
  • camelCase T_world_sensorRef etc. -> all internal transforms use the T_<source>[_time]_<target>[_time] snake_case convention.
  • T_from_to frame-name consistency -> reworked so names match the actual composition (e.g. T_sensor_pointtime_sensor_reftime = T_world_sensor_pointtime @ T_sensor_reftime_world).
  • stray "sensorend" in code + comments -> swept the whole repo; the only legitimate remaining T_world_sensor_end usages are the unrelated end-of-frame poses in the camera/lidar/common sensor modules.
  • "make this free of the argoverse stuff" -> this branch is off main with no Argoverse 2 changes. The AV2 converter (PR feat(argoverse2): add Argoverse 2 Sensor Dataset to NCore V4 converter #145) goes on top afterwards and will call motion_decompensate_points(..., reference_timestamp_us=sweep_ts).

Targets main per the merge-first plan.

@janickm janickm marked this pull request as ready for review June 10, 2026 14:59
@janickm janickm self-assigned this Jun 10, 2026
…de)compensation

Add an optional `reference_timestamp_us` parameter to
`MotionCompensator.motion_compensate_points` and
`motion_decompensate_points`. It defaults to `frame_end_timestamp_us`,
preserving the existing end-of-frame behavior used by the existing
converters. Datasets that compensate to a different in-frame reference
(e.g. the start of the frame) can now pass it explicitly.

`motion_decompensate_points` is reimplemented as the exact inverse of
`motion_compensate_points`: per point it evaluates the sensor pose at the
point's own timestamp relative to the reference sensor pose
(`T_sensor_reftime_sensor_pointtime = T_world_sensor_pointtime @
T_sensor_reftime_world`) rather than linearly interpolating the end->start
relative transform. This generalizes to any reference and is at least as
accurate (it uses the real trajectory at each point time).

Naming / API cleanups:
- `MotionCompensationResult` fields renamed from the misleading
  `xyz_s_sensorend` / `xyz_e_sensorend` to `xyz_s_reftime` / `xyz_e_reftime`,
  and the struct now carries `reference_timestamp_us` unconditionally.
- `motion_decompensate_points` parameter `xyz_sensorend` -> `xyz_reftime`.
- Internal transforms use the consistent `T_<source>[_time]_<target>[_time]`
  snake_case convention (no camelCase, no stray `sensorend`/`sensorRef`).
- All callers updated: ncore/impl/data/v4/compat.py, the nuScenes and Waymo
  converters, structured_lidar_model.py, and the ncore_project_pc_to_img /
  ncore_evaluate_lidar_model tools.

Tests: a custom-reference compensate/decompensate round-trip sweeping several
references across the [frame_start, frame_end] range (boundaries plus interior
fractions), and a check that the default reference matches an explicit
end-of-frame reference. The existing idempotence test still passes.
@janickm janickm force-pushed the compensator-reference-time-v2 branch from ce1e5ef to b0556d4 Compare June 10, 2026 15:30
@janickm

janickm commented Jun 10, 2026

Copy link
Copy Markdown
Collaborator Author

/ok to test b0556d4

@janickm janickm enabled auto-merge June 10, 2026 15:31
@janickm janickm added this pull request to the merge queue Jun 10, 2026
Merged via the queue into NVIDIA:main with commit 131e86a Jun 10, 2026
5 checks passed
@janickm janickm deleted the compensator-reference-time-v2 branch June 10, 2026 15:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant