feat(streampetr): project 2D annotations and fold augmentations into ego poses - #4
Draft
vividf wants to merge 1 commit into
Draft
feat(streampetr): project 2D annotations and fold augmentations into ego poses#4vividf wants to merge 1 commit into
vividf wants to merge 1 commit into
Conversation
vividf
force-pushed
the
feat/streampetr-training-infra
branch
from
September 2, 2026 08:43
ce301f5 to
6579398
Compare
vividf
force-pushed
the
feat/streampetr-2d-annotations
branch
from
September 2, 2026 08:43
6fc9075 to
f94c6cb
Compare
vividf
force-pushed
the
feat/streampetr-training-infra
branch
from
September 2, 2026 08:57
6579398 to
57c4562
Compare
vividf
force-pushed
the
feat/streampetr-2d-annotations
branch
2 times, most recently
from
September 2, 2026 09:31
262f7ee to
4e78b19
Compare
vividf
force-pushed
the
feat/streampetr-training-infra
branch
from
September 2, 2026 09:31
57c4562 to
428e760
Compare
vividf
force-pushed
the
feat/streampetr-2d-annotations
branch
from
September 2, 2026 21:02
4e78b19 to
d6579bd
Compare
vividf
force-pushed
the
feat/streampetr-training-infra
branch
from
September 2, 2026 21:02
428e760 to
2228774
Compare
vividf
marked this pull request as draft
September 3, 2026 17:24
vividf
force-pushed
the
feat/streampetr-training-infra
branch
from
September 4, 2026 06:24
2228774 to
4a2e1bf
Compare
vividf
force-pushed
the
feat/streampetr-2d-annotations
branch
from
September 4, 2026 06:24
d6579bd to
7ae5816
Compare
vividf
force-pushed
the
feat/streampetr-training-infra
branch
from
September 4, 2026 07:29
4a2e1bf to
5af526a
Compare
vividf
force-pushed
the
feat/streampetr-2d-annotations
branch
from
September 4, 2026 07:29
7ae5816 to
a750d87
Compare
This was referenced Sep 4, 2026
…ego poses LoadAnnotations2DFromBoxes3D projects the augmented 3D boxes onto every camera for the auxiliary 2D head; the camera geometric augmentations fold their transforms into the ego poses so the temporal memory warp stays consistent; train-time camera-order shuffling follows the reference recipe. Signed-off-by: vividf <yihsiang.fang@tier4.jp>
vividf
force-pushed
the
feat/streampetr-training-infra
branch
from
September 4, 2026 08:00
5af526a to
7a02cde
Compare
vividf
force-pushed
the
feat/streampetr-2d-annotations
branch
from
September 4, 2026 08:00
a750d87 to
2bdd538
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Everything the auxiliary 2D head needs from the data side, plus augmentation-consistent ego poses for the temporal memory.
Changes
LoadAnnotations2DFromBoxes3D(transforms/camera/annotations2d.py): projects the (already augmented) 3D gravity-center boxes onto every camera and emits per-cameragt_bboxes_2d/centers_2d/gt_labels_2d. Vectorized over boxes; corners behind the camera are masked out of the 2D extent; a box is kept only when its projected center lands inside the image (matching the reference recipe — a clamped center could fall outside its own clipped box and distort the center-based assignment); boxes are clipped to the image and zero-area slivers dropped. Runs after all geometric augmentations so the projection matrices and pixels agree.transforms/geometry3d.py+ camera / camera-lidar geometry): a scene augmentationAupdatesego_pose' = ego_pose @ A⁻¹(and the inverse accordingly), keeping lidar→global consistent — exactly what StreamPETR's memory warp (ego_pose_inv(t) @ ego_pose(t-1)) needs under train-time augmentation.transforms/camera/loading.py): per-frame train-time shuffle following the referenceshuffle_cameras=True; images, intrinsics, and extrinsics are reordered together by construction.Testing
Projective edge cases are unit-tested (box behind the camera, center leaving the image, partial visibility/clipping, multi-camera assignment, empty and malformed GT), plus ego-pose round-trip tests and a velocity-scaling regression test.
Stack 6/8 — base:
feat/streampetr-training-infra.