Skip to content

feat(models): add the StreamPETR detector - #9

Draft
vividf wants to merge 1 commit into
feat/streampetr-model-componentsfrom
feat/streampetr-detector
Draft

feat(models): add the StreamPETR detector#9
vividf wants to merge 1 commit into
feat/streampetr-model-componentsfrom
feat/streampetr-detector

Conversation

@vividf

@vividf vividf commented Sep 2, 2026

Copy link
Copy Markdown
Owner

Summary

The StreamPETR detector: query-based head with a temporal memory bank, and the model wrapper with the deployment export ABI. Ported from the reference implementation (exiawsh/StreamPETR) with training parity as the goal.

Changes

  • StreamPETRHead (models/detection3d/heads/streampetr.py):
    • Temporal memory: pre_update_memory aligns the bank into the current ego frame (timestamp/ego-pose bookkeeping), post_update_memory appends the top-k strongest detached proposals; prev_exists resets streams at scene boundaries; pseudo reference points seed cold starts.
    • Geometry-aware position embeddings (LID depth bins, frustum → lidar frame) with a float32 island — kilometer-scale translations exceed bf16 resolution.
    • Denoising queries with group attention masks extended over the propagated queries and temporal memory keys.
    • Hungarian-matched focal + L1 losses over every decoder layer; shared cls/reg branches across layers per the reference.
  • StreamPETRDetectionModel (models/detection3d/streampetr.py): backbone/neck/head composition, grid-mask hook, memory reset on train/eval phase switches, backbone-vs-rest optimizer groups, and the three-module ONNX export (extract_img_feat / position_embedding / pts_head_memory) carrying the five memory tensors through the graph so the runtime manages the stream outside it.

Testing

tests/models/test_streampetr.py: forward/loss/predict shapes, memory recurrence across frames, prev_exists reset behavior, and export-wrapper vs eager-head consistency.

Stack 3/8 — base: feat/streampetr-model-components.

@vividf vividf mentioned this pull request Sep 2, 2026
@vividf
vividf force-pushed the feat/streampetr-model-components branch from cb98ac5 to 847203c Compare September 2, 2026 08:57
@vividf
vividf force-pushed the feat/streampetr-detector branch 2 times, most recently from 523051f to e62ece7 Compare September 2, 2026 09:31
@vividf
vividf force-pushed the feat/streampetr-model-components branch 2 times, most recently from 57c44d2 to c9424dd Compare September 2, 2026 21:02
@vividf
vividf force-pushed the feat/streampetr-detector branch from e62ece7 to b942752 Compare September 2, 2026 21:02
@vividf
vividf marked this pull request as draft September 3, 2026 17:24
@vividf
vividf force-pushed the feat/streampetr-model-components branch from c9424dd to 3359518 Compare September 4, 2026 06:24
@vividf
vividf force-pushed the feat/streampetr-detector branch 2 times, most recently from 52e8791 to df4bf63 Compare September 4, 2026 07:29
@vividf
vividf force-pushed the feat/streampetr-model-components branch from 3359518 to 7bc4dc7 Compare September 4, 2026 07:29
The query-based head with temporal memory (pre/post memory update, top-k
propagation, denoising queries, geometry-aware position embeddings) and the
model wrapper with the three-module ONNX export ABI.

Signed-off-by: vividf <yihsiang.fang@tier4.jp>
@vividf
vividf force-pushed the feat/streampetr-model-components branch from 7bc4dc7 to a005ccd Compare September 4, 2026 08:00
@vividf
vividf force-pushed the feat/streampetr-detector branch from df4bf63 to c8d3fbd Compare September 4, 2026 08:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant