Skip to content

feat(streampetr): add auxiliary 2D head, partial-ignore, and DDP loss normalization - #5

Draft
vividf wants to merge 1 commit into
feat/streampetr-2d-annotationsfrom
feat/streampetr-aux-2d-head
Draft

feat(streampetr): add auxiliary 2D head, partial-ignore, and DDP loss normalization#5
vividf wants to merge 1 commit into
feat/streampetr-2d-annotationsfrom
feat/streampetr-aux-2d-head

Conversation

@vividf

@vividf vividf commented Sep 2, 2026

Copy link
Copy Markdown
Owner

Summary

The training-quality core of the stack: auxiliary 2D supervision, partial-ignore for sparsely annotated classes, and DDP-correct loss normalization.

Changes

  • FocalHead2D (models/detection3d/heads/focal2d.py): Focal-PETR-style dense head over the stride-16 neck features — per-token class scores, centerness, LTRB boxes, and projected 3D centers, supervised by quality focal, weighted L1, GIoU, center L1, and Gaussian-heatmap centerness losses. Cost/loss weights follow the reference recipe (2/5/2/10 + 1); the default HungarianAssigner2D mirrors the same cost weights. Loss-only: inference and deployment never run it. The reference's top-k token pruning is intentionally not ported.
  • CPFPN (models/common/necks/cp_fpn.py): the reference StreamPETR neck (1×1 laterals, nearest top-down, 3×3 refine on level 0), with mm-compatible parameter names.
  • Partial-ignore (models/detection3d/partial_ignore.py): traffic_cone/barrier handled purely as classification-column weights — on an un-annotated frame, every query/token drops only the ignored class columns. Structurally unable to perturb Hungarian matching, bbox targets, or the normalization factor. Both heads share one masking helper and the same semantics; missing or mismatched status flags raise instead of silently training the ignored classes as background.
  • DDP loss normalization (task_modules/streaming.py + both heads): positive counts are averaged across ranks (mmdetection's reduce_mean), so the sharded loss equals the single-batch equivalent regardless of GPU count. Every collective sits on a rank-uniform code path — a rank whose batch has no ground truth still participates in the DN count collective with a zero contribution, so mixed-GT steps cannot deadlock.

Testing

Real multi-process gloo tests (test_streampetr_loss_distributed.py): a sharding-invariance property test (DDP-averaged sharded loss == combined-batch loss), the mixed-GT deadlock regression, and reduce-mean semantics. Partial-ignore has unit tests down to exact masked rows/columns and an end-to-end head.loss wiring test.

Stack 7/8 — base: feat/streampetr-2d-annotations.

@vividf
vividf force-pushed the feat/streampetr-aux-2d-head branch from 42434be to a40d08b Compare September 2, 2026 08:43
@vividf
vividf force-pushed the feat/streampetr-2d-annotations branch from 6fc9075 to f94c6cb Compare September 2, 2026 08:43
@vividf
vividf force-pushed the feat/streampetr-aux-2d-head branch from a40d08b to 95a151c Compare September 2, 2026 08:57
@vividf
vividf force-pushed the feat/streampetr-2d-annotations branch 2 times, most recently from 262f7ee to 4e78b19 Compare September 2, 2026 09:31
@vividf
vividf force-pushed the feat/streampetr-aux-2d-head branch from 95a151c to 7635f26 Compare September 2, 2026 09:31
@vividf
vividf force-pushed the feat/streampetr-2d-annotations branch from 4e78b19 to d6579bd Compare September 2, 2026 21:02
@vividf
vividf force-pushed the feat/streampetr-aux-2d-head branch from 7635f26 to 6065857 Compare September 2, 2026 21:02
@vividf
vividf marked this pull request as draft September 3, 2026 17:24
@vividf
vividf force-pushed the feat/streampetr-2d-annotations branch from d6579bd to 7ae5816 Compare September 4, 2026 06:24
@vividf
vividf force-pushed the feat/streampetr-aux-2d-head branch from 6065857 to abdccfa Compare September 4, 2026 06:24
@vividf
vividf force-pushed the feat/streampetr-2d-annotations branch from 7ae5816 to a750d87 Compare September 4, 2026 07:29
@vividf
vividf force-pushed the feat/streampetr-aux-2d-head branch from abdccfa to 7b21f5f Compare September 4, 2026 07:29
… normalization

Focal-PETR-style FocalHead2D supervising the image features during
training, CPFPN as the reference neck, traffic_cone/barrier partial-ignore
as classification-column weights shared by both heads, and cross-rank
positive-count normalization (mmdetection reduce_mean) with real
multi-process regression tests.

Signed-off-by: vividf <yihsiang.fang@tier4.jp>
@vividf
vividf force-pushed the feat/streampetr-2d-annotations branch from a750d87 to 2bdd538 Compare September 4, 2026 08:00
@vividf
vividf force-pushed the feat/streampetr-aux-2d-head branch from 7b21f5f to 26158be Compare September 4, 2026 08:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant