Skip to content

Fix/postprocess memory oom#656

Draft
lixiaolei1982 wants to merge 1 commit intoroboflow:developfrom
lixiaolei1982:fix/postprocess-memory-oom
Draft

Fix/postprocess memory oom#656
lixiaolei1982 wants to merge 1 commit intoroboflow:developfrom
lixiaolei1982:fix/postprocess-memory-oom

Conversation

@lixiaolei1982
Copy link
Contributor

Problem:

  • PostProcess fails with CUDA out-of-memory when processing sparse_forward() outputs
  • Boolean mask conversion creates large tensors on GPU

Solution:

  • Implement OptimizedPostProcess class
  • Add batch processing for mask conversion (configurable batch size)
  • Enable early CPU offloading to reduce GPU memory pressure
  • Support automatic mixed precision for further optimization

Changes:

  • Added optimized_postprocess.py with memory-efficient implementation
  • Modified init.py to export the new class (if needed)

Testing:

  • Resolves OOM on 11GB GPU with 1024x1024 masks
  • Compatible with both forward() and sparse_forward()
  • Maintains identical output format"

Problem:
- PostProcess fails with CUDA out-of-memory when processing sparse_forward() outputs
- Boolean mask conversion creates large tensors on GPU

Solution:
- Implement OptimizedPostProcess class
- Add batch processing for mask conversion (configurable batch size)
- Enable early CPU offloading to reduce GPU memory pressure
- Support automatic mixed precision for further optimization

Changes:
- Added optimized_postprocess.py with memory-efficient implementation
- Modified __init__.py to export the new class (if needed)

Testing:
- Resolves OOM on 11GB GPU with 1024x1024 masks
- Compatible with both forward() and sparse_forward()
- Maintains identical output format
@Borda Borda force-pushed the fix/postprocess-memory-oom branch from 86cd7de to 6c0c78d Compare February 10, 2026 16:50
@Borda Borda marked this pull request as draft February 10, 2026 16:51
@codecov
Copy link

codecov bot commented Feb 10, 2026

Codecov Report

❌ Patch coverage is 24.56140% with 43 lines in your changes missing coverage. Please review.
✅ Project coverage is 52%. Comparing base (cfeba5d) to head (6c0c78d).
⚠️ Report is 6 commits behind head on develop.

❌ Your patch check has failed because the patch coverage (25%) is below the target coverage (95%). You can increase the patch coverage or adjust the target coverage.
❌ Your project check has failed because the head coverage (52%) is below the target coverage (95%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files
@@          Coverage Diff           @@
##           develop   #656   +/-   ##
======================================
- Coverage       52%    52%   -0%     
======================================
  Files           48     48           
  Lines         6350   6389   +39     
======================================
+ Hits          3311   3314    +3     
- Misses        3039   3075   +36     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Borda Borda force-pushed the develop branch 4 times, most recently from 60b16c1 to 523f9df Compare February 14, 2026 06:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

Comments