Skip to content

Fix memory copies getz#120

Merged
steven-murray merged 12 commits intomainfrom
fix-memory-copies-getz
May 7, 2026
Merged

Fix memory copies getz#120
steven-murray merged 12 commits intomainfrom
fix-memory-copies-getz

Conversation

@steven-murray
Copy link
Copy Markdown
Contributor

Stacks on #119

Fixes an issue where memory copies were made when using multiple beams.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 1, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.88%. Comparing base (f6f93ee) to head (f5cfd85).
⚠️ Report is 9 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #120      +/-   ##
==========================================
+ Coverage   98.46%   98.88%   +0.42%     
==========================================
  Files          22       22              
  Lines         975      989      +14     
  Branches       98      103       +5     
==========================================
+ Hits          960      978      +18     
+ Misses          8        6       -2     
+ Partials        7        5       -2     
Flag Coverage Δ
unittests 78.46% <64.70%> (+0.20%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses excess memory usage when simulating with per-antenna beam selection by avoiding fancy-indexing copies in Z-matrix formation, and extends the chunk-sizing logic with a configurable free-memory safety margin.

Changes:

  • Avoid fancy-indexing beam selection in ZMatrixCalc to prevent large temporary allocations when beam_idx is used.
  • Add a memory_buffer parameter to CPU/GPU simulate() and propagate it into chunk estimation utilities.
  • Adjust chunk-estimation thresholds to account for memory_buffer, and tweak docstring composition in the GPU entrypoint.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 8 comments.

File Description
src/matvis/gpu/gpu.py Adds memory_buffer to GPU simulate and passes it into chunk sizing.
src/matvis/cpu/cpu.py Adds memory_buffer to CPU simulate and documents the new parameter.
src/matvis/core/getz.py Replaces beam fancy-indexing with a per-antenna loop to avoid memory copies.
src/matvis/_utils.py Extends chunk-sizing helpers with memory_buffer and uses it in the stopping criterion.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/matvis/cpu/cpu.py Outdated
Comment thread src/matvis/cpu/cpu.py Outdated
Comment thread src/matvis/cpu/cpu.py
Comment thread src/matvis/_utils.py Outdated
Comment thread src/matvis/_utils.py
Comment thread src/matvis/_utils.py
Comment thread src/matvis/_utils.py
Comment thread src/matvis/gpu/gpu.py Outdated
@steven-murray
Copy link
Copy Markdown
Contributor Author

@copilot please re-review given the rebase

…and GPU simulate

Agent-Logs-Url: https://github.com/HERA-Team/matvis/sessions/b7466356-8902-46eb-b67c-a9b2b60b6fdf

Co-authored-by: steven-murray <1272030+steven-murray@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

Copilot AI commented May 6, 2026

@copilot please re-review given the rebase

Re-reviewed after the rebase. The remaining open items were:

  1. Missing input validation for source_buffer and memory_buffer in cpu/cpu.py
  2. Incomplete validation in gpu/gpu.py (only checked source_buffer > 1.0, missing lower-bound and memory_buffer validation)

Both are now addressed in commit 945ac01 — both functions now raise ValueError when source_buffer or memory_buffer is outside (0, 1], consistent with the validation already in _utils.get_required_chunks. The previously-resolved items (_utils.py validation, docstring fixes, and memory_buffer tests) were already in place from prior commits.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • data.astropy.org
    • Triggering command: /usr/bin/python3 python3 -m pytest tests/test_utils.py tests/test_matvis_cpu.py -x -v _TOKEN&#34;; }; f (dns block)
  • www.astropy.org
    • Triggering command: /usr/bin/python3 python3 -m pytest tests/test_utils.py tests/test_matvis_cpu.py -x -v _TOKEN&#34;; }; f (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copy link
Copy Markdown
Contributor

@tyler-a-cox tyler-a-cox left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is all looking good to me. Thanks @steven-murray!

@steven-murray steven-murray merged commit 6eafd90 into main May 7, 2026
15 checks passed
@steven-murray steven-murray deleted the fix-memory-copies-getz branch May 7, 2026 07:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants