Skip to content

fix: compute Experiment start/end time as union across all devices - #137

Merged
schewskone merged 26 commits into
sensorium-competition:mainfrom
binary69:fix/experiment-global-time-range
Mar 20, 2026
Merged

fix: compute Experiment start/end time as union across all devices#137
schewskone merged 26 commits into
sensorium-competition:mainfrom
binary69:fix/experiment-global-time-range

Conversation

@binary69

@binary69 binary69 commented Mar 16, 2026

Copy link
Copy Markdown
Collaborator

Closes #134

Problem
In _load_devices(), start_time and end_time were assigned with plain assignment inside a loop, so they reflected only the last loaded device instead of the global range across all devices.

Fix

changes experiment.py
Updated _load_devices() to use min/max to track the earliest start and latest end across all devices.
Devices with None, NaN, inf, -inf timestamps are skipped with warning, not silently corrupting
ValueError if no device with valid time ranges are loaded
ValueError if compute start_time > end_time

New Tests

create_experiment.py
contextmanager fixture with configurable signals, sampling rate, and override_meta for invalid metadata testing
builds per-device interpolation config dicts

test_experiment.py
parametrized across 1, 2, and 3 device scenarios and signal counts - (5+1)*2 = 12 runs
parametrized across 9 invalid metadata cases - 9 runs
Verifies that start_time > end_time raises ValueError - 1 run
verifies that an invalid device is excluded and a warning is logged, while a co-loaded valid device initialises the experiment correctly. - 9 runs

All 31 tests are passing.

Copilot AI review requested due to automatic review settings March 16, 2026 10:12
@gitnotebooks

gitnotebooks Bot commented Mar 16, 2026

Copy link
Copy Markdown

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes Experiment.start_time / end_time so they reflect the global time span across all loaded devices (earliest start + latest end), matching the class docstring and avoiding “last device wins” behavior.

Changes:

  • Update Experiment._load_devices() to compute start_time/end_time via min/max across devices.
  • Add new tests covering 1-, 2-, and 3-device scenarios for the computed experiment time span.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
experanto/experiment.py Compute experiment start/end time as min/max across loaded devices.
tests/test_experiment.py New regression tests for experiment start/end time spanning multiple devices.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread tests/test_experiment.py Outdated
@codecov

codecov Bot commented Mar 16, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@pollytur pollytur left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We typically separate the test data creation and the actual testing - please do the same here :)

Ideally look at PR #113 and name the files like create_experiment.py‎ and test_experiment.py such that we could merge your tests together later

@binary69

Copy link
Copy Markdown
Collaborator Author

Thanks for the feedback! I'll restructure the tests following the pattern in PR #113 — separating the data creation helpers into create_experiment.py and keeping the tests in test_experiment.py. Will push the update shortly.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes Experiment.start_time/end_time so they reflect the global time span across all loaded devices (earliest start + latest end), instead of only the last device processed.

Changes:

  • Update Experiment._load_devices() to track start_time via min(...) and end_time via max(...) across devices.
  • Add new tests covering single-, two-, and three-device scenarios for the computed union time range.
  • Add a small test helper to generate multi-device experiment folders under tmp_path.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
experanto/experiment.py Computes experiment-wide start/end as min/max across device ranges.
tests/test_experiment.py Adds regression tests validating union start/end time behavior.
tests/create_experiment.py Adds helpers to build multi-device experiment fixtures for tests.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread experanto/experiment.py Outdated
Comment thread tests/test_experiment.py
Comment thread tests/create_experiment.py Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes Experiment.start_time / end_time so they reflect the union of time ranges across all loaded devices (earliest start, latest end), aligning behavior with the class docstring and preventing “last device wins” behavior.

Changes:

  • Update Experiment._load_devices() to track experiment-wide start_time/end_time via min/max across devices.
  • Add warnings/validation when devices have undefined time bounds (though current placement/logic needs adjustment).
  • Add unit tests covering single-, two-, and three-device unions using new test fixtures.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
experanto/experiment.py Compute experiment time range as min/max across devices; adds (currently flawed) validation/warnings for missing/invalid ranges.
tests/test_experiment.py Adds tests asserting Experiment.start_time/end_time represent the union across devices.
tests/create_experiment.py Adds helpers to generate minimal on-disk experiment/device fixtures for the new tests.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread experanto/experiment.py Outdated
Comment thread experanto/experiment.py Outdated
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes Experiment.start_time / end_time so they reflect the global time span across all loaded devices (earliest start + latest end), rather than being overwritten by the last device processed.

Changes:

  • Update Experiment._load_devices() to compute experiment-wide start_time/end_time via min/max across devices (skipping devices with undefined bounds).
  • Add a small post-load validation/warning block for missing devices / invalid computed range.
  • Add new tests covering single-, two-, and three-device union behavior (with new test fixtures for building a minimal experiment folder).

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
experanto/experiment.py Compute experiment-wide time range across devices and add validation/warnings.
tests/test_experiment.py Add regression tests for experiment-wide start/end time union behavior.
tests/create_experiment.py Add helpers to create minimal on-disk device folders + modality config for tests.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread experanto/experiment.py
Comment thread experanto/experiment.py Outdated
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@binary69
binary69 requested a review from Copilot March 16, 2026 14:28
@binary69
binary69 requested review from pollytur and schewskone March 18, 2026 09:08
Comment thread tests/create_experiment.py Outdated
@binary69

Copy link
Copy Markdown
Collaborator Author

Addressed the changes abovementioned and expanded test coverage with additional edge cases, including parametrized invalid metadata tests, a dedicated inverted range test, and caplog assertion to verify warnings are emitted when invalid devices are skipped. All 31 tests are passing. Looking forward to your feedback :)

@pollytur

Copy link
Copy Markdown
Contributor

@binary69 could we please merge main into the current branch?

Comment thread tests/create_experiment.py
@binary69
binary69 force-pushed the fix/experiment-global-time-range branch from 8408995 to 863e52c Compare March 19, 2026 12:25
Comment thread tests/test_experiment.py
Comment thread tests/test_experiment.py Outdated
@binary69

Copy link
Copy Markdown
Collaborator Author

Thanks @pollytur :) will be updating soon

Comment thread tests/test_experiment.py Outdated
@binary69

Copy link
Copy Markdown
Collaborator Author

Made the suggested changes, happy to get more feedback :)

@pollytur pollytur left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!
@schewskone please take a look as well :)

@schewskone schewskone left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM as well. I'm not 100% sure about randomness in parameter generalization for testing but I also don't mind.
Thanks for the contribution

@schewskone
schewskone merged commit d61d9b8 into sensorium-competition:main Mar 20, 2026
@binary69

Copy link
Copy Markdown
Collaborator Author

Thank you so much, and I really appreciate the feedback. On the randomness concern, we can either use seeded randomness or fixed non-default valued to avoid potential flakiness. Looking forward to contribute more :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Experiment.start_time and end_time reflect only the last loaded device, not the global range in experiment.py

4 participants