Skip to content

Functional outputs & regressor reuse (Stage 5 of #301)#315

Merged
nx10 merged 11 commits into
mainfrom
refactor/longitudinal-stage-5
Apr 16, 2026
Merged

Functional outputs & regressor reuse (Stage 5 of #301)#315
nx10 merged 11 commits into
mainfrom
refactor/longitudinal-stage-5

Conversation

@nx10
Copy link
Copy Markdown
Contributor

@nx10 nx10 commented Apr 16, 2026

Summary

  • Split cross-sectional FunctionalOutputs.regressor_file into regressor_file (raw, unfiltered) and bpf_regressor_file (bandpass-filtered, export only), so the longitudinal pipeline can reuse raw regressors without recomputation
  • FunctionalLongOutputs gains regressed_bold and cleaned_bold dicts keyed by regressor strategy; bold_mask is now mandatory
  • longitudinal_process accepts regressor_files: dict[str, Path] and iterates its keys to re-run apply_regression + apply_regression_bandpass on the warped BOLD (no redundant regressor_set parameter; the resolve layer filters to the requested strategies)
  • bids/longitudinal/functional.py exports per-regressor desc-regressed and desc-preproc BOLD with reg-<strategy> entity
  • rbc longitudinal functional gains --regressor with the same choices as cross-sectional (36-parameter, aCompCor)
  • Bugfix: Filters.apply() task filter (--task) was dropping anat/dwi/fmap rows because task=null for non-functional files fails an equality check. Fixed to pl.col("task").is_null() | (pl.col("task") == value). Affects rbc functional --task and rbc all --task globally, not just longitudinal.
  • Ports regression reuse logic from closed PR Additional outputs to longitudinal functional workflow Fix #190 #282

Closes the Stage 5 checkbox on #301. Ports work from #282.

Test plan

  • Unit tests for resolve_longitudinal_func with single/multiple regressors (7 tests)
  • Unit tests for export_longitudinal_func file counts and entity validation (3 tests)
  • Existing unit tests updated for bpf_regressor_file field and mandatory bold_mask
  • CLI test for --regressor round-trip through FunctionalLongArgs
  • Filters.apply() task filter unit tests updated: anat rows preserved, combined filter count corrected
  • Tier-2 integration test: rbc functional then rbc longitudinal functional end-to-end on ds000114 sub-01 ses-test via subprocess with docker; asserts expected BIDS tree, non-degenerate variance, binary mask
  • All CI checks green (unit x3 platforms, integration with podman, ruff, mypy, codegen, deptry)

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 16, 2026

Coverage

Tests Skipped Failures Errors Time
781 0 💤 0 ❌ 0 🔥 11.519s ⏱️

nx10 and others added 4 commits April 16, 2026 14:04
regressor_file now carries the raw (unfiltered) regressor .1D files as
computed from native-space BOLD.  The new bpf_regressor_file field
holds the bandpass-filtered version that 3dTproject actually applied,
exported with desc-<strategy>Filtered for provenance.

This lets the longitudinal pipeline reuse the raw regressors without
recomputation, matching the cross-sectional principle: one regressor
computation per run, applied in each target space.

Port of the regressor-split idea from #282.

Co-authored-by: Janhavi Pillai <janhavi.pillai@gmail.com>
FunctionalLongOutputs gains regressed_bold and cleaned_bold dicts
keyed by regressor strategy.  longitudinal_process now accepts
regressor_files (raw .1D from cross-sectional) and re-runs
apply_regression + apply_regression_bandpass on the warped BOLD.
No regressor recomputation -- same matrix, different target space.

bold_mask is now mandatory (was Path | None).

BIDS export writes per-regressor desc-regressed and desc-preproc BOLD
with reg-<strategy> entity.  resolve_longitudinal_func resolves the
raw regressor files per strategy.

CLI gains --regressor with the same choices as cross-sectional
(36-parameter, aCompCor).

Port of the longitudinal regression logic from #282.

Co-authored-by: Janhavi Pillai <janhavi.pillai@gmail.com>
Unit tests: resolve_longitudinal_func with single/multiple regressors,
missing regressor raises, mandatory bold_mask; export file counts and
reg-<strategy> entity validation.

Tier-2 integration: apply_regression and apply_regression_bandpass on
warped BOLD with raw regressors produce non-degenerate outputs.

Tier-4 full_pipeline: longitudinal functional outputs exist, regressed
and cleaned BOLD have non-zero variance, bold mask is binary.
… integration

- Drop regressor_set param from longitudinal_process; iterate
  regressor_files.keys() instead.  The resolve layer already filters to
  the requested strategies, so the dict keys ARE the set.  Eliminates
  the caller sync burden.

- Delete tests/full_pipeline/longitudinal/ -- the conftest hacked the
  cross-sectional anat brain as a "longitudinal template", which
  exercises the code path but not the actual transform chain.

- Rewrite tests/integration/longitudinal/test_regression_reuse.py as a
  proper end-to-end CLI test: rbc functional -> rbc longitudinal
  functional on ds000114 sub-01 ses-test via subprocess.  Asserts
  expected BIDS tree, non-degenerate variance, and binary mask.

- Add ds000114_func_derivatives and longitudinal_func_output fixtures
  to the integration conftest so the full cross-sectional -> longitudinal
  chain runs on real multi-session data with docker.
@nx10 nx10 force-pushed the refactor/longitudinal-stage-5 branch from fba822b to a3e7707 Compare April 16, 2026 18:06
nx10 added 7 commits April 16, 2026 14:25
…ssue

ds000114_func_derivatives now depends on ds000114_anat_derivatives
directly instead of longitudinal_template_output.  This ensures
rbc functional runs before the template step writes ses-longitudinal
files into the derivatives dir, avoiding a potential bids2table
discovery issue where the ses-longitudinal anat files interfere with
the functional pipeline's anat resolution.

longitudinal_func_output depends on both ds000114_func_derivatives
and longitudinal_template_output to ensure both cross-sectional
functional outputs and the longitudinal template are present before
rbc longitudinal functional runs.
Filters.apply() applies the --task filter to ALL rows including anat.
Anat files have task=null, so --task fingerfootlips drops them,
causing resolve_functional to fail with FileNotFoundError on
desc-brain_T1w.

Drop --task from both rbc functional and rbc longitudinal functional
fixture calls.  ds000114 only has one task per session so the filter
isn't needed.  Remove debug logging from previous commits.
The --task filter applied pl.col("task") == value to all rows, but
anat/dwi files have task=null, so they were silently dropped.  This
caused rbc functional --task <label> to fail with FileNotFoundError
on desc-brain_T1w for any dataset.

Fix: use pl.col("task").is_null() | (pl.col("task") == value) so
rows without a task entity (anat, dwi, fmap) pass through.

Affects functional and all orchestration; metrics/qc pre-filter to
datatype=func so they were never hit.

Restore --task in integration fixtures now that the fix is in place.
@nx10 nx10 merged commit 5ac4dda into main Apr 16, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant