RCAL-1380: Populate SFD-estimated extinction in catalogs.

[mairanteodoro]

Resolves RCAL-1380

This PR adds support for populating a new dust_ebv extinction column in Roman source and multiband catalogs using SFD dust maps resolved through CRDS. It updates the catalog schema and source-catalog generation logic, adds tests and regression checks for the new field.

Note

This adjusts CI/tox setup to use temporary git-based dependency installs needed for the change. We need to revert it before merging this PR. Done (unit tests will fail until the RAD PR is merged).

Regression tests

• https://github.com/spacetelescope/RegressionTests/actions/runs/25570563604 (05/08/26; all tests passing except for one that needs to have the truth file updated later).

Tasks

• request a review from someone specific, to avoid making the maintainers review every PR
• add a build milestone, i.e. 24Q4_B15 (use the latest build if not sure)
• Does this PR change user-facing code / API? (if not, label with no-changelog-entry-needed)
  • write news fragment(s) in changes/: echo "changed something" > changes/<PR#>.<changetype>.rst (see changelog readme for instructions)
    • if your change breaks existing functionality, also add a changes/<PR#>.breaking.rst news fragment
  • update or add relevant tests
  • update relevant docstrings and / or docs/ page
  • start a regression test and include a link to the running job (click here for instructions)
    • Do truth files need to be updated ("okified")?
      • after the reviewer has approved these changes, run okify_regtests to update the truth files
• if a JIRA ticket exists, make sure it is resolved properly

[codecov]

Codecov Report

❌ Patch coverage is 98.11321% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 80.70%. Comparing base (97f0385) to head (c56d8f1).

Files with missing lines	Patch %	Lines
romancal/source_catalog/_source_catalog.py	98.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2313      +/-   ##
==========================================
+ Coverage   80.60%   80.70%   +0.09%     
==========================================
  Files         155      155              
  Lines        9409     9462      +53     
==========================================
+ Hits         7584     7636      +52     
- Misses       1825     1826       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[schlafly]

Looks good to me. One minor comment inline. Thanks Mairan!

[schlafly]

The oldestdeps python 3.11 tests were failing with NoneType in various source catalog steps. We probably need a rad version requirement to do this; we're saying that's okay here since it should pass in the regtests with the pyproject / roman_datamodels pointing to the right rad version.

[zacharyburnett]

@zacharyburnett and I had a quick chat about this issue and he will draft a PR to try and solve this issue of RAD and RDM depending on each other while being two separate packages. For now, patching tox.ini/tests.yml is the temporary solution that I was able to come up with. At least it shows that the guard won't be needed when the RAD changes are in.

I drafted spacetelescope/roman_datamodels#676 to see what making a monorepo of rad and roman_datamodels would look like, but from talking with @braingram and @WilliamJamieson I don't think that's a viable solution either unfortunately. For now we'll just have to work with this dependency chain as-is

[schlafly]

Thanks, approved. I'll let you back out the pyproject toml changes and merge when the corresponding rad PR is in. We'll also need to okify the differences; I'll let you handle that when you merge?

[mairanteodoro]

We'll also need to okify the differences; I'll let you handle that when you merge?

Sounds good.

[braingram]

@zacharyburnett and I had a quick chat about this issue and he will draft a PR to try and solve this issue of RAD and RDM depending on each other while being two separate packages. For now, patching tox.ini/tests.yml is the temporary solution that I was able to come up with. At least it shows that the guard won't be needed when the RAD changes are in.

I drafted spacetelescope/roman_datamodels#676 to see what making a monorepo of rad and roman_datamodels would look like, but from talking with @braingram and @WilliamJamieson I don't think that's a viable solution either unfortunately. For now we'll just have to work with this dependency chain as-is

The unit tests are failing with a TypeError because:

romancal/romancal/source_catalog/_source_catalog.py

Line 652 in 1937ff3

catalog[column].info.description = definition["description"]

is attempting to get the "description" where definition is None. Merging spacetelescope/rad#870 won't fix this as:

• romancal is requiring the released rdm version:

  romancal/pyproject.toml

  Line 28 in 1937ff3

  "roman_datamodels>=0.31.0, <0.32",

• the development version of rdm is using the released version of rad: https://github.com/spacetelescope/roman_datamodels/blob/210ae0eb8d44d7e16647c30176eb29e4f68c7d01/pyproject.toml#L19

We will need:

• this PR to be updated to point to rdm main
• a rdm PR setting the required rad version to rad main

@mairanteodoro would you update the rdm requirement in this PR? I'll open a rdm PR setting the rad requirement to main.

[mairanteodoro]

Merging spacetelescope/rad#870 won't fix this as:

• romancal is requiring the released rdm version:

  romancal/pyproject.toml

  Line 28 in 1937ff3

  "roman_datamodels>=0.31.0, <0.32",

• the development version of rdm is using the released version of rad: https://github.com/spacetelescope/roman_datamodels/blob/210ae0eb8d44d7e16647c30176eb29e4f68c7d01/pyproject.toml#L19

We will need:

• this PR to be updated to point to rdm main
• a rdm PR setting the required rad version to rad main

@mairanteodoro would you update the rdm requirement in this PR? I'll open a rdm PR setting the rad requirement to main.

Yep, I can do that. I assumed that we would have a released version soon, so I didn't think of changing the requirements.