Release by descriptor work

[tech3371]

Change Summary

closes #1478

Overview

Adding support to read manifest file with release information.

File content looks like this:

instrument, data_type, descriptor, release_flag
hit, l1a, *, true
hit, l1b, *, false
hit, ancillary, cal-data, true

Each instrument will have their own manifest.

File changes

Testing

[Copilot]

Pull request overview

Adds support for driving “release” operations from a per-instrument manifest file that specifies releases by (instrument, data_level/data_type, descriptor, release_flag), and refactors “latest version” selection logic to be reusable across release paths.

Changes:

• Replaces exclude-list based releasing with manifest-driven selection (including * wildcards) for science and ancillary releases.
• Introduces a shared SQLAlchemy window-function query builder to select latest-version science files.
• Updates and reshapes release API integration tests to cover the new manifest format and selection behavior.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 8 comments.

File	Description
tests/lambda_endpoints/test_release_api.py	Updates release API tests to use manifest-driven release selection and new helper entry points.
sds_data_manager/lambda_code/SDSCode/database/models.py	Centralizes the “logical file identity” column set for version grouping/indexing.
sds_data_manager/lambda_code/SDSCode/api_lambdas/utils.py	Adds build_latest_version_query() for selecting latest-version science files via a window function.
sds_data_manager/lambda_code/SDSCode/api_lambdas/release_api.py	Implements manifest-driven releasing and refactors latest-version selection for science/ancillary files.

Comments suppressed due to low confidence (2)

sds_data_manager/lambda_code/SDSCode/api_lambdas/release_api.py:142

• latest_ancillary_release docstring says it returns a single-column query for use in an .in_() subquery, but the function actually loads ORM rows (.all()), updates them, and returns a list of AncillaryFiles. This mismatch makes the function contract unclear for callers and future refactors.

    Returns
    -------
    Query
        A query of a single file_path column, for use as an `.in_()`
        subquery so a large release stays a single UPDATE statement.
    """

tests/lambda_endpoints/test_release_api.py:283

• This assertion message still refers to an "Excluded" file, but this scenario is now testing "latest-version only" behavior (v001 should remain unreleased because v002 is the latest). The message should reflect that to avoid confusion.

    assert (
        rows["imap/ancillary/swe/imap_swe_l1b-in-flight-cal_20260420_v001.csv"] is False
    ), "Excluded file should not be released"

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[tmplummer]

I gave this a quick review and found nothing obvious. I'll approve now and come back to this this afternoon when I have more time

[tmplummer]

One minor nit. I think this looks good and it is great to hear that testing in dev went well. 🎉

[tech3371]

I will merge this PR Monday and test again on Monday with list of files on this ticket #1452.