Skip to content

Allow use of wildcards in definition of seismic observations/responses - #14118

Merged
achaikou merged 4 commits into
equinor:mainfrom
achaikou:seismic_wildcard
Aug 19, 2026
Merged

Allow use of wildcards in definition of seismic observations/responses#14118
achaikou merged 4 commits into
equinor:mainfrom
achaikou:seismic_wildcard

Conversation

@achaikou

@achaikou achaikou commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Issue
Resolves #13913

Approach
It has been confirmed that we want wildcards, not regex.

  • PR title captures the intent of the changes, and is fitting for release notes.
  • Added appropriate release note label
  • Commit history is consistent and clean, in line with the contribution guidelines.
  • Make sure unit tests pass locally after every commit (git rebase -i main --exec 'just rapid-tests')

When applicable

  • When screenshots are changed: Review screenshot-PR in ert-testdata,
    merge screenshot-PR in ert-testdata before merging this PR.
  • When there are user facing changes: Updated documentation
  • New behavior or changes to existing untested code: Ensured that unit tests are added (See Ground Rules).
  • Large PR: Prepare changes in small commits for more convenient review
  • Bug fix: Add regression test for the bug
  • Bug fix: Add backport label to latest release (format: 'backport release-branch-name')

@achaikou achaikou added the release-notes:unreleased-feature-changes PR with changes to a feature which is not yet released. Not for introduction of new features! label Aug 6, 2026
@achaikou
achaikou requested a lite review from Copilot August 6, 2026 14:36

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds support for glob-style wildcards when defining seismic observations and seismic response input files, reducing configuration duplication for families of sim2seis-generated CSVs. The change centralizes shared seismic utilities (tolerance checks, response/observation alignment, and pattern-to-filepath resolution) and updates config + GUI/storage call sites accordingly.

Changes:

  • Introduce SeismicData utilities, including filename-only wildcard expansion and shared tolerance/location alignment helpers.
  • Update seismic response/observation loading paths to support wildcard patterns and reuse shared utilities.
  • Extend/adjust unit tests to cover wildcard expansion and multi-realization visualization behavior.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
tests/ert/unit_tests/gui/ertwidgets/test_ensemble_widget.py Refactors response-plot assertions and updates test to cover multiple seismic responses.
tests/ert/unit_tests/config/test_seismic_config.py Updates tests to use real temp files and adds coverage for wildcard patterns in SEISMIC response inputs.
tests/ert/unit_tests/config/test_observation_declaration.py Adds coverage for wildcard patterns (and literal metacharacters) in seismic observation CSV paths.
src/ert/storage/local_ensemble.py Switches seismic response/observation alignment helper to SeismicData.
src/ert/gui/tools/manage_experiments/ensemble_widget.py Switches seismic response/observation alignment helper to SeismicData.
src/ert/config/seismic_config.py Uses wildcard expansion for seismic response file discovery and shared tolerance logic.
src/ert/config/_reservoir_data_utils.py Adds SeismicData utilities (pattern resolution, KDTree proximity checks, and alignment helper).
src/ert/config/_observations.py Enables wildcard expansion for seismic observation CSV loading via SeismicData.resolve_pattern_filepaths.
Suppressed comments (1)

src/ert/config/_reservoir_data_utils.py:128

  • resolve_pattern_filepaths() returns matches in filesystem iteration order (Path.iterdir()), which is not guaranteed to be deterministic. Sorting the matched paths (e.g., by filename) would make response/observation key resolution stable across platforms and runs.
        compiled_pattern = re.compile(fnmatch.translate(filename_pattern))
        matching_paths = [
            path
            for path in search_dir.iterdir()
            if path.is_file() and compiled_pattern.fullmatch(path.name)
        ]

Comment thread src/ert/config/_reservoir_data_utils.py
Comment thread src/ert/config/seismic_config.py
@codecov-commenter

codecov-commenter commented Aug 6, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.92%. Comparing base (47757c5) to head (84fdd0d).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #14118      +/-   ##
==========================================
- Coverage   91.93%   91.92%   -0.01%     
==========================================
  Files         482      483       +1     
  Lines       33516    33556      +40     
==========================================
+ Hits        30813    30848      +35     
- Misses       2703     2708       +5     
Flag Coverage Δ
cli-tests 36.47% <32.39%> (-0.01%) ⬇️
fuzz 44.43% <33.80%> (-0.03%) ⬇️
gui-tests 58.79% <33.80%> (-0.05%) ⬇️
performance-and-unit-tests 80.90% <100.00%> (+<0.01%) ⬆️
test 45.96% <33.80%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/ert/config/_observations.py 95.63% <100.00%> (+0.01%) ⬆️
src/ert/config/_reservoir_data_utils.py 100.00% <100.00%> (ø)
src/ert/config/seismic_config.py 100.00% <100.00%> (ø)
...rt/gui/tools/manage_experiments/ensemble_widget.py 97.17% <100.00%> (ø)
src/ert/storage/local_ensemble.py 96.77% <100.00%> (ø)

... and 2 files with indirect coverage changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated no new comments.

Suppressed comments (2)

src/ert/config/_reservoir_data_utils.py:122

  • resolve_pattern_filepaths() only checks search_dir.exists(). If basedir/pattern_path.parent exists but is a file (not a directory), the later iterdir() call will raise NotADirectoryError instead of the intended on_error(...). Similarly, the non-glob branch should ensure the resolved path is a file (not just exists()) so directory paths don’t get treated as response/observation files.
        search_dir = Path(basedir) / pattern_path.parent
        if not search_dir.exists():
            raise on_error(
                f"Directory '{search_dir.absolute()}' does not exist "
                "or is not accessible."
            )

src/ert/config/_reservoir_data_utils.py:129

  • The wildcard branch returns matching_paths in the filesystem’s iterdir() order, which is not guaranteed to be stable across platforms/filesystems. Sorting the matches (e.g., by filename) makes response/observation ordering deterministic and avoids potential order-dependent behavior/flaky tests.
        matching_paths = [
            path
            for path in search_dir.iterdir()
            if path.is_file() and compiled_pattern.fullmatch(path.name)
        ]

@achaikou
achaikou marked this pull request as ready for review August 6, 2026 15:39
@codspeed-hq

codspeed-hq Bot commented Aug 6, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 36 untouched benchmarks


Comparing achaikou:seismic_wildcard (84fdd0d) with main (47757c5)

Open in CodSpeed

@ajaust ajaust self-assigned this Aug 10, 2026

@ajaust ajaust left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes look good. 🙂
I only have some minor comments and questions.

Comment thread src/ert/config/_reservoir_data_utils.py
Comment thread tests/ert/unit_tests/config/test_observation_declaration.py
Comment thread tests/ert/unit_tests/config/test_seismic_config.py Outdated
Comment thread tests/ert/unit_tests/config/test_observation_declaration.py Outdated
Comment thread src/ert/config/_reservoir_data_utils.py Outdated
@ajaust

ajaust commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Should we also add the wildcard support somewhere to the documentation?

@berland berland changed the title Allow use of wildcards in definition of seimisc observations/responses Allow use of wildcards in definition of seismic observations/responses Aug 18, 2026

@achaikou achaikou left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review!

Unfortunately, due to severe merge conflict I had to rebase first and do significant changes in the last 2 commits.
Mostly changes affected test_seismic_config.

The docs I plan to add/go through before the feature is officially released, which apparently should be this month now 😄.

Comment thread src/ert/config/_reservoir_data_utils.py
Comment thread tests/ert/unit_tests/config/test_observation_declaration.py

@ajaust ajaust left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall. I looked especially into the two last commits. I have only a few questions/remarks. 🙂

Comment thread src/ert/config/_reservoir_data_utils.py
Comment thread tests/ert/unit_tests/config/test_observation_declaration.py Outdated
Comment thread tests/ert/unit_tests/config/test_observation_declaration.py Outdated
Comment thread src/ert/config/seismic_config.py

@ajaust ajaust left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉

As it has been pointed out, we start having too much duplication between
observations and config, yet we do not have a good place to put it into.
So creating a new file to keep common logic.

ReservoirData was chosen as a general term to describe
RFT/Seismic/Summary response or observation data.
As there was no constrain on realizations, all realizations were joined
with all responses, resulting in too many dots on the plot.
Changes to other tests are mostly caused by additional check for
directory existence that doesn't play nicely with mocked_files.
@achaikou
achaikou merged commit 9465d20 into equinor:main Aug 19, 2026
120 of 134 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release-notes:unreleased-feature-changes PR with changes to a feature which is not yet released. Not for introduction of new features!

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Regex or wildcards in seismic filenames in response/observation

4 participants