Skip to content

Commit eed9254

Browse files
olelodolelod
committed
test(neqsim): add compatibility suite for the vendored jar
Adds an on-demand compatibility suite under tests/ecalc_neqsim_wrapper/compatibility/ that verifies the vendored NeqSim jar and wrapper across representative ecalc operating conditions. The suite includes a strict reference snapshot for numerical drift, sanity checks for finite and physically plausible properties, and behaviour checks for TP flash, PH flash, gas-phase extraction, mixing, EoS routing, and validator integration. A GitHub Actions workflow runs the suite when the jar, wrapper, suite, or workflow changes, and the README documents the rationale, operating envelope, commands, and failure triage. Snapshot tolerances allow Linux runner drift observed for c3_rich_wellstream_dry at 50 bara and 330 K: vapor_fraction_molar drift 1.293e-08 (abs tolerance relaxed from 1.0e-08 to 2.0e-08) and density relative drift 1.054e-08 (relative tolerance relaxed from 1.0e-08 to 2.0e-08).
1 parent 77e5d7e commit eed9254

26 files changed

Lines changed: 9784 additions & 3 deletions

‎.github/workflows/test-library.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ jobs:
4040
# in order to verify that the package is correctly installed
4141
- name: Run tests
4242
timeout-minutes: 10
43-
run: uv run python -m pytest -m "not arch" --splits 5 --splitting-algorithm least_duration --group=${{ matrix.group }}
43+
run: uv run python -m pytest -m "not arch and not neqsim_compat" --splits 5 --splitting-algorithm least_duration --group=${{ matrix.group }}
4444

4545
- name: Build artifacts
4646
run: uv build

‎.github/workflows/test-neqsim-compatibility.yml‎

Lines changed: 98 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,98 @@
1+
name: NeqSim Compatibility Suite
2+
3+
# The NeqSim compatibility suite pins the behaviour of the vendored
4+
# NeqSim jar (src/ecalc_neqsim_wrapper/lib/NeqSim.jar) over the
5+
# operating envelope ecalc exercises. It is deselected from the default
6+
# test run (see pyproject.toml `addopts`) and is intended to run only
7+
# when the jar, the wrapper, or the suite itself changes.
8+
9+
on:
10+
pull_request:
11+
paths:
12+
- 'src/ecalc_neqsim_wrapper/lib/NeqSim.jar'
13+
- 'src/ecalc_neqsim_wrapper/lib/neqsim_version_info.md'
14+
- 'src/ecalc_neqsim_wrapper/**.py'
15+
- 'tests/ecalc_neqsim_wrapper/compatibility/**'
16+
- '.github/workflows/test-neqsim-compatibility.yml'
17+
workflow_dispatch:
18+
19+
permissions: { }
20+
21+
jobs:
22+
enforce-version-note-update:
23+
name: Enforce neqsim_version_info.md update when the jar changes
24+
if: github.event_name == 'pull_request'
25+
permissions:
26+
contents: read
27+
runs-on: ubuntu-24.04
28+
steps:
29+
- name: Checkout code
30+
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
31+
with:
32+
persist-credentials: false
33+
fetch-depth: 0
34+
35+
- name: Refuse jar bump without version-note update
36+
run: |
37+
set -euo pipefail
38+
base="origin/${{ github.base_ref }}"
39+
changed=$(git diff --name-only "$base"...HEAD)
40+
echo "Changed files:"
41+
echo "$changed"
42+
if echo "$changed" | grep -qx 'src/ecalc_neqsim_wrapper/lib/NeqSim.jar'; then
43+
if ! echo "$changed" | grep -qx 'src/ecalc_neqsim_wrapper/lib/neqsim_version_info.md'; then
44+
echo "::error::NeqSim.jar changed but neqsim_version_info.md was not updated."
45+
exit 1
46+
fi
47+
fi
48+
49+
test-neqsim-compatibility:
50+
name: Run NeqSim compatibility suite
51+
permissions:
52+
contents: read
53+
runs-on: ubuntu-24.04
54+
strategy:
55+
fail-fast: false
56+
matrix:
57+
python-version: ["3.12", "3.14"]
58+
steps:
59+
- name: Checkout code
60+
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
61+
with:
62+
persist-credentials: false
63+
64+
- name: Install uv
65+
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
66+
with:
67+
enable-cache: true
68+
python-version: ${{ matrix.python-version }}
69+
70+
- name: Set up Python
71+
run: uv python install
72+
73+
- name: Install the project
74+
run: uv sync --locked --all-extras --dev
75+
76+
- name: Report NeqSim jar fingerprint
77+
run: |
78+
ls -l src/ecalc_neqsim_wrapper/lib/NeqSim.jar
79+
sha256sum src/ecalc_neqsim_wrapper/lib/NeqSim.jar
80+
echo '--- neqsim_version_info.md ---'
81+
cat src/ecalc_neqsim_wrapper/lib/neqsim_version_info.md || true
82+
83+
- name: Run NeqSim compatibility suite
84+
timeout-minutes: 60
85+
run: |
86+
uv run python -m pytest \
87+
-m neqsim_compat \
88+
tests/ecalc_neqsim_wrapper/compatibility/ \
89+
--junitxml=neqsim-compatibility.xml \
90+
-v
91+
92+
- name: Upload JUnit results
93+
if: always()
94+
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
95+
with:
96+
name: neqsim-compatibility-${{ matrix.python-version }}
97+
path: neqsim-compatibility.xml
98+
if-no-files-found: ignore

‎pyproject.toml‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -98,7 +98,7 @@ build-backend = "hatchling.build"
9898
[tool.pytest.ini_options]
9999
testpaths = ["tests"]
100100
pythonpath = ["."] # add this to avoid import errors during test collection
101-
addopts = "--import-mode=importlib"
101+
addopts = ["--import-mode=importlib", "-m", "not neqsim_compat"]
102102
filterwarnings = [
103103
"error::DeprecationWarning", # Treat all DeprecationWarnings as errors and ignore explicitly below if needed
104104
"ignore:Avoid using the dto.*:DeprecationWarning", # Ignore internal deprecation warnings
@@ -112,7 +112,8 @@ markers = [
112112
"inlinesnapshot: inline snapshot tests, often used to test error messages",
113113
"dockersnapshot: tests that need x86 architecture for snapshot creation, so we make them in a container",
114114
"x86: tests that need x86 to pass",
115-
"arch: architecture tests."
115+
"arch: architecture tests.",
116+
"neqsim_compat: NeqSim compatibility suite — pins jar behaviour over the operating envelope ecalc exercises. Run when bumping the NeqSim jar; deselected by default."
116117
]
117118

118119
[tool.basedpyright]

‎tests/ecalc_neqsim_wrapper/compatibility/README.md‎

Lines changed: 175 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,175 @@
1+
# NeqSim compatibility suite
2+
3+
NeqSim is a third-party Java library we ship as a jar file. That
4+
arrangement is uncomfortable for several reasons. A new jar can shift the
5+
value of a flash without anyone in the ecalc team being told. When a
6+
NeqSim release regresses — i.e. starts returning different numbers than
7+
it used to for inputs that previously worked — it usually does so
8+
silently. The values look finite and reasonable, but they propagate
9+
through compressor and pressure logic until something crashes far from
10+
the root cause. And we query NeqSim in regions no one hand-checks: up
11+
to 2000 bara at 450 K during max-speed probes, on whatever composition
12+
the user feeds in. The existing application tests do not pin NeqSim's
13+
outputs, so today the only signal that a jar bump broke something is
14+
often an unrelated-looking failure several layers downstream. We would
15+
like to stop being surprised this way.
16+
17+
A compatibility suite should give us three guarantees, in priority order.
18+
A jar bump that changes any output we read should produce a failing test,
19+
named after the (composition, P, T, EoS) state where it changed — not a
20+
slow regression in some integration metric. A wrapper change that breaks
21+
an operation we depend on (flashes, phase extraction, mixing, EoS
22+
differentiation) should fail in the same checkout, not in the next ecalc
23+
run. And an operating point ecalc can plausibly query, where NeqSim
24+
returns garbage, should fail here before that garbage reaches the
25+
compressor solver.
26+
27+
Equally important is what this suite is *not*. It is not an ecalc test
28+
suite — bugs in solver logic, validation, units, caches, or process
29+
units belong in the existing tests. It is not a NeqSim correctness suite
30+
— we are detecting *change*, not certifying *physics*. And it is not run
31+
on every PR; it is deselected by default and runs only when the jar, the
32+
wrapper, or the suite itself changes, or when an engineer asks for it.
33+
34+
Structurally we want three layers, each answering a single question and
35+
all reading from one source of truth for the operating envelope (pressure
36+
range, temperature range, EoS models, compositions). The first layer asks
37+
whether NeqSim returned a sensible number at all — finite, in physical
38+
bounds, no stale defaults in the output — across the full envelope. The
39+
second asks whether NeqSim plays by the rules under the operations we
40+
depend on, with targeted tests that pin each operation's invariants rather
41+
than its exact numerical output. The third asks whether any number we read
42+
has drifted: a snapshot of every property ecalc reads at every state in
43+
the envelope, generated against a known-good jar and reviewed on every
44+
bump.
45+
46+
A failure must name the failing state in a form the engineer can paste
47+
into a notebook to reproduce, and must make clear which layer caught it —
48+
garbage value, broken invariant, or drifted number — without anyone
49+
needing to read test source.
50+
51+
We accept upfront that this design will not catch ecalc-side bugs, bugs
52+
at operating points outside the envelope, drift smaller than the snapshot
53+
tolerance, or regressions in NeqSim operations the wrapper does not
54+
exercise. Those trade-offs are deliberate. The suite is doing its job if
55+
every jar bump PR ends in either a clean run, an explicitly accepted
56+
snapshot diff, or a rejection of the bump — and if we stop finding NeqSim
57+
regressions by way of downstream `NaN` crashes.
58+
59+
The GitHub Actions workflow `test-neqsim-compatibility.yml` gates on jar,
60+
wrapper, suite, and workflow path changes.
61+
62+
## Operating envelope
63+
64+
`envelope.py` is the single source of truth. Every state-generating
65+
helper in this directory reads it from there:
66+
67+
| Dimension | Range |
68+
| ------------- | ---------------------------------------------------------- |
69+
| Pressure | 1 – 2000 bara (`MAX_FIRST_GUESS_BAR` cap on max-speed probes) |
70+
| Temperature | 250 – 460 K (per-stage PH-flash outlet at off-design) |
71+
| EoS | SRK, PR, GERG_SRK, GERG_PR |
72+
| Operations | TP-flash, PH-flash, remove_liquid, mixing, property extraction |
73+
74+
Three named sub-grids cover three regimes ecalc realistically hits:
75+
76+
- `nominal_grid()` — 1–200 bara × 250–380 K (compressor suction to mid-discharge).
77+
- `high_pressure_grid()` — 300–400 bara × 300–360 K (e.g. Sverdrup gas injection).
78+
- `max_speed_probe_grid()` — 500–2000 bara × 400–450 K (dense-supercritical regime hit during max-speed probes).
79+
80+
When ecalc's envelope changes, update `envelope.py`. Everything
81+
downstream re-derives.
82+
83+
## How it's organised
84+
85+
Three groups, each answering a different question:
86+
87+
1. **`sanity/` — does NeqSim return numbers that look real?**
88+
Point-wise sanity checks (no NaN, no default kappa, values in
89+
physical ranges) plus trajectory continuity (density monotonic in
90+
P on single-phase segments, enthalpy monotonic in T at fixed P).
91+
Also the external-reference checks for selected (composition, P,
92+
T, EoS) states with published values, and the structural guard
93+
that no test parametrises a wet composition below its temperature
94+
floor.
95+
96+
2. **`behaviour/` — does NeqSim follow the rules our pipeline code
97+
assumes?** Self-consistency invariants for the operations ecalc
98+
composes in production: state identities (round-trip, idempotency,
99+
getter/setter consistency), phase operations (gas-phase extraction
100+
matches a clean PT flash of the gas-phase composition), flash
101+
operations (PH-flash returns the requested enthalpy, an enthalpy
102+
increase raises temperature, a five-stage chain stays well-defined),
103+
mixing (mass and molar balance), EoS differentiation (SRK vs PR
104+
produce measurably different densities), and the wrapper's own
105+
degenerate-state validators.
106+
107+
3. **`regression/` — do the numbers match the previous jar?** A pinned
108+
reference snapshot (`reference_snapshot.json`) holds every property
109+
for every cell in the regression spec at strict 1e-9 relative
110+
tolerance. Any drift fails the test; the fix is either to revert
111+
the bump or to deliberately regenerate the snapshot with a brief
112+
justification of the accepted drift.
113+
114+
## Design constraints
115+
116+
These are baked into the state generators. The structural guard
117+
`sanity/test_temperature_floors.py` will fail at collection time if a
118+
new test or pin violates them.
119+
120+
- **No wet composition is flashed below ~273 K.** NeqSim does not
121+
model the solid water phase. Below water's freezing point a flash
122+
on a water-bearing composition can return NaN / default kappa /
123+
non-physical Z, flip phase splits between EoS models, or in some
124+
jar versions tear down the JVM gateway. The suite therefore
125+
declines to test wet compositions below a conservative floor of
126+
**280 K**. The floor lives in
127+
`compositions.MIN_TEMPERATURE_KELVIN_PER_COMPOSITION` and is
128+
populated automatically for any composition with `water > 0`.
129+
Every state generator consults `is_state_supported(name, T)`. To
130+
probe the cold/high-P region for a heavy composition that is
131+
normally used wet, add a dry sister composition (see
132+
`c3_rich_wellstream_dry`).
133+
134+
- **`sanity/` and `behaviour/` use lenient predicates; `regression/`
135+
is strict.** The sanity and behaviour groups catch categorical
136+
breakage (NaN, default, non-physical, broken algebraic identities).
137+
Strict numerical pinning lives only in the regression snapshot.
138+
139+
## Running it
140+
141+
```bash
142+
# Full suite, locally. Targeting the directory auto-enables the
143+
# `neqsim_compat` marker; you don't need `-m neqsim_compat`.
144+
uv run pytest tests/ecalc_neqsim_wrapper/compatibility/
145+
146+
# Just the sanity group.
147+
uv run pytest tests/ecalc_neqsim_wrapper/compatibility/sanity/
148+
149+
# Regenerate the regression snapshot against the currently vendored jar.
150+
uv run pytest tests/ecalc_neqsim_wrapper/compatibility/ --regenerate-neqsim-snapshot
151+
```
152+
153+
The full suite takes around 20 minutes. JVM startup dominates the
154+
first few seconds; the rest is NeqSim flash calls.
155+
156+
## How to react to a failure
157+
158+
1. **Sanity failure** — NeqSim is returning garbage at a state ecalc
159+
exercises. Either the jar regressed in that region, or the suite
160+
has hit a documented constraint that should be expressed in the
161+
floor registry. Look at the failing state first; if it's a
162+
wet composition below 273 K, the fix is in `compositions.py`,
163+
not in NeqSim.
164+
165+
2. **Behaviour failure** — NeqSim is returning numbers that violate
166+
an algebraic identity ecalc relies on (e.g. PH-flash output
167+
enthalpy doesn't match the input target). This is a real
168+
regression and blocks the bump.
169+
170+
3. **Regression failure** — A number drifted outside the strict
171+
tolerance. Investigate before regenerating. If the drift is an
172+
improvement (e.g. bug fix in a NeqSim correlation), regenerate
173+
the snapshot and commit it alongside the jar bump with a brief
174+
justification. If the drift is a quiet correctness loss, block
175+
the bump.

‎tests/ecalc_neqsim_wrapper/compatibility/__init__.py‎

Whitespace-only changes.

‎tests/ecalc_neqsim_wrapper/compatibility/behaviour/__init__.py‎

Whitespace-only changes.

‎tests/ecalc_neqsim_wrapper/compatibility/behaviour/test_eos_differentiation.py‎

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
"""EoS routing checks: SRK/PR differ, GERG setters refresh properties."""
2+
3+
from __future__ import annotations
4+
5+
import pytest
6+
7+
from ecalc_neqsim_wrapper.thermo import NeqsimFluid
8+
from libecalc.process.fluid_stream.fluid_model import EoSModel
9+
10+
from ..compositions import COMPOSITIONS
11+
12+
_GAS_DOMINANT_COMPOSITIONS = (
13+
"pure_methane",
14+
"lean_natural_gas",
15+
"typical_export_gas",
16+
"rich_associated_gas",
17+
"co2_heavy_injection",
18+
"n2_heavy",
19+
)
20+
21+
22+
@pytest.mark.parametrize("composition_name", _GAS_DOMINANT_COMPOSITIONS)
23+
def test_srk_and_pr_produce_measurably_different_densities(composition_name):
24+
"""SRK and PR are different cubic equations of state. At moderate
25+
pressure they must produce measurably different (but close)
26+
densities. Bit-identical output would mean the EoS selector is
27+
wired wrong."""
28+
pressure_bara = 100.0
29+
temperature_kelvin = 320.0
30+
srk = NeqsimFluid.create_thermo_system(
31+
composition=COMPOSITIONS[composition_name],
32+
pressure_bara=pressure_bara,
33+
temperature_kelvin=temperature_kelvin,
34+
eos_model=EoSModel.SRK,
35+
)
36+
pr = NeqsimFluid.create_thermo_system(
37+
composition=COMPOSITIONS[composition_name],
38+
pressure_bara=pressure_bara,
39+
temperature_kelvin=temperature_kelvin,
40+
eos_model=EoSModel.PR,
41+
)
42+
density_diff = abs(srk.density - pr.density) / srk.density
43+
assert density_diff > 1.0e-4, (
44+
f"SRK and PR densities are suspiciously close for {composition_name}: "
45+
f"srk={srk.density!r} pr={pr.density!r} rel_diff={density_diff:.2e}"
46+
)
47+
assert density_diff < 0.10, (
48+
f"SRK and PR densities differ by more than 10 % for {composition_name}: "
49+
f"srk={srk.density!r} pr={pr.density!r} rel_diff={density_diff:.2%}"
50+
)
51+
kappa_diff = abs(srk.kappa - pr.kappa) / srk.kappa
52+
assert kappa_diff > 1.0e-4, (
53+
f"SRK and PR kappa are suspiciously close for {composition_name}: "
54+
f"srk={srk.kappa!r} pr={pr.kappa!r} rel_diff={kappa_diff:.2e}"
55+
)
56+
z_diff = abs(srk.z - pr.z) / srk.z
57+
assert z_diff > 1.0e-4, (
58+
f"SRK and PR Z are suspiciously close for {composition_name}: srk={srk.z!r} pr={pr.z!r} rel_diff={z_diff:.2e}"
59+
)
60+
61+
62+
@pytest.mark.parametrize("eos_model", (EoSModel.GERG_SRK, EoSModel.GERG_PR))
63+
@pytest.mark.parametrize("composition_name", _GAS_DOMINANT_COMPOSITIONS)
64+
def test_gerg_properties_refresh_after_setting_new_state(composition_name, eos_model):
65+
"""Under the GERG code path, properties cached at construction time
66+
must reflect the new state after a TP setter — not the original
67+
construction state."""
68+
state_a_p, state_a_t = 20.0, 310.0
69+
state_b_p, state_b_t = 80.0, 350.0
70+
71+
direct_state_b = NeqsimFluid.create_thermo_system(
72+
composition=COMPOSITIONS[composition_name],
73+
pressure_bara=state_b_p,
74+
temperature_kelvin=state_b_t,
75+
eos_model=eos_model,
76+
)
77+
via_setter = NeqsimFluid.create_thermo_system(
78+
composition=COMPOSITIONS[composition_name],
79+
pressure_bara=state_a_p,
80+
temperature_kelvin=state_a_t,
81+
eos_model=eos_model,
82+
).set_new_pressure_and_temperature(state_b_p, state_b_t)
83+
84+
for prop in ("density", "z", "kappa", "enthalpy_joule_per_kg"):
85+
direct_value = getattr(direct_state_b, prop)
86+
setter_value = getattr(via_setter, prop)
87+
denom = max(abs(direct_value), 1.0)
88+
relative_error = abs(direct_value - setter_value) / denom
89+
assert relative_error < 1.0e-6, (
90+
f"{eos_model.name} cache appears stale: {prop} via setter "
91+
f"({setter_value!r}) does not match direct construction "
92+
f"at the same state ({direct_value!r}) for {composition_name}; "
93+
f"rel_err={relative_error:.2e}"
94+
)

0 commit comments

Comments
 (0)