Skip to content

Commit e04c4eb

Browse files
olelodolelod
committed
test(neqsim): add compatibility suite for the vendored jar
Adds an on-demand compatibility suite under tests/ecalc_neqsim_wrapper/compatibility/ that verifies the vendored NeqSim jar and wrapper across representative ecalc operating conditions. The suite includes a strict reference snapshot for numerical drift, sanity checks for finite and physically plausible properties, and behaviour checks for TP flash, PH flash, gas-phase extraction, mixing, EoS routing, and validator integration. A GitHub Actions workflow runs the suite when the jar, wrapper, suite, or workflow changes, and the README documents the rationale, operating envelope, commands, and failure triage.
1 parent faa2f4e commit e04c4eb

24 files changed

Lines changed: 9557 additions & 2 deletions

‎.github/workflows/test-neqsim-compatibility.yml‎

Lines changed: 98 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,98 @@
1+
name: NeqSim Compatibility Suite
2+
3+
# The NeqSim compatibility suite pins the behaviour of the vendored
4+
# NeqSim jar (src/ecalc_neqsim_wrapper/lib/NeqSim.jar) over the
5+
# operating envelope ecalc exercises. It is deselected from the default
6+
# test run (see pyproject.toml `addopts`) and is intended to run only
7+
# when the jar, the wrapper, or the suite itself changes.
8+
9+
on:
10+
pull_request:
11+
paths:
12+
- 'src/ecalc_neqsim_wrapper/lib/NeqSim.jar'
13+
- 'src/ecalc_neqsim_wrapper/lib/neqsim_version_info.md'
14+
- 'src/ecalc_neqsim_wrapper/**.py'
15+
- 'tests/ecalc_neqsim_wrapper/compatibility/**'
16+
- '.github/workflows/test-neqsim-compatibility.yml'
17+
workflow_dispatch:
18+
19+
permissions: { }
20+
21+
jobs:
22+
enforce-version-note-update:
23+
name: Enforce neqsim_version_info.md update when the jar changes
24+
if: github.event_name == 'pull_request'
25+
permissions:
26+
contents: read
27+
runs-on: ubuntu-24.04
28+
steps:
29+
- name: Checkout code
30+
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
31+
with:
32+
persist-credentials: false
33+
fetch-depth: 0
34+
35+
- name: Refuse jar bump without version-note update
36+
run: |
37+
set -euo pipefail
38+
base="origin/${{ github.base_ref }}"
39+
changed=$(git diff --name-only "$base"...HEAD)
40+
echo "Changed files:"
41+
echo "$changed"
42+
if echo "$changed" | grep -qx 'src/ecalc_neqsim_wrapper/lib/NeqSim.jar'; then
43+
if ! echo "$changed" | grep -qx 'src/ecalc_neqsim_wrapper/lib/neqsim_version_info.md'; then
44+
echo "::error::NeqSim.jar changed but neqsim_version_info.md was not updated."
45+
exit 1
46+
fi
47+
fi
48+
49+
test-neqsim-compatibility:
50+
name: Run NeqSim compatibility suite
51+
permissions:
52+
contents: read
53+
runs-on: ubuntu-24.04
54+
strategy:
55+
fail-fast: false
56+
matrix:
57+
python-version: ["3.12", "3.14"]
58+
steps:
59+
- name: Checkout code
60+
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
61+
with:
62+
persist-credentials: false
63+
64+
- name: Install uv
65+
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
66+
with:
67+
enable-cache: true
68+
python-version: ${{ matrix.python-version }}
69+
70+
- name: Set up Python
71+
run: uv python install
72+
73+
- name: Install the project
74+
run: uv sync --locked --all-extras --dev
75+
76+
- name: Report NeqSim jar fingerprint
77+
run: |
78+
ls -l src/ecalc_neqsim_wrapper/lib/NeqSim.jar
79+
sha256sum src/ecalc_neqsim_wrapper/lib/NeqSim.jar
80+
echo '--- neqsim_version_info.md ---'
81+
cat src/ecalc_neqsim_wrapper/lib/neqsim_version_info.md || true
82+
83+
- name: Run NeqSim compatibility suite
84+
timeout-minutes: 60
85+
run: |
86+
uv run python -m pytest \
87+
-m neqsim_compat \
88+
tests/ecalc_neqsim_wrapper/compatibility/ \
89+
--junitxml=neqsim-compatibility.xml \
90+
-v
91+
92+
- name: Upload JUnit results
93+
if: always()
94+
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
95+
with:
96+
name: neqsim-compatibility-${{ matrix.python-version }}
97+
path: neqsim-compatibility.xml
98+
if-no-files-found: ignore

‎pyproject.toml‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -98,7 +98,7 @@ build-backend = "hatchling.build"
9898
[tool.pytest.ini_options]
9999
testpaths = ["tests"]
100100
pythonpath = ["."] # add this to avoid import errors during test collection
101-
addopts = "--import-mode=importlib"
101+
addopts = ["--import-mode=importlib", "-m", "not neqsim_compat"]
102102
filterwarnings = [
103103
"error::DeprecationWarning", # Treat all DeprecationWarnings as errors and ignore explicitly below if needed
104104
"ignore:Avoid using the dto.*:DeprecationWarning", # Ignore internal deprecation warnings
@@ -112,7 +112,8 @@ markers = [
112112
"inlinesnapshot: inline snapshot tests, often used to test error messages",
113113
"dockersnapshot: tests that need x86 architecture for snapshot creation, so we make them in a container",
114114
"x86: tests that need x86 to pass",
115-
"arch: architecture tests."
115+
"arch: architecture tests.",
116+
"neqsim_compat: NeqSim compatibility suite — pins jar behaviour over the operating envelope ecalc exercises. Run when bumping the NeqSim jar; deselected by default."
116117
]
117118

118119
[tool.basedpyright]

‎tests/ecalc_neqsim_wrapper/compatibility/README.md‎

Lines changed: 175 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,175 @@
1+
# NeqSim compatibility suite
2+
3+
NeqSim is a third-party Java library we ship as a jar file. That
4+
arrangement is uncomfortable for several reasons. A new jar can shift the
5+
value of a flash without anyone in the ecalc team being told. When a
6+
NeqSim release regresses — i.e. starts returning different numbers than
7+
it used to for inputs that previously worked — it usually does so
8+
silently. The values look finite and reasonable, but they propagate
9+
through compressor and pressure logic until something crashes far from
10+
the root cause. And we query NeqSim in regions no one hand-checks: up
11+
to 2000 bara at 450 K during max-speed probes, on whatever composition
12+
the user feeds in. The existing application tests do not pin NeqSim's
13+
outputs, so today the only signal that a jar bump broke something is
14+
often an unrelated-looking failure several layers downstream. We would
15+
like to stop being surprised this way.
16+
17+
A compatibility suite should give us three guarantees, in priority order.
18+
A jar bump that changes any output we read should produce a failing test,
19+
named after the (composition, P, T, EoS) state where it changed — not a
20+
slow regression in some integration metric. A wrapper change that breaks
21+
an operation we depend on (flashes, phase extraction, mixing, EoS
22+
differentiation) should fail in the same checkout, not in the next ecalc
23+
run. And an operating point ecalc can plausibly query, where NeqSim
24+
returns garbage, should fail here before that garbage reaches the
25+
compressor solver.
26+
27+
Equally important is what this suite is *not*. It is not an ecalc test
28+
suite — bugs in solver logic, validation, units, caches, or process
29+
units belong in the existing tests. It is not a NeqSim correctness suite
30+
— we are detecting *change*, not certifying *physics*. And it is not run
31+
on every PR; it is deselected by default and runs only when the jar, the
32+
wrapper, or the suite itself changes, or when an engineer asks for it.
33+
34+
Structurally we want three layers, each answering a single question and
35+
all reading from one source of truth for the operating envelope (pressure
36+
range, temperature range, EoS models, compositions). The first layer asks
37+
whether NeqSim returned a sensible number at all — finite, in physical
38+
bounds, no stale defaults in the output — across the full envelope. The
39+
second asks whether NeqSim plays by the rules under the operations we
40+
depend on, with targeted tests that pin each operation's invariants rather
41+
than its exact numerical output. The third asks whether any number we read
42+
has drifted: a snapshot of every property ecalc reads at every state in
43+
the envelope, generated against a known-good jar and reviewed on every
44+
bump.
45+
46+
A failure must name the failing state in a form the engineer can paste
47+
into a notebook to reproduce, and must make clear which layer caught it —
48+
garbage value, broken invariant, or drifted number — without anyone
49+
needing to read test source.
50+
51+
We accept upfront that this design will not catch ecalc-side bugs, bugs
52+
at operating points outside the envelope, drift smaller than the snapshot
53+
tolerance, or regressions in NeqSim operations the wrapper does not
54+
exercise. Those trade-offs are deliberate. The suite is doing its job if
55+
every jar bump PR ends in either a clean run, an explicitly accepted
56+
snapshot diff, or a rejection of the bump — and if we stop finding NeqSim
57+
regressions by way of downstream `NaN` crashes.
58+
59+
The GitHub Actions workflow `test-neqsim-compatibility.yml` gates on jar,
60+
wrapper, suite, and workflow path changes.
61+
62+
## Operating envelope
63+
64+
`envelope.py` is the single source of truth. Every state-generating
65+
helper in this directory reads it from there:
66+
67+
| Dimension | Range |
68+
| ------------- | ---------------------------------------------------------- |
69+
| Pressure | 1 – 2000 bara (`MAX_FIRST_GUESS_BAR` cap on max-speed probes) |
70+
| Temperature | 250 – 460 K (per-stage PH-flash outlet at off-design) |
71+
| EoS | SRK, PR, GERG_SRK, GERG_PR |
72+
| Operations | TP-flash, PH-flash, remove_liquid, mixing, property extraction |
73+
74+
Three named sub-grids cover three regimes ecalc realistically hits:
75+
76+
- `nominal_grid()` — 1–200 bara × 250–380 K (compressor suction to mid-discharge).
77+
- `high_pressure_grid()` — 300–400 bara × 300–360 K (e.g. Sverdrup gas injection).
78+
- `max_speed_probe_grid()` — 500–2000 bara × 400–450 K (dense-supercritical regime hit during max-speed probes).
79+
80+
When ecalc's envelope changes, update `envelope.py`. Everything
81+
downstream re-derives.
82+
83+
## How it's organised
84+
85+
Three groups, each answering a different question:
86+
87+
1. **`sanity/` — does NeqSim return numbers that look real?**
88+
Point-wise sanity checks (no NaN, no default kappa, values in
89+
physical ranges) plus trajectory continuity (density monotonic in
90+
P on single-phase segments, enthalpy monotonic in T at fixed P).
91+
Also the external-reference checks for selected (composition, P,
92+
T, EoS) states with published values, and the structural guard
93+
that no test parametrises a wet composition below its temperature
94+
floor.
95+
96+
2. **`behaviour/` — does NeqSim follow the rules our pipeline code
97+
assumes?** Self-consistency invariants for the operations ecalc
98+
composes in production: state identities (round-trip, idempotency,
99+
getter/setter consistency), phase operations (gas-phase extraction
100+
matches a clean PT flash of the gas-phase composition), flash
101+
operations (PH-flash returns the requested enthalpy, an enthalpy
102+
increase raises temperature, a five-stage chain stays well-defined),
103+
mixing (mass and molar balance), EoS differentiation (SRK vs PR
104+
produce measurably different densities), and the wrapper's own
105+
degenerate-state validators.
106+
107+
3. **`regression/` — do the numbers match the previous jar?** A pinned
108+
reference snapshot (`reference_snapshot.json`) holds every property
109+
for every cell in the regression spec at strict 1e-9 relative
110+
tolerance. Any drift fails the test; the fix is either to revert
111+
the bump or to deliberately regenerate the snapshot with a brief
112+
justification of the accepted drift.
113+
114+
## Design constraints
115+
116+
These are baked into the state generators. The structural guard
117+
`sanity/test_temperature_floors.py` will fail at collection time if a
118+
new test or pin violates them.
119+
120+
- **No wet composition is flashed below ~273 K.** NeqSim does not
121+
model the solid water phase. Below water's freezing point a flash
122+
on a water-bearing composition can return NaN / default kappa /
123+
non-physical Z, flip phase splits between EoS models, or in some
124+
jar versions tear down the JVM gateway. The suite therefore
125+
declines to test wet compositions below a conservative floor of
126+
**280 K**. The floor lives in
127+
`compositions.MIN_TEMPERATURE_KELVIN_PER_COMPOSITION` and is
128+
populated automatically for any composition with `water > 0`.
129+
Every state generator consults `is_state_supported(name, T)`. To
130+
probe the cold/high-P region for a heavy composition that is
131+
normally used wet, add a dry sister composition (see
132+
`c3_rich_wellstream_dry`).
133+
134+
- **`sanity/` and `behaviour/` use lenient predicates; `regression/`
135+
is strict.** The sanity and behaviour groups catch categorical
136+
breakage (NaN, default, non-physical, broken algebraic identities).
137+
Strict numerical pinning lives only in the regression snapshot.
138+
139+
## Running it
140+
141+
```bash
142+
# Full suite, locally. Targeting the directory auto-enables the
143+
# `neqsim_compat` marker; you don't need `-m neqsim_compat`.
144+
uv run pytest tests/ecalc_neqsim_wrapper/compatibility/
145+
146+
# Just the sanity group.
147+
uv run pytest tests/ecalc_neqsim_wrapper/compatibility/sanity/
148+
149+
# Regenerate the regression snapshot against the currently vendored jar.
150+
uv run pytest tests/ecalc_neqsim_wrapper/compatibility/ --regenerate-neqsim-snapshot
151+
```
152+
153+
The full suite takes around 20 minutes. JVM startup dominates the
154+
first few seconds; the rest is NeqSim flash calls.
155+
156+
## How to react to a failure
157+
158+
1. **Sanity failure** — NeqSim is returning garbage at a state ecalc
159+
exercises. Either the jar regressed in that region, or the suite
160+
has hit a documented constraint that should be expressed in the
161+
floor registry. Look at the failing state first; if it's a
162+
wet composition below 273 K, the fix is in `compositions.py`,
163+
not in NeqSim.
164+
165+
2. **Behaviour failure** — NeqSim is returning numbers that violate
166+
an algebraic identity ecalc relies on (e.g. PH-flash output
167+
enthalpy doesn't match the input target). This is a real
168+
regression and blocks the bump.
169+
170+
3. **Regression failure** — A number drifted outside the strict
171+
tolerance. Investigate before regenerating. If the drift is an
172+
improvement (e.g. bug fix in a NeqSim correlation), regenerate
173+
the snapshot and commit it alongside the jar bump with a brief
174+
justification. If the drift is a quiet correctness loss, block
175+
the bump.

‎tests/ecalc_neqsim_wrapper/compatibility/__init__.py‎

Whitespace-only changes.

‎tests/ecalc_neqsim_wrapper/compatibility/behaviour/__init__.py‎

Whitespace-only changes.

‎tests/ecalc_neqsim_wrapper/compatibility/behaviour/test_eos_differentiation.py‎

Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
"""EoS routing checks: SRK/PR differ, GERG setters refresh properties."""
2+
3+
from __future__ import annotations
4+
5+
import pytest
6+
7+
from ecalc_neqsim_wrapper.thermo import NeqsimFluid
8+
from libecalc.process.fluid_stream.fluid_model import EoSModel
9+
10+
from ..compositions import COMPOSITIONS
11+
12+
_GAS_DOMINANT_COMPOSITIONS = (
13+
"pure_methane",
14+
"lean_natural_gas",
15+
"typical_export_gas",
16+
"rich_associated_gas",
17+
"co2_heavy_injection",
18+
"n2_heavy",
19+
)
20+
21+
22+
@pytest.mark.parametrize("composition_name", _GAS_DOMINANT_COMPOSITIONS)
23+
def test_srk_and_pr_produce_measurably_different_densities(composition_name):
24+
"""SRK and PR are different cubic equations of state. At moderate
25+
pressure they must produce measurably different (but close)
26+
densities. Bit-identical output would mean the EoS selector is
27+
wired wrong."""
28+
pressure_bara = 100.0
29+
temperature_kelvin = 320.0
30+
srk = NeqsimFluid.create_thermo_system(
31+
composition=COMPOSITIONS[composition_name],
32+
pressure_bara=pressure_bara,
33+
temperature_kelvin=temperature_kelvin,
34+
eos_model=EoSModel.SRK,
35+
)
36+
pr = NeqsimFluid.create_thermo_system(
37+
composition=COMPOSITIONS[composition_name],
38+
pressure_bara=pressure_bara,
39+
temperature_kelvin=temperature_kelvin,
40+
eos_model=EoSModel.PR,
41+
)
42+
density_diff = abs(srk.density - pr.density) / srk.density
43+
assert density_diff > 1.0e-4, (
44+
f"SRK and PR densities are suspiciously close for {composition_name}: "
45+
f"srk={srk.density!r} pr={pr.density!r} rel_diff={density_diff:.2e}"
46+
)
47+
assert density_diff < 0.10, (
48+
f"SRK and PR densities differ by more than 10 % for {composition_name}: "
49+
f"srk={srk.density!r} pr={pr.density!r} rel_diff={density_diff:.2%}"
50+
)
51+
kappa_diff = abs(srk.kappa - pr.kappa) / srk.kappa
52+
assert kappa_diff > 1.0e-4, (
53+
f"SRK and PR kappa are suspiciously close for {composition_name}: "
54+
f"srk={srk.kappa!r} pr={pr.kappa!r} rel_diff={kappa_diff:.2e}"
55+
)
56+
z_diff = abs(srk.z - pr.z) / srk.z
57+
assert z_diff > 1.0e-4, (
58+
f"SRK and PR Z are suspiciously close for {composition_name}: "
59+
f"srk={srk.z!r} pr={pr.z!r} rel_diff={z_diff:.2e}"
60+
)
61+
62+
63+
@pytest.mark.parametrize("eos_model", (EoSModel.GERG_SRK, EoSModel.GERG_PR))
64+
@pytest.mark.parametrize("composition_name", _GAS_DOMINANT_COMPOSITIONS)
65+
def test_gerg_properties_refresh_after_setting_new_state(composition_name, eos_model):
66+
"""Under the GERG code path, properties cached at construction time
67+
must reflect the new state after a TP setter — not the original
68+
construction state."""
69+
state_a_p, state_a_t = 20.0, 310.0
70+
state_b_p, state_b_t = 80.0, 350.0
71+
72+
direct_state_b = NeqsimFluid.create_thermo_system(
73+
composition=COMPOSITIONS[composition_name],
74+
pressure_bara=state_b_p,
75+
temperature_kelvin=state_b_t,
76+
eos_model=eos_model,
77+
)
78+
via_setter = NeqsimFluid.create_thermo_system(
79+
composition=COMPOSITIONS[composition_name],
80+
pressure_bara=state_a_p,
81+
temperature_kelvin=state_a_t,
82+
eos_model=eos_model,
83+
).set_new_pressure_and_temperature(state_b_p, state_b_t)
84+
85+
for prop in ("density", "z", "kappa", "enthalpy_joule_per_kg"):
86+
direct_value = getattr(direct_state_b, prop)
87+
setter_value = getattr(via_setter, prop)
88+
denom = max(abs(direct_value), 1.0)
89+
relative_error = abs(direct_value - setter_value) / denom
90+
assert relative_error < 1.0e-6, (
91+
f"{eos_model.name} cache appears stale: {prop} via setter "
92+
f"({setter_value!r}) does not match direct construction "
93+
f"at the same state ({direct_value!r}) for {composition_name}; "
94+
f"rel_err={relative_error:.2e}"
95+
)

0 commit comments

Comments
 (0)