|
| 1 | +# NeqSim compatibility suite |
| 2 | + |
| 3 | +NeqSim is a third-party Java library we ship as a jar file. That |
| 4 | +arrangement is uncomfortable for several reasons. A new jar can shift the |
| 5 | +value of a flash without anyone in the ecalc team being told. When a |
| 6 | +NeqSim release regresses — i.e. starts returning different numbers than |
| 7 | +it used to for inputs that previously worked — it usually does so |
| 8 | +silently. The values look finite and reasonable, but they propagate |
| 9 | +through compressor and pressure logic until something crashes far from |
| 10 | +the root cause. And we query NeqSim in regions no one hand-checks: up |
| 11 | +to 2000 bara at 450 K during max-speed probes, on whatever composition |
| 12 | +the user feeds in. The existing application tests do not pin NeqSim's |
| 13 | +outputs, so today the only signal that a jar bump broke something is |
| 14 | +often an unrelated-looking failure several layers downstream. We would |
| 15 | +like to stop being surprised this way. |
| 16 | + |
| 17 | +A compatibility suite should give us three guarantees, in priority order. |
| 18 | +A jar bump that changes any output we read should produce a failing test, |
| 19 | +named after the (composition, P, T, EoS) state where it changed — not a |
| 20 | +slow regression in some integration metric. A wrapper change that breaks |
| 21 | +an operation we depend on (flashes, phase extraction, mixing, EoS |
| 22 | +differentiation) should fail in the same checkout, not in the next ecalc |
| 23 | +run. And an operating point ecalc can plausibly query, where NeqSim |
| 24 | +returns garbage, should fail here before that garbage reaches the |
| 25 | +compressor solver. |
| 26 | + |
| 27 | +Equally important is what this suite is *not*. It is not an ecalc test |
| 28 | +suite — bugs in solver logic, validation, units, caches, or process |
| 29 | +units belong in the existing tests. It is not a NeqSim correctness suite |
| 30 | +— we are detecting *change*, not certifying *physics*. And it is not run |
| 31 | +on every PR; it is deselected by default and runs only when the jar, the |
| 32 | +wrapper, or the suite itself changes, or when an engineer asks for it. |
| 33 | + |
| 34 | +Structurally we want three layers, each answering a single question and |
| 35 | +all reading from one source of truth for the operating envelope (pressure |
| 36 | +range, temperature range, EoS models, compositions). The first layer asks |
| 37 | +whether NeqSim returned a sensible number at all — finite, in physical |
| 38 | +bounds, no stale defaults in the output — across the full envelope. The |
| 39 | +second asks whether NeqSim plays by the rules under the operations we |
| 40 | +depend on, with targeted tests that pin each operation's invariants rather |
| 41 | +than its exact numerical output. The third asks whether any number we read |
| 42 | +has drifted: a snapshot of every property ecalc reads at every state in |
| 43 | +the envelope, generated against a known-good jar and reviewed on every |
| 44 | +bump. |
| 45 | + |
| 46 | +A failure must name the failing state in a form the engineer can paste |
| 47 | +into a notebook to reproduce, and must make clear which layer caught it — |
| 48 | +garbage value, broken invariant, or drifted number — without anyone |
| 49 | +needing to read test source. |
| 50 | + |
| 51 | +We accept upfront that this design will not catch ecalc-side bugs, bugs |
| 52 | +at operating points outside the envelope, drift smaller than the snapshot |
| 53 | +tolerance, or regressions in NeqSim operations the wrapper does not |
| 54 | +exercise. Those trade-offs are deliberate. The suite is doing its job if |
| 55 | +every jar bump PR ends in either a clean run, an explicitly accepted |
| 56 | +snapshot diff, or a rejection of the bump — and if we stop finding NeqSim |
| 57 | +regressions by way of downstream `NaN` crashes. |
| 58 | + |
| 59 | +The GitHub Actions workflow `test-neqsim-compatibility.yml` gates on jar, |
| 60 | +wrapper, suite, and workflow path changes. |
| 61 | + |
| 62 | +## Operating envelope |
| 63 | + |
| 64 | +`envelope.py` is the single source of truth. Every state-generating |
| 65 | +helper in this directory reads it from there: |
| 66 | + |
| 67 | +| Dimension | Range | |
| 68 | +| ------------- | ---------------------------------------------------------- | |
| 69 | +| Pressure | 1 – 2000 bara (`MAX_FIRST_GUESS_BAR` cap on max-speed probes) | |
| 70 | +| Temperature | 250 – 460 K (per-stage PH-flash outlet at off-design) | |
| 71 | +| EoS | SRK, PR, GERG_SRK, GERG_PR | |
| 72 | +| Operations | TP-flash, PH-flash, remove_liquid, mixing, property extraction | |
| 73 | + |
| 74 | +Three named sub-grids cover three regimes ecalc realistically hits: |
| 75 | + |
| 76 | +- `nominal_grid()` — 1–200 bara × 250–380 K (compressor suction to mid-discharge). |
| 77 | +- `high_pressure_grid()` — 300–400 bara × 300–360 K (e.g. Sverdrup gas injection). |
| 78 | +- `max_speed_probe_grid()` — 500–2000 bara × 400–450 K (dense-supercritical regime hit during max-speed probes). |
| 79 | + |
| 80 | +When ecalc's envelope changes, update `envelope.py`. Everything |
| 81 | +downstream re-derives. |
| 82 | + |
| 83 | +## How it's organised |
| 84 | + |
| 85 | +Three groups, each answering a different question: |
| 86 | + |
| 87 | +1. **`sanity/` — does NeqSim return numbers that look real?** |
| 88 | + Point-wise sanity checks (no NaN, no default kappa, values in |
| 89 | + physical ranges) plus trajectory continuity (density monotonic in |
| 90 | + P on single-phase segments, enthalpy monotonic in T at fixed P). |
| 91 | + Also the external-reference checks for selected (composition, P, |
| 92 | + T, EoS) states with published values, and the structural guard |
| 93 | + that no test parametrises a wet composition below its temperature |
| 94 | + floor. |
| 95 | + |
| 96 | +2. **`behaviour/` — does NeqSim follow the rules our pipeline code |
| 97 | + assumes?** Self-consistency invariants for the operations ecalc |
| 98 | + composes in production: state identities (round-trip, idempotency, |
| 99 | + getter/setter consistency), phase operations (gas-phase extraction |
| 100 | + matches a clean PT flash of the gas-phase composition), flash |
| 101 | + operations (PH-flash returns the requested enthalpy, an enthalpy |
| 102 | + increase raises temperature, a five-stage chain stays well-defined), |
| 103 | + mixing (mass and molar balance), EoS differentiation (SRK vs PR |
| 104 | + produce measurably different densities), and the wrapper's own |
| 105 | + degenerate-state validators. |
| 106 | + |
| 107 | +3. **`regression/` — do the numbers match the previous jar?** A pinned |
| 108 | + reference snapshot (`reference_snapshot.json`) holds every property |
| 109 | + for every cell in the regression spec at strict 1e-9 relative |
| 110 | + tolerance. Any drift fails the test; the fix is either to revert |
| 111 | + the bump or to deliberately regenerate the snapshot with a brief |
| 112 | + justification of the accepted drift. |
| 113 | + |
| 114 | +## Design constraints |
| 115 | + |
| 116 | +These are baked into the state generators. The structural guard |
| 117 | +`sanity/test_temperature_floors.py` will fail at collection time if a |
| 118 | +new test or pin violates them. |
| 119 | + |
| 120 | +- **No wet composition is flashed below ~273 K.** NeqSim does not |
| 121 | + model the solid water phase. Below water's freezing point a flash |
| 122 | + on a water-bearing composition can return NaN / default kappa / |
| 123 | + non-physical Z, flip phase splits between EoS models, or in some |
| 124 | + jar versions tear down the JVM gateway. The suite therefore |
| 125 | + declines to test wet compositions below a conservative floor of |
| 126 | + **280 K**. The floor lives in |
| 127 | + `compositions.MIN_TEMPERATURE_KELVIN_PER_COMPOSITION` and is |
| 128 | + populated automatically for any composition with `water > 0`. |
| 129 | + Every state generator consults `is_state_supported(name, T)`. To |
| 130 | + probe the cold/high-P region for a heavy composition that is |
| 131 | + normally used wet, add a dry sister composition (see |
| 132 | + `c3_rich_wellstream_dry`). |
| 133 | + |
| 134 | +- **`sanity/` and `behaviour/` use lenient predicates; `regression/` |
| 135 | + is strict.** The sanity and behaviour groups catch categorical |
| 136 | + breakage (NaN, default, non-physical, broken algebraic identities). |
| 137 | + Strict numerical pinning lives only in the regression snapshot. |
| 138 | + |
| 139 | +## Running it |
| 140 | + |
| 141 | +```bash |
| 142 | +# Full suite, locally. Targeting the directory auto-enables the |
| 143 | +# `neqsim_compat` marker; you don't need `-m neqsim_compat`. |
| 144 | +uv run pytest tests/ecalc_neqsim_wrapper/compatibility/ |
| 145 | + |
| 146 | +# Just the sanity group. |
| 147 | +uv run pytest tests/ecalc_neqsim_wrapper/compatibility/sanity/ |
| 148 | + |
| 149 | +# Regenerate the regression snapshot against the currently vendored jar. |
| 150 | +uv run pytest tests/ecalc_neqsim_wrapper/compatibility/ --regenerate-neqsim-snapshot |
| 151 | +``` |
| 152 | + |
| 153 | +The full suite takes around 20 minutes. JVM startup dominates the |
| 154 | +first few seconds; the rest is NeqSim flash calls. |
| 155 | + |
| 156 | +## How to react to a failure |
| 157 | + |
| 158 | +1. **Sanity failure** — NeqSim is returning garbage at a state ecalc |
| 159 | + exercises. Either the jar regressed in that region, or the suite |
| 160 | + has hit a documented constraint that should be expressed in the |
| 161 | + floor registry. Look at the failing state first; if it's a |
| 162 | + wet composition below 273 K, the fix is in `compositions.py`, |
| 163 | + not in NeqSim. |
| 164 | + |
| 165 | +2. **Behaviour failure** — NeqSim is returning numbers that violate |
| 166 | + an algebraic identity ecalc relies on (e.g. PH-flash output |
| 167 | + enthalpy doesn't match the input target). This is a real |
| 168 | + regression and blocks the bump. |
| 169 | + |
| 170 | +3. **Regression failure** — A number drifted outside the strict |
| 171 | + tolerance. Investigate before regenerating. If the drift is an |
| 172 | + improvement (e.g. bug fix in a NeqSim correlation), regenerate |
| 173 | + the snapshot and commit it alongside the jar bump with a brief |
| 174 | + justification. If the drift is a quiet correctness loss, block |
| 175 | + the bump. |
0 commit comments