Part of a set of structural observations — see #1695 (index) for context.
Description
eCalc lets users define input time series (e.g. "gas injection rate") as arithmetic combinations of other columns, via expression strings inside YAML, evaluated by a purpose-built interpreter at libecalc/expression/ (expression_evaluator.py, expression.py, expression_tree.py). A representative expression from the docs:
HCEXPORT: SIM;OIL_PROD {+} SIM;GAS_SALES {/} 1000This is a genuine interpreter — its own tokenizer, its own shunting-yard operator-precedence handling, its own parenthesis matcher — for what is a fairly standard "safe formula evaluation over named variables" problem.
1. Implementation. Libraries like asteval or simpleeval already solve exactly this, with correct operator precedence, error messages with source positions, and years of edge-case hardening, while presenting the user the same "type an arithmetic string" surface as today. Right now the custom interpreter has to be maintained, documented, and taught as if it were a real differentiator, when it isn't one. Separately: Node.evaluate calls np.nan_to_num(...) directly on results, so divide-by-zero and invalid float ops silently become 0.0 or ~1.8e308 with no warning — worth checking whether that's intentional, since it's a second silent-failure surface independent of the parser question.
2. Scope. Is there a reason this arithmetic needs to live inside eCalc's YAML schema at all, rather than "upstream"? A domain-specific language (DSL) of this kind is defensible when the host language is compiled and can't easily expose a programmatic API, which is the traditional FORTRAN/reservoir-simulator situation. It's much harder to defend when the library underneath is Python itself, and a first-class, typed, IDE-completable Python API should exist. For manual authoring, the stated audience (reservoir/process/sustainability engineers) is already deriving one time series from others routinely in the tool they use for exactly that — a spreadsheet — so the derived column could be exported to the same CSV eCalc already reads for other inputs. For the programmatic/outer-loop case (optimization, sensitivity sweeps), the arithmetic would naturally happen in numpy/pandas in the orchestrating process, before eCalc is even invoked. I raise this separately because it changes whether (1) is worth fixing in place versus the surface being deprecated over time in favor of preprocessing.
Examples
HCEXPORT: SIM;OIL_PROD {+} SIM;GAS_SALES {/} 1000With asteval/simpleeval, the same string (modulo bracket-vs-infix operator syntax, which could be kept for backward compatibility) would parse with standard precedence and produce a standard error with a source position on malformed input, instead of a bespoke error path.
Business/User Value
Less bespoke parsing/formatting code to maintain and test; better error messages for users authoring expressions; removes a silent-failure mode (NaN/inf collapsing to 0.0/1.8e308 with no warning); and, if the scope question lands on "move it upstream," removes an entire subsystem from eCalc's surface area rather than just improving it in place.
Relevant Links
Files
libecalc/expression/expression_evaluator.pylibecalc/expression/expression.pylibecalc/expression/expression_tree.py
Part of a set of structural observations — see #1695 (index) for context.
Description
eCalc lets users define input time series (e.g. "gas injection rate") as arithmetic combinations of other columns, via expression strings inside YAML, evaluated by a purpose-built interpreter at
libecalc/expression/(expression_evaluator.py,expression.py,expression_tree.py). A representative expression from the docs:This is a genuine interpreter — its own tokenizer, its own shunting-yard operator-precedence handling, its own parenthesis matcher — for what is a fairly standard "safe formula evaluation over named variables" problem.
1. Implementation. Libraries like
astevalorsimpleevalalready solve exactly this, with correct operator precedence, error messages with source positions, and years of edge-case hardening, while presenting the user the same "type an arithmetic string" surface as today. Right now the custom interpreter has to be maintained, documented, and taught as if it were a real differentiator, when it isn't one. Separately:Node.evaluatecallsnp.nan_to_num(...)directly on results, so divide-by-zero and invalid float ops silently become0.0or ~1.8e308 with no warning — worth checking whether that's intentional, since it's a second silent-failure surface independent of the parser question.2. Scope. Is there a reason this arithmetic needs to live inside eCalc's YAML schema at all, rather than "upstream"? A domain-specific language (DSL) of this kind is defensible when the host language is compiled and can't easily expose a programmatic API, which is the traditional FORTRAN/reservoir-simulator situation. It's much harder to defend when the library underneath is Python itself, and a first-class, typed, IDE-completable Python API should exist. For manual authoring, the stated audience (reservoir/process/sustainability engineers) is already deriving one time series from others routinely in the tool they use for exactly that — a spreadsheet — so the derived column could be exported to the same CSV eCalc already reads for other inputs. For the programmatic/outer-loop case (optimization, sensitivity sweeps), the arithmetic would naturally happen in numpy/pandas in the orchestrating process, before eCalc is even invoked. I raise this separately because it changes whether (1) is worth fixing in place versus the surface being deprecated over time in favor of preprocessing.
Examples
With
asteval/simpleeval, the same string (modulo bracket-vs-infix operator syntax, which could be kept for backward compatibility) would parse with standard precedence and produce a standard error with a source position on malformed input, instead of a bespoke error path.Business/User Value
Less bespoke parsing/formatting code to maintain and test; better error messages for users authoring expressions; removes a silent-failure mode (NaN/inf collapsing to
0.0/1.8e308 with no warning); and, if the scope question lands on "move it upstream," removes an entire subsystem from eCalc's surface area rather than just improving it in place.Relevant Links
Files
libecalc/expression/expression_evaluator.pylibecalc/expression/expression.pylibecalc/expression/expression_tree.py