Skip to content

Leverage opm-common keyword definitions to expand VS Code extension features #6

Description

@magnesj

Summary

The repo ships opm-common as a submodule and merges its parser-truth keyword
definitions into the extension's keyword index. The index now exposes sections,
per-item value_type / dimension / default / options, per-record arity
(expected_columns, records_meta), and the derived size_kind / size_count /
templated / variadic_record shape fields. This plan enumerates the remaining
opportunities to deepen the editor's understanding of OPM Flow decks.

Plan status (updated 2026-06-29). The diagnostics, multi-record, outline and
docs-panel work has landed and the index pipeline has grown the per-record /
shape fields it needed. Items 1 (requires/prohibits diagnostics), 2
(positional value_type checking) and 4 (deck_names aliases) are now done

all verified to add zero false positives on the known-good opm-tests corpus.
A UDQ / ACTIONX sub-language layer also shipped (highlighting, hover,
completion, two new diagnostics, and expression-block alignment) — recorded
under Shipped beyond the original plan. The remaining gaps are the rest of
item 9's index fields (size_type/dialect), code actions (quick-fixes), and the
richer completions (snippets, UDA-aware, well-name). See the Progress section
at the bottom for a per-item rollup.

Why now

  • opm-common is already wired in as a sparse submodule, the index build merges it, and CI rebuilds on every release. The data is reliable and free.
  • We have 41 keywords with requires, 35 with prohibits, 236 with size_type, 35 with deck_names (aliases), and a records[] schema for multi-record keywords like ACTIONX. The records[] schema is now consumed; the others are still not surfaced.
  • The arity diagnostic is currently total-tokens-vs-expected_columns (now per-record for multi-record keywords). We have full per-position value_type and dimension available, so we can do positional type checking for free.

Current vs. available metadata

Metadata in opm-common Used today Possible feature
sections ✔ section-validity diagnostic, hover, completion
value_type per item ✔ shown in hover/sidebar + per-position type diagnostic smarter completions
dimension per item ✔ shown in hover/sidebar unit-aware hover (FIELD/METRIC/LAB)
default per item ✔ shown in hover/sidebar inline-hint adornment, snippet placeholders
options per item ✔ value completion + sidebar
records[] (multi-record schema) records_meta: per-record arity, hover & docs tables
size shape (→ size_kind/size_count) ✔ terminator & continuation diagnostics
requires (41 kw) ✔ emitted + "X requires Y" diagnostic quick-fix (item 5)
prohibits (34 kw) ✔ emitted + "X conflicts with Y" diagnostic
size_type, size{keyword,item} row-count validation against TABDIMS etc.
deck_names + deck_name_regex ✔ aliases resolved + alias_of in hover; PROBE/region families expanded; container schema names suppressed
data (free-form bulk keywords) partial (size_kind: array) mark COORD/ZCORN/ACTNUM blocks differently
Dialect dir (000_Eclipse100900_OPM) tag completions/hover with dialect (E100/E300/OPM)

Proposed work

1. Diagnostics — cross-keyword constraints — ✅ done

  • requires check: a keyword with non-empty requires whose required partner is absent gets a "X requires Y, which is not present in the deck" warning. Suppressed for INCLUDE-bearing decks (the partner may live in an included file) and for bare INCLUDE fragments (no section header — a .inc/.grdecl legitimately omits the phase keywords its tables require). Phase requirements (OIL/GAS/WATER) are also treated as satisfied under implicit-phase modes (CO2STORE/H2STORE).
  • prohibits check: warns "X cannot be used together with Y; they are mutually exclusive" when both are present, reported once per unordered pair (fires even when only one side declares the constraint, e.g. THERMAL prohibits TEMP).
  • Tests added in vscode-extension/src/analysis.test.ts; zero false positives on the known-good opm-tests corpus.

2. Diagnostics — positional type checking — ✅ done (enum check intentionally dropped)

  • computeDiagnostics now checks each record token against the matching item's value_type with a precise per-token underline range:
    • INT: flags a decimal/float or a quoted string; integers and defaults (*, N*) pass.
    • DOUBLE: flags a quoted string; numerics (incl. scientific) pass.
    • Bare identifiers in numeric slots (UDA/UDQ references, macro substitutions) are left alone — the line-oriented engine can't resolve them. UDA items accept anything. Repeat-values (N*VALUE) check the value; multi-record keywords use the active record's params; variadic-record table keywords are skipped.
  • Arity-overflow check unchanged; columns beyond the mapped parameters are not type-checked.
  • STRING options[] membership checkdeliberately not implemented. The option sets are extracted heuristically from manual prose and are frequently incomplete/abbreviated (e.g. WCONINJE's phase lists WAT, not the valid WATER), producing ~3000 false positives on the known-good opm-tests corpus. Options remain in the index for completions only. Revisit only if a reliable enum source (opm-common itself) becomes available.

3. Diagnostics — multi-record schema (records[]) — ✅ done

  • Per-record arity: records_meta[currentRecord-1].expected_columns drives the over-arity check; the message names which record overflowed.
  • A record cursor (currentRecord) advances on each terminating /, capped so the trailing variadic record absorbs all remaining rows (WELSEGS, VFPPROD, COMPSEGS, ACTIONX, TUNING…).
  • Hover and the docs sidebar resolve the parameter against the active record and render one parameter table per record.

4. Aliases (deck_names) — ✅ done

  • Emit deck_names from the index build: each concrete deck keyword becomes (or is tagged on) an entry carrying alias_of naming its opm-common family (WOPRWELL_PROBE, KRNUMXKRNUM). The 31 family/container schema names (WELL_PROBE, AQUIFER_PROBE_ANALYTIC, MULT_XYZ, …) are suppressed from the index — they are never typed in a deck — so they no longer pollute completions or pass as valid keywords. Hover surfaces a Deck-name alias of <family> line.
  • Honour deck_name_regex for the open-ended summary-vector families (UDQ, tracer, water-cut-bucket mnemonics) via summary_name_patterns.json; PROBE deck_names, directional KRNUM/IMBNUM region variants, L-modifier vectors (WOPRL, LWWIR) and FIP region-set vectors (ROIP_ABC) are all recognised. Verified zero new false positives on the opm-tests corpus.

5. Code actions (quick-fixes)

  • Move keyword to correct section when section-validity fails: insert the keyword under the first valid sections[] entry that exists in the document.
  • Insert missing requires keyword with default record (using item defaults) under RUNSPEC.
  • Trim record to allowed length for the arity diagnostic.
  • Insert missing / terminator for the terminator diagnostics.
  • Uppercase / unindent keyword quick-fixes for the lowercase and column-1 diagnostics.
  • Fill defaults — replace a record line with one expanded to all items using the documented defaults (handy for WCONPROD/WCONINJE).

6. Smarter completions

  • Value completion inside records when the active item has options[] (e.g. COMPDAT STATUS), honouring an opm-flow.completion.stringValueStyle setting (bare/quoted/both).
  • UDQ / ACTIONX completion — inside a UDQ block the start of a statement offers the control words (ASSIGN/DEFINE/UNITS/UPDATE); inside a UDQ formula or ACTIONX condition the UDQ functions (SORTA, SUM, ABS, …) are offered and inserted with parentheses.
  • Snippet completion for keywords that scaffolds a full record line with tab-stop placeholders pre-filled from default/options[0] (e.g. typing WCONPROD inserts the keyword and a record line scaffold).
  • UDA-aware completion: when the active item has value_type: UDA, scan the deck for UDQ definitions and offer those names alongside numeric input.
  • Cross-keyword name completion: when the parameter is a well name (heuristic by item name WELL / GROUP), collect names from WELSPECS/GRUPTREE and offer them. #21

7. Outline & navigation

  • Outline tree view (opm-flow.outlineView): sections become folders, keywords become leaves; selection follows the cursor and clicking reveals the keyword. (Implemented as a custom TreeDataProvider rather than a DocumentSymbolProvider.)
  • Folding of section blocks and individual keyword blocks via a FoldingRangeProvider.
  • File navigationCtrl+click quoted paths on INCLUDE/IMPORT/RESTART/GDFILE, with PATHS alias ($NAME) expansion (DocumentLinkProvider).
  • Consider also exposing a DocumentSymbolProvider so the built-in Outline view / breadcrumbs / Ctrl+Shift+O work too (the tree view is separate from VS Code's native outline).
  • DefinitionProvider so Ctrl+click on a well name in WCONPROD jumps to the WELSPECS entry that introduced it.

8. Hover & sidebar additions

  • Sidebar docs panel that follows the cursor, with the active parameter row highlighted and a per-record table for multi-record keywords.
  • Configurable Type/Field/Metric/Lab/Default columns (opm-flow.columns.*, opm-flow.units.*) and an embedded vs. columnar layout (opm-flow.docs.layout).
  • Hover warns when a keyword is used in the wrong section and when it is on the diagnostics exclusion list.
  • Hover on a deck-name alias shows its opm-common family (Deck-name alias of <family>); hover on a UDQ control word or function shows its meaning/signature.
  • Show the dialect tag (E100 / E300 / Frontsim / OPM-only) next to the section list (requires emitting dialect from the index build).
  • Show requires / prohibits as a "Related keywords" line.
  • Convert dimension → a concrete unit string for the active unit system (FIELD/METRIC/LAB) using the manual's unit table; controlled by an opm-flow.unitSystem setting. (Today the three unit columns are shown/hidden but not resolved to one active system.)

9. Index build pipeline

  • Emit per-record / shape fields the diagnostics engine needs: records_meta, size_kind, size_count, templated, variadic_record, plus per-parameter value_type / dimension / options / units.
  • Emit the cross-keyword constraint fields requires / prohibits (loaded in load_opm_common_index, copied onto merged + synthesized entries via _attach_cross_keyword, emitted in the compact JSON; 41 requires, 34 prohibits).
  • Emit alias fields: expand_probe_deck_names expands PROBE deck_names and directional variants into index entries tagged with alias_of; synthesize_opm_only_entries now skips family/container schema names that only carry deck_names; deck_name_regex families ship as summary_name_patterns.json.
  • Still to emit: size_type (+ size{keyword,item}) and a dialect tag.
  • Add a fast smoke check: every keyword in opm-common either has a manual .fodt source or is marked synthesised (formalise the gap report).
  • Watch VSIX size as the new fields land; consider gzip-on-disk + decode-on-load if the compact JSON pushes past ~3 MB.

10. Documentation & examples

  • README.md Features list tracks shipped features (diagnostics, hover, sidebar, outline, folding, align/headers, file navigation, toggle comment, open PRT, generate reference).
  • Add a short section to DEVELOPMENT.md describing the opm-common JSON schema fields the extension consumes, so contributors know where to look.

Shipped beyond the original plan

These were not in the first cut of the plan but landed alongside it and are worth recording:

  • Richer diagnostics — unrecognised-keyword, wrong-section, per-record arity, missing per-record /, missing list/array close /, keyword-not-in-column-1, and lowercase-keyword checks; a opm-flow.diagnostics.excludedKeywords opt-out list with a default set.
  • Templated SUMMARY mnemonicsFTPRFTPRSEA style tokens resolve to the base template for hover/docs/diagnostics (templated, variadic_record, optional_body).
  • UDQ / ACTIONX sub-language — recognition of the UDQ expression language and ACTIONX action blocks: tmLanguage highlighting for control words, UDQ functions, expression operators and the ACTIONX/ENDACTIO delimiters; hover and completion for UDQ control words/functions; two diagnostics (a UDQ statement that doesn't start with a control word; an ACTIONX block left unclosed by ENDACTIO); and a dedicated three-column alignment for UDQ expression blocks (control word and variable both left-aligned, expression right-aligned, division-safe, comment/blank-line tolerant).
  • Corpus false-positive harnessvscode-extension/src/corpus.test.ts runs the diagnostics engine over the OPM/opm-tests decks (known-good) to catch regressions; documented in DEVELOPMENT.md.
  • Formatting commands — Align Record Columns, Add Column Headers (idempotent, heading-aware), Toggle Line Comment.
  • Open PRT File command + Ctrl+Alt+P keybinding.
  • opm-flow.additionalFileExtensions setting to retag arbitrary extensions as opm-flow.
  • Security hardening of the webview/hover and CI (no localResourceRoots, CSP, no command: links).

Out of scope (call out, don't implement here)

  • A full deck parser. We stay line-oriented and lean on the index for correctness signals.
  • Running OPM Flow as a language server. Static analysis only.

Progress rollup

Plan item Status
1. requires / prohibits diagnostics ✅ done
2. Positional value_type checking ✅ done (numeric/quoted checks; enum-options check intentionally dropped)
3. Multi-record records[] schema ✅ done
4. deck_names aliases ✅ done (aliases resolved + alias_of hover; PROBE/region/regex families expanded; container schema names suppressed)
5. Code actions / quick-fixes ⬜ not started
6. Smarter completions 🟡 value/options + UDQ control-word/function completion done; snippets / UDA / well-name pending
7. Outline & navigation 🟡 tree view + folding + file links done; DocumentSymbolProvider & DefinitionProvider pending
8. Hover & sidebar 🟡 panel + columns/layout + section/exclusion warnings + alias/UDQ hover done; dialect tag / related-keywords / unit-system conversion pending
9. Index build pipeline 🟡 shape, per-item, requires/prohibits & alias (deck_names) fields done; size_type/dialect fields pending
10. Documentation 🟡 README tracked; DEVELOPMENT.md schema notes pending

Suggested staging (revised)

  1. Phase A — index + cross-keyword diagnostics: items 9 (cross-keyword fields), 1, 2, 4. Purely additive; unlocks the highest-value diagnostics. (✅ complete: 1, 2 and 4 done; item 9 has only size_type/dialect left.)
  2. Phase B — code actions & hover polish: items 5, 8 (dialect tag, related keywords, unit-system conversion).
  3. Phase C — navigation & completions: items 6 (snippets/UDA/well-name), 7 (DefinitionProvider, optional DocumentSymbolProvider).

Links

  • opm-common keywords directory: opm-common/opm/input/eclipse/share/keywords/{000_Eclipse100,001_Eclipse300,002_Frontsim,900_OPM}/
  • Index builder: scripts/build_keyword_index.py
  • Diagnostics module: vscode-extension/src/analysis.ts
  • Extension entry point: vscode-extension/src/extension.ts
  • Outline builder: vscode-extension/src/outline.ts
  • Reference manual submodule: opm-reference-manual/
  • Upstream opm-common: https://github.com/OPM/opm-common

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions