feature: golden-dataset COBOL/JS serverless parity harness - #268
Open
hannahhkyme wants to merge 2 commits into
Open
hannahhkyme wants to merge 2 commits into
hannahhkyme wants to merge 2 commits into
Conversation
Add captured COBOL evidence, fixed-width fixtures, and JavaScript parity handlers with unit tests. Co-Authored-By: hannah.kyme <hannahhkyme@gmail.com>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
Co-Authored-By: hannah.kyme <hannahhkyme@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a reproducible parity harness that proves a JS serverless conversion of the portfolio programs behaves like the COBOL, per case, including error paths — plus the reference handlers themselves, since no JS conversion existed on
mainto compare against (golden/README.mddocuments the whole pipeline).49 cases, all passing. 39 expected results are captured from real COBOL runs (GnuCOBOL 3.1.2.0, BDB indexed handler) and committed under
golden/cobol-run/; the 10 PORTTRAN cases arederived: truebecausePORTTRAN.cblcan't compile here (it copies aPORTRECcopybook absent fromsrc/copybookand referencesPORT-TOTAL-UNITS/PORT-TOTAL-COST, defined nowhere in the repo). Each derived golden carries aderivationfield quoting the paragraphs it came from, and the report marks themDERIVED EXPECTED.Flow, wired into
npm testso divergence breaks the build:Both sides reduce to one canonical form (
serverless/src/parity/canonical.js): padding trimmed, COMP-3 decoded to exact decimal strings compared by value not bytes (BigIntthroughout — no float money anywhere), clock-stamped fields tokenized to@RUNDATE/@RUNSTAMP. The JS side is fed the same fixed-width files the COBOL was fed rather than the case JSON, so the two sides can't differ in what they were asked to process — only in how. Failures name the exact path:finalState[1].createDate: expected(COBOL)="@RUNDATE" actual(JS)="BROKEN".Layouts are transcribed from
PORTFLIO.cpy(148 bytes, 18-bytePORT-KEY) andTRNREC.cpy; the simplified layout intest-data-specs.mdis used only for its list of canonical cases. Determinism comes from a fixedrandomSeed(123456789) ingolden/config/golden-config.jsonfeeding a seeded LCG, mirroring TSTGEN00's RANDSEED.The goldens encode what the programs do, not what they claim
Several captured behaviors contradict the source comments and the docs; the handlers reproduce them deliberately (all listed in
golden/README.md):PORTVALDrejects every portfolio ID, including a well-formedPORT0001: after thePORTprefix check it moves the 4 digits into anX(10)work field and applies theNUMERICclass test, which the 6 trailing spaces can never pass. Implemented as that same two-step so the quirk is visible rather than hardcoded.PORTUPDTactionVinheritsMOVE <X(50)> TO S9(13)V99semantics:...2500075becomes2500075.00, not25000.75(sharedcobol-move.js, which throws on non-digit input rather than guessing GnuCOBOL's junk handling).PORTTRANnever performs2200-UPDATE-POSITIONS. Buy/sell/transfer/fee, the insufficient-units check and theAUDPROCaudit trail are unreachable dead code — the program reads, validates, counts, and never rewrites the file it opens I-O. So the requested "insufficient units on sell" case (TRN-003) passes validation and changes nothing, and the JS handler implements no position keeping: doing so would make the harness green against behavior the COBOL doesn't have.PORTVALDamount validation via the same MOVE can never fall outside the documented range; account validation runs against the wholeX(50), so trailing spaces fail where 50-digit zero-padding passes;PORTDELdoesn't validate reason codes and leavesAUD-FILLERatINITIALIZEd low-values (27 NUL bytes in the golden audit record);PORTUPDTdoesn't restampPORT-LAST-MAINT.Notes for review
src/is untouched. Capture needs three patches (fixed-format comment columns, a duplicate-copybook name clash in PORTADD, anACCEPT ... FROM TIME STAMPGnuCOBOL rejects) applied to copies underbuild/src/; seegolden/patches/README.md.golden/cobol/GLDLOAD.cblloads seeds viaACCESS MODE IS RANDOM— withSEQUENTIALany out-of-key-order seed silently failed with status 21, which is exactly what theRED-001/RED-004browse-order cases need. The capture script now hard-fails if loaded/dumped counts don't match the manifest.npm run parityandnpm testneed only Node 20+ and the committed fixtures; GnuCOBOL is required only to re-capture (npm run golden:capture).golden/reports/parity-report.txtin TSTVAL00's layout (132-col banner, fixed-width detail lines, total/passed/failed/success-rate); it's gitignored as a build artifact.Devin-Org: engineering
Link to Devin session: https://app.devin.ai/sessions/434efed784834b52aa12d9d7da4d8e4f
Requested by: @hannahhkyme
Devin Review