Skip to content

Add migration test harness for COBOL-to-Java validation - #217

Open
devin-ai-integration[bot] wants to merge 3 commits into
mainfrom
devin/1781785212-migration-test-harness
Open

devin-ai-integration[bot] wants to merge 3 commits into
mainfrom
devin/1781785212-migration-test-harness

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Jun 18, 2026

Copy link
Copy Markdown

Summary

Adds a comprehensive migration test harness to validate functional equivalence between the COBOL CardDemo system and its Java replacement. Four deliverables:

TEST_STRATEGY.md — Documents four testing approaches:

  • Golden-file tests (capture COBOL outputs as JSON, assert Java matches)
  • Differential tests (side-by-side execution model)
  • Batch reconciliation (record counts, numeric sums, checksums)
  • Contract tests (record layouts, LRECL, key positions, cross-reference invariants)

golden-files/ — JSON golden references for all 9 ASCII data files, parsed from copybook PIC clause definitions. Each file documents field names, types, and semantics via golden-files/README.md.

test-harness/ — Python project with:

  • harness/parser.py — Fixed-width file parser using PIC clause specs. Handles zoned-decimal sign encoding ({ABCDEFGHI → +0–9, }JKLMNOPQR → -0–9), implied decimal (V99), and all 9 copybook layouts registered in LAYOUT_REGISTRY.
  • harness/comparator.py — Field-by-field diff with configurable numeric tolerance. Supports both positional (compare_records) and key-based (compare_by_key) matching.
  • harness/reconciliation.py — Record count validation, numeric sum checks, and referential integrity (xref→card, xref→account, xref→customer, tcatbal→account, dailytran→xref). run_data_integrity_checks() runs the full suite.
  • 76 passing tests covering unit tests, layout validation, and live data integrity checks against app/data/ASCII/.

RECONCILIATION_CHECKS.md — Documents every JCL batch job: what it reads, writes, what reconciliation checks should pass (record counts, totals, referential integrity), and business rules enforced (e.g., POSTTRAN rejection codes 100–103, INTCALC interest formula (balance * rate) / 1200).

Link to Devin session: https://partner-workshops.devinenterprise.com/sessions/ef1ee1a1c17c4e2dacf4baa2802ffe66
Requested by: @bsmitches


Open in Devin Review

- TEST_STRATEGY.md: testing approach (golden-file, differential, batch
  reconciliation, contract tests)
- golden-files/: JSON golden references for all 9 ASCII data files parsed
  from copybook PIC clause definitions
- test-harness/: Python project with copybook parser, field-by-field
  comparator, and reconciliation check utilities (76 tests passing)
- RECONCILIATION_CHECKS.md: I/O documentation and checks for all JCL
  batch jobs
@devin-ai-integration

Copy link
Copy Markdown
Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

devin-ai-integration[bot]

This comment was marked as resolved.

- Wrap decode_signed_numeric in try/except to handle space-filled
  signed numeric fields gracefully (returns stripped string on error)
- Fix build-backend from nonexistent setuptools.backends._legacy to
  setuptools.build_meta

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

Open in Devin Review

Comment on lines +13 to +14
"ACCT-ADDR-ZIP": "A000000000",
"ACCT-GROUP-ID": ""

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚩 All 50 accounts share ACCT-ADDR-ZIP='A000000000' while ACCT-GROUP-ID is empty

In the golden file golden-files/acctdata.json, every account has ACCT-ADDR-ZIP: "A000000000" and ACCT-GROUP-ID: "". The value A000000000 matches one of the disclosure group IDs in golden-files/discgrp.json, which could suggest the data was placed in the wrong field. However, verified against the raw data and the COBOL copybook CVACT01Y.cpy:15-16 — the parser correctly assigns position 102-111 to ACCT-ADDR-ZIP and 112-121 to ACCT-GROUP-ID. The sample data genuinely has this layout. The INTCALC business logic at RECONCILIATION_CHECKS.md:234 documents a DEFAULT fallback when the group ID is missing, so this may be intentional test data design. Worth confirming with the data generator if this is by design.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed by inspecting the raw data at byte positions 103-122: A000000000 is genuinely at offset 102 (ACCT-ADDR-ZIP) and positions 112-121 (ACCT-GROUP-ID) are space-filled across all 50 records. The parser is correct per the copybook layout. This appears to be intentional sample data design — the INTCALC logic falls back to the DEFAULT disclosure group when the group ID is empty.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants