fix(ledger): duplicate-input rejection only at PV>=9, matching Haskell Set decode (#759)

Phase-1 false positive on a confirmed mainnet Babbage block (epoch 482,
PV8, slot 123,728,795): tx 5ca83e21… rejected with 'Duplicate input in
transaction' — its spend-inputs array carried ab2829f03f…#1 twice. On-chain
(Haskell-accepted), so dugite's rejection was wrong.

Haskell cardano-ledger-binary decodeSet at PV<9 (Alonzo/Babbage) routes
through Set.fromList ∘ decodeList, which SILENTLY drops duplicates (no
DuplicateInput predicate exists in BabbageUtxoPredFailure). Only PV>=9
(Conway) uses decodeSetEnforceNoDuplicates and hard-fails. dugite's
Rule 1b fired at all PVs.

Fix (one line, phase1.rs Rule 1b): gate the DuplicateInput error on
params.protocol_version_major >= 9. Conway+ rejection is unchanged
(test_duplicate_inputs_rejected_at_conway_pv9 + proptests still pass);
PV8 now accepts, pinned by the real on-chain tx fixture
(test_mainnet_babbage_duplicate_input_5ca83e21_no_false_positive).
1574/1574 ledger tests.

Author: michaeljfazio

crates/dugite-ledger/src/validation/fixtures/tx-5ca83e21.hex

@@ -0,0 +1 @@
+84ab0083825820ab2829f03f185af3eb048e1cd256899c7ddb575a112fcfe41a324e94d21707aa01825820ab2829f03f185af3eb048e1cd256899c7ddb575a112fcfe41a324e94d21707aa018258203bd13603e5e051f0b501da15260d26ad5948e4a3db54f4e3038416edf4f4d95e000183a300583901919e5c413a2e3d75a9b8d977d0c9818703a0b23d9ad87c29cf800de4d714cdab9f419c2c3d8b414a936a02fbeac8fe48dc841ba34e1e51c2011a00cce451028201d818582fd8799fd8799fd8799f5820ab2829f03f185af3eb048e1cd256899c7ddb575a112fcfe41a324e94d21707aaff01ffff825839015694f676db64afc70ff5b1cb0b3ce1b33d3d5b77a2a46a8f327dc6d35089ac70dc53a538a2052897ecdc45638776889783a46a2274fe59711ad57ce924825839015694f676db64afc70ff5b1cb0b3ce1b33d3d5b77a2a46a8f327dc6d35089ac70dc53a538a2052897ecdc45638776889783a46a2274fe59711a009549e0021a00034ca0031a07600fb3081a075ff3630b58206bc59041a3746e726ecc76cb6071a413c629644ab89bd91d67780960963d23120d818258203bd13603e5e051f0b501da15260d26ad5948e4a3db54f4e3038416edf4f4d95e000e82581c5694f676db64afc70ff5b1cb0b3ce1b33d3d5b77a2a46a8f327dc6d3581c19524fed350e6a9e928c231f51f737388eaf3f8d547732f68e02bab110825839015694f676db64afc70ff5b1cb0b3ce1b33d3d5b77a2a46a8f327dc6d35089ac70dc53a538a2052897ecdc45638776889783a46a2274fe59711a00895440111a000f42401281825820e92ac620bf095094d58a19616d1b6debb9b1cf305870264b1a58446c51d7f4b000a20083825820ca7c3c7870c113704ce57e6db6fd385e43e7ea399848cc682cc86b0a2965e1565840835b49da4857ef54b0d35fa66ae68df7722d7e241e18e111e287fb9b7aefa31c975dd61c8f032989937a7a49c3f4a9894b06e50a0e3fb05d07373257db08fd01825820bf8cca1c95f23d3864c21ce6301508644b883dfcf46ecc739ca2df382aaad2ff5840af72a39b186570860a16f88822cffb6e2ee90df35ef55350559853358a5d66508fdea32106be1a95a0ca40ac05879f487837b4ac1fd7bbaa569ed65686b6010182582037fb59abf680ba541f1664ce98cd20d5b730875fef834b4403fc483727db96d9584059fc744e3ac62b160547f6e366bf9a36ef912d2d5ba26f29921f241b72ceb237f3dff914e85eae5061727bd50200533b1403c9900c53b136000e7450100c7e000581840001d87b80821a00029d491a03b6aaadf5f6

crates/dugite-ledger/src/validation/phase1.rs

@@ -4,7 +4,7 @@
 //! execution. Each numbered rule corresponds to a distinct ledger invariant:
 //!
 //! - Rule 1  — at least one input
-//! - Rule 1b — no duplicate inputs
+//! - Rule 1b — no duplicate inputs (Conway+ PV≥9 only; Haskell silently dedups at PV<9)
 //! - Rule 1c — auxiliary data hash / auxiliary data consistency
 //! - Rule 1d — era gating (Conway-only certs/governance in pre-Conway eras)
 //! - Rule 2  — all inputs exist in the UTxO set
@@ -622,11 +622,31 @@ pub(super) fn run_phase1_rules(
 
     // ------------------------------------------------------------------
     // Rule 1b: No duplicate inputs
+    //
+    // Haskell semantic is protocol-version gated:
+    //
+    // PV < 9 (Alonzo/Babbage): `decodeSet` routes through the lenient path
+    //   (`Set.fromList`), which silently deduplicates.  No `BabbageUtxoPredFailure`
+    //   constructor for duplicate inputs exists, so Haskell accepts such txs.
+    //   Real mainnet Babbage blocks contain transactions with duplicate spend
+    //   inputs encoded in a plain CBOR array (e.g. tx 5ca83e21… at epoch 484,
+    //   slot 123728795, PV8) — Haskell silently dedups and accepts.
+    //
+    // PV >= 9 (Conway+): `decodeSetEnforceNoDuplicates` hard-fails at the
+    //   binary layer.  We mirror that by surfacing `DuplicateInput` at
+    //   Phase-1 time (the net effect is the same rejection).
+    //
+    // Reference: `cardano-ledger-binary` `decodeSet` / `decodeSetEnforceNoDuplicates`
+    //   (Cardano.Ledger.Binary.Decoding.Coders), and the absence of a
+    //   DuplicateInput constructor in `AlonzoUtxoPredFailure` / `BabbageUtxoPredFailure`.
     // ------------------------------------------------------------------
     {
         let mut seen = HashSet::new();
         for input in &body.inputs {
-            if !seen.insert(input) {
+            // PV < 9 (Alonzo/Babbage): Haskell `Set.fromList` silently dedups —
+            // no rejection.  PV >= 9 (Conway+): hard-fail mirrors
+            // `decodeSetEnforceNoDuplicates`.
+            if !seen.insert(input) && params.protocol_version_major >= 9 {
                 errors.push(ValidationError::DuplicateInput(input.to_string()));
             }
         }
@@ -2869,20 +2889,110 @@ mod tests {
     }
 
     // -----------------------------------------------------------------------
-    // Test 31 — Rule 1b: duplicate inputs rejected
+    // Test 31 — Rule 1b: duplicate inputs rejected at Conway PV9+
+    //
+    // Haskell `decodeSetEnforceNoDuplicates` (PV >= 9) hard-fails on duplicates.
+    // Dugite mirrors this at Phase-1 time.  `mainnet_defaults()` has PV9.
     // -----------------------------------------------------------------------
     #[test]
-    fn test_duplicate_inputs_rejected() {
+    fn test_duplicate_inputs_rejected_at_conway_pv9() {
         let (utxo_set, mut tx, input) = make_valid_tx();
         // Add the same input a second time.
         tx.body.inputs.push(input.clone());
-        let params = ProtocolParameters::mainnet_defaults();
+        let params = ProtocolParameters::mainnet_defaults(); // PV9
         let errors = validate_transaction(&tx, &utxo_set, &params, 100, 300, None).unwrap_err();
         assert!(
             errors
                 .iter()
                 .any(|e| matches!(e, ValidationError::DuplicateInput(_))),
-            "expected DuplicateInput, got {errors:?}"
+            "expected DuplicateInput at PV9, got {errors:?}"
+        );
+    }
+
+    // -----------------------------------------------------------------------
+    // Test 31b — Rule 1b: duplicate inputs silently accepted at Babbage PV8
+    //
+    // Haskell `decodeSet` at PV < 9 routes through `Set.fromList` which
+    // silently deduplicates.  `BabbageUtxoPredFailure` has no DuplicateInput
+    // constructor.  Real mainnet tx 5ca83e21… (epoch 484, slot 123728795,
+    // PV8) has body key 0 = array(3) with the same TxIn listed twice; it was
+    // accepted by cardano-node 8.x and is on-chain.
+    // -----------------------------------------------------------------------
+    #[test]
+    fn test_duplicate_inputs_accepted_at_babbage_pv8() {
+        let (utxo_set, mut tx, input) = make_valid_tx();
+        // Add the same input a second time — simulating the PV8 wire encoding.
+        tx.body.inputs.push(input.clone());
+        let mut params = ProtocolParameters::mainnet_defaults();
+        params.protocol_version_major = 8; // Babbage
+                                           // With a duplicate input and PV8, Phase-1 must NOT emit DuplicateInput.
+                                           // (Other errors may fire — the point is DuplicateInput is absent.)
+        let result = validate_transaction(&tx, &utxo_set, &params, 100, 300, None);
+        let no_dup_error = match &result {
+            Ok(()) => true,
+            Err(errors) => !errors
+                .iter()
+                .any(|e| matches!(e, ValidationError::DuplicateInput(_))),
+        };
+        assert!(
+            no_dup_error,
+            "DuplicateInput must not fire at PV8 (Babbage), got {result:?}"
+        );
+    }
+
+    // -----------------------------------------------------------------------
+    // Test 31c — Rule 1b: real mainnet Babbage tx 5ca83e21 with duplicate
+    //             spend input accepted (issue #759 regression pin)
+    //
+    // tx 5ca83e216eb4fce8e907ed3597bd290261136ae97fc4cd7fbd5eadf9bbedf09f
+    // mainnet epoch 484, slot 123728795, block 10294413 (PV8 = Babbage).
+    // Body key 0 = plain array(3): [ab2829f0…#1, ab2829f0…#1, 3bd13603…#0]
+    // The same TxIn `ab2829f03f…#1` appears twice.  Haskell accepted it
+    // (it is on-chain); dugite must not reject with DuplicateInput.
+    //
+    // Note: this test decodes the raw CBOR and checks that DuplicateInput is
+    // absent; it does NOT reconstruct a full UTxO environment, so Phase-1
+    // may emit BadInputs/ValueNotConserved.  The critical invariant is that
+    // DuplicateInput is NEVER in the error list when PV < 9.
+    // -----------------------------------------------------------------------
+    const TX_5CA83E21_HEX: &str = include_str!("fixtures/tx-5ca83e21.hex");
+
+    #[test]
+    fn test_mainnet_babbage_duplicate_input_5ca83e21_no_false_positive() {
+        let s = TX_5CA83E21_HEX.trim();
+        let bytes: Vec<u8> = (0..s.len())
+            .step_by(2)
+            .map(|i| u8::from_str_radix(&s[i..i + 2], 16).expect("valid hex"))
+            .collect();
+        // Decode as era 5 (Babbage)
+        let tx = dugite_serialization::decode::decode_transaction(5, &bytes)
+            .expect("decode real mainnet Babbage tx 5ca83e21");
+
+        // Verify the wire-level duplicate is preserved by the decoder
+        assert_eq!(
+            tx.body.inputs.len(),
+            3,
+            "wire has array(3); decoder must preserve physical element count"
+        );
+        assert_eq!(
+            tx.body.inputs[0], tx.body.inputs[1],
+            "first two inputs must be identical (wire duplicate)"
+        );
+
+        // Validate against an empty UTxO (BadInputs expected, DuplicateInput forbidden)
+        let empty_utxo = UtxoSet::new();
+        let mut params = ProtocolParameters::mainnet_defaults();
+        params.protocol_version_major = 8; // PV8 = Babbage
+        let result = validate_transaction(&tx, &empty_utxo, &params, 123_728_795, 500, None);
+        let has_dup_error = match &result {
+            Ok(()) => false,
+            Err(errors) => errors
+                .iter()
+                .any(|e| matches!(e, ValidationError::DuplicateInput(_))),
+        };
+        assert!(
+            !has_dup_error,
+            "DuplicateInput must not fire for PV8 Babbage tx 5ca83e21 (issue #759): {result:?}"
         );
     }
 

reports/issue-759-duplicate-input.md

@@ -0,0 +1,155 @@
+# Issue #759 — Phase-1 False Positive: Duplicate Input in Babbage Tx
+
+## Summary
+
+**Classification:** SMALL+SAFE one-liner fix, strong regression tests, zero behavioral change for Conway+.
+
+**Root cause:** Phase-1 Rule 1b fired unconditionally for all protocol versions. Haskell only enforces uniqueness at PV >= 9 (Conway+). The rule must be gated on `protocol_version_major >= 9`.
+
+---
+
+## Evidence
+
+### Transaction under investigation
+
+```
+tx hash : 5ca83e216eb4fce8e907ed3597bd290261136ae97fc4cd7fbd5eadf9bbedf09f
+block   : 10294413
+epoch   : 484  (PV8 = Babbage)
+slot    : 123,728,795
+```
+
+### Wire structure of body key 0 (spend inputs)
+
+Decoded from Koios MAINNET REST (`/api/v1/tx_cbor`):
+
+```
+Key 0 (inputs): plain array(3) — NO tag 258
+  [0]  ab2829f03f185af3eb048e1cd256899c7ddb575a112fcfe41a324e94d21707aa  idx=1
+  [1]  ab2829f03f185af3eb048e1cd256899c7ddb575a112fcfe41a324e94d21707aa  idx=1   ← IDENTICAL to [0]
+  [2]  3bd13603e5e051f0b501da15260d26ad5948e4a3db54f4e3038416edf4f4d95e  idx=0
+
+Key 13 (collateral): array(1)
+  [0]  3bd13603e5e051f0b501da15260d26ad5948e4a3db54f4e3038416edf4f4d95e  idx=0
+
+Key 18 (reference_inputs): array(1)
+  [0]  e92ac620bf095094d58a19616d1b6debb9b1cf305870264b1a58446c51d7f4b0  idx=0
+```
+
+`ab2829f03f...#1` appears **twice in the spend inputs**. This is not a collateral/reference overlap — it is a genuine wire-level duplicate in a plain (non-tag-258) CBOR array.
+
+---
+
+## Haskell Behavior (PV8, Babbage)
+
+### Reference: `cardano-ledger-binary` `decodeSet`
+
+At PV < 9, `decodeSet` routes through the **lenient path**:
+
+```haskell
+-- cardano-ledger-binary Cardano.Ledger.Binary.Decoding.Coders
+-- PV < 9:  Set.fromList <$> decodeList decoder    (silent dedup)
+-- PV >= 9: decodeSetEnforceNoDuplicates (hard fail)
+```
+
+`Set.fromList [A, A, B]` = `{A, B}` — the duplicate is silently dropped. No exception, no predicate failure.
+
+### Reference: `BabbageUtxoPredFailure` constructors
+
+```haskell
+-- eras/babbage/impl/src/Cardano/Ledger/Babbage/Rules/Utxo.hs
+data BabbageUtxoPredFailure era
+  = AlonzoInBabbageUtxoPredFailure !(AlonzoUtxoPredFailure era)
+  | BabbageNonDisjointRefInputs !(Set TxIn)  -- PV9-10 only
+```
+
+`AlonzoUtxoPredFailure` has **no** `DuplicateInput` or `DuplicateInputs` constructor. The complete list of Alonzo UTXO failures is: OutsideValidityIntervalUTxO, InputSetEmptyUTxO, FeeTooSmallUTxO, BadInputsUTxO, ValueNotConservedUTxO, OutputTooSmallUTxO, OutputBootAddrAttrsTooBig, MaxTxSizeUTxO, WrongNetwork, WrongNetworkWithdrawal, OutputTooBigUTxO, InsufficientCollateral, ScriptsNotPaidUTxO, ExUnitsTooBigUTxO, CollateralContainsNonADA, WrongNetworkInTxBody, OutsideForecast, TooManyCollateralInputs, NoCollateralInputs.
+
+**Conclusion:** Haskell silently accepts a Babbage tx with duplicate spend inputs. No predicate failure is produced. The duplicate is eliminated by `Set.fromList` before any validation sees it.
+
+---
+
+## Root Cause in Dugite
+
+**File:** `crates/dugite-ledger/src/validation/phase1.rs`
+
+**Location:** lines 624-633 (before fix)
+
+```rust
+// Rule 1b: No duplicate inputs   ← NO era/PV gate
+{
+    let mut seen = HashSet::new();
+    for input in &body.inputs {
+        if !seen.insert(input) {
+            errors.push(ValidationError::DuplicateInput(input.to_string()));
+        }
+    }
+}
+```
+
+The check runs unconditionally for ALL protocol versions. For the Babbage tx at PV8 with `ab2829f03f...#1` appearing twice, dugite emits `DuplicateInput("ab2829f03f…")` and rejects the tx. Haskell accepts it.
+
+**Note:** The stake-distribution deduplication in `eras/common.rs:184-189` (the `seen_inputs` HashSet filter) is already correct — it silently deduplicates before consuming UTxOs. That code path only needed the Phase-1 validation gate removed.
+
+---
+
+## The Fix
+
+**File:** `crates/dugite-ledger/src/validation/phase1.rs`, Rule 1b (lines 624-653 after fix)
+
+**Change:** Add `&& params.protocol_version_major >= 9` guard on the error push:
+
+```rust
+{
+    let mut seen = HashSet::new();
+    for input in &body.inputs {
+        // PV < 9 (Alonzo/Babbage): Haskell `Set.fromList` silently dedups —
+        // no rejection.  PV >= 9 (Conway+): hard-fail mirrors
+        // `decodeSetEnforceNoDuplicates`.
+        if !seen.insert(input) && params.protocol_version_major >= 9 {
+            errors.push(ValidationError::DuplicateInput(input.to_string()));
+        }
+    }
+}
+```
+
+This is a **one-line semantic change** with zero risk to Conway+. Conway/Dijkstra txs with duplicate inputs still get `DuplicateInput`. Alonzo/Babbage txs with wire-duplicate inputs are silently accepted (matching Haskell).
+
+---
+
+## Tests Added
+
+Three new tests in `crates/dugite-ledger/src/validation/phase1.rs`:
+
+| Test | Purpose |
+|------|---------|
+| `test_duplicate_inputs_rejected_at_conway_pv9` | Conway (PV9) still rejects — renamed from old Test 31 |
+| `test_duplicate_inputs_accepted_at_babbage_pv8` | Babbage (PV8) accepts duplicate inputs — negative control |
+| `test_mainnet_babbage_duplicate_input_5ca83e21_no_false_positive` | Real mainnet tx 5ca83e21 with fixture pinned at `crates/dugite-ledger/src/validation/fixtures/tx-5ca83e21.hex` |
+
+All 3 new tests PASS. Full suite: 1574/1574 pass, 0 failures, 6 skipped (pre-existing skips).
+
+**CI checks:**
+- `cargo clippy -p dugite-ledger --all-targets -- -D warnings`: CLEAN
+- `cargo fmt --all -- --check`: CLEAN
+- `cargo nextest run -p dugite-ledger`: 1574/1574 PASS
+
+---
+
+## Files Changed
+
+| File | Change |
+|------|--------|
+| `crates/dugite-ledger/src/validation/phase1.rs` | Rule 1b: gate `DuplicateInput` on `pv >= 9`; add 3 tests |
+| `crates/dugite-ledger/src/validation/fixtures/tx-5ca83e21.hex` | Real mainnet Babbage tx CBOR fixture (issue #759 pin) |
+
+---
+
+## Risk Assessment: SMALL+SAFE
+
+- **Blast radius:** One `&&` condition on one `errors.push()` call.
+- **Conway+ behavior:** Unchanged. PV9 test still fires and passes.
+- **Pre-Conway behavior:** Duplicate inputs now silently accepted (Haskell-exact).
+- **Downstream impact:** `eras/common.rs` `seen_inputs` dedup was already correct — this fix only removes the false Phase-1 rejection.
+- **No SNAPSHOT_VERSION bump needed** — no ledger state struct changes.
+- **Recommended placement:** v2.0.6 (already on `main`).