Default output is separable without graph structure — decorrelate amounts & descriptions, or document

[Leonardasvekrikas-source]

What the generator gets right

The fraud injection is genuinely relational: cyclic money-laundering rings with configurable
depth (4–7 hops), as the README describes, and the cycles are
real — we independently verified that every declared ring closes with actual edges (10/10
closed on seed-42 output, 0 accounts reused across rings:
findings/results.md "Ring closure"). This is
exactly the kind of structure a graph-based fraud benchmark needs. The issue below is not
that the structure is fake; it is that a default makes it unnecessary.

The precise flaw

Every ring transaction is stamped with the hardcoded default amount = 9999.00
(typologies.py:52), while normal edges draw from uniform(10, 500) (generator.py:198).
The consequence, measured on generated output:

• The label collapses to a one-feature threshold. On 2,000 accounts (scale 0.0002,
  seeded via our adapter): is_fraud == (max_out_amount >= 500) → 0/2000 mismatches.
  Written up at findings/results.md L19–L22; re-derived fresh on
  2026-07-09 with the same result (rows=2000 mismatches=0).
• Size of the class gap (re-derived on the seed-42 data, same check): the largest
  legitimate max_out_amount is 499.99 and the smallest fraud value is 9999.0 — a
  ~20× separation with no near-threshold region at all. There are no hard cases: every
  fraud account sits exactly at the hardcoded default, so the decision boundary can be drawn
  anywhere in (500, 9999) with zero error.
• Consequence for benchmarks: the one-line threshold scores F1/AUC/MCC = 1.000; a LightGBM
  model can only tie it (findings/results.md L12–L40).
• The bundled risk_score carries no detectable signal: AUC 0.438 full-set / 0.473
  test-split — indistinguishable from chance at n = 48 positives
  (findings/results.md L23–L30).
• Root cause is the constant-amount injection described in Expose amount ranges and difficulty/overlap in config #26; no Config/CLI knob
  changes it. First observed at
  findings/what-broke.md L59–L78.
• The description field is a second, disclosed channel with the same effect. The README
  says fraud rings carry "suspicious transaction descriptions" (disclosed by design, not a
  hidden flaw); measured on seed-42 output, the two hardcoded vocabularies never overlap —
  10 unique normal vs 8 unique fraud description values, overlap = 0 — so text alone also
  separates the classes (findings/results.md "Description channel"). It reinforces the same point: default output is not benchmark-grade without
  customization.

Why it matters — by the README's own use cases

The README positions the generator for "training and benchmarking graph-based fraud detection
models" and lists benchmarking AML detection algorithms and GNN training/evaluation among the
use cases (accessed 2026-07-09). A dataset
where a one-line threshold scores a perfect 1.0 cannot exercise a GNN: the injected 4–7-hop
structure is unnecessary for detection, because it is perfectly confounded with the constant
amount. A GNN benchmarked on this data can score 1.0 without using the graph at all, and a
tabular baseline will tie it — the benchmark cannot distinguish "understands relational fraud
patterns" from "found the constant." So the default output defeats the stated use case. (To be
precise about what we did not show: we did not train a GNN or claim graphs are useless —
only that no graph method can be required for a label already perfectly recoverable
without one.)

Proposed fix — two options

1. Decorrelate both channels from ring membership. Draw ring-transaction amounts from a
   distribution that overlaps normal transactions (or expose the ranges — see Expose amount ranges and difficulty/overlap in config #26),
   and source
   ring-edge descriptions from the same (or an overlapping) vocabulary as normal edges, so
   that detecting the rings requires the graph structure rather than reading either
   feature. This is the fix that makes the GNN/AML benchmarking use case real, and the ring
   construction needed for it already exists. One implication we note but did not measure:
   since the generator offers optional embedding vectors over the description text, disjoint
   description vocabularies would propagate the leak into exactly the vector features a GNN
   benchmark would consume.
2. Minimally: document the consequence. The README already lists the values (L172:
   10–500 normal, 9999 fraud); what it does not state is the implication — that default
   output is separable across both the amount and description channels and is not
   benchmark-grade without customization. One or two sentences in the README close that
   gap. Zero code change; honest expectations.

We recognize a bank may deliberately avoid publishing realistic fraud typologies and amount
distributions for abuse-prevention reasons — if that is the constraint here, option 2 is the
right minimal move.

---

Context: this came out of a small independent teardown that wired several SantanderAI tools
into one fraud-decision pipeline: https://github.com/Leonardasvekrikas-source/santander-ai-governance-teardown