Enable structural dedup by default with 2-day date window - #145
Merged
Conversation
When the same real transaction is filed from two sources that describe it with different narrations, cmd/beanfile previously treated the records as unrelated and wrote both as active entries: the skip-capable exact match keys on narration, and the transfer-aware structural rule was gated behind a date window that defaulted to disabled. Turn the structural rule on out of the box by defaulting the date window to 2 days. Such near-duplicates now surface for review (written commented-out) instead of being silently written twice. The structural rule stays review-only — it is non-transitive and must not auto-skip — so this never drops data; a same-amount coincidence at worst yields a review marker. The window remains fully configurable: an explicit date_window_days = 0 at the section or override level opts back out, and narration continues to participate in the exact match so genuinely identical records still skip. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017oc8DY7Sgyd42zSaMZCKrU
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Enable the structural-dedup rule out of the box by introducing a built-in default date window of 2 days. This allows duplicate transactions (same date, same postings, different narration) to surface for review rather than being silently skipped. An explicit
date_window_days = 0in configuration opts out of the rule.Changes
Built-in default: Introduce
defaultDateWindowDays = 2constant inroute.go. The structural-dedup window now defaults to 2 days instead of 0 (disabled), enabling the feature automatically.Resolution logic: Update
resolveDateWindow()to returndefaultDateWindowDayswhen no override or section-level value is declared, implementing proper inheritance: override > section > built-in default.Test coverage: Add two integration tests in
main_test.go:TestRun_StructuralDedupCommentsByDefault: Verifies that duplicate transfers with different narrations are filed as commented-out transactions when the default window applies.TestRun_StructuralDedupDisabledWithZeroWindow: Confirms that an explicitdate_window_days = 0disables the rule and writes duplicates as active transactions.Unit tests: Add three unit tests in
route_test.goanddedup_test.go:TestDecide_DateWindowDefault: Verifies the built-in default is applied when no configuration exists.TestDecide_DateWindowSectionZeroDisables: Confirms explicit zero at section level disables the rule.TestEquivalent_StructuralElidedSwappedNarration: Tests the dedup behavior with and without the date window.Documentation: Update doc comments and configuration documentation to clarify that the structural-dedup window defaults to 2 days and can be disabled with an explicit 0.
Design Notes
The change preserves backward compatibility for users with explicit
date_window_daysdeclarations. Users without any configuration will now have structural dedup enabled, surfacing potential duplicate entries for review rather than silently skipping them. This improves data quality by default while remaining opt-out for those who prefer the previous behavior.https://claude.ai/code/session_017oc8DY7Sgyd42zSaMZCKrU