Skip to content

Retain upstream evidence for incremental reconciliation - #236

Merged
Nitjsefnie merged 4 commits into
mainfrom
issue-196-incremental-reconciliation
Sep 8, 2026
Merged

Retain upstream evidence for incremental reconciliation#236
Nitjsefnie merged 4 commits into
mainfrom
issue-196-incremental-reconciliation

Conversation

@Nitjsefnie

@Nitjsefnie Nitjsefnie commented Sep 8, 2026

Copy link
Copy Markdown
Owner

Summary

Reconciliation now reuses durable upstream evidence and asks for changes since the last completed scan, so recent restarts and quiet passes avoid rereading every issue and pull request. Full repair and re-derivation still rebuild authoritative evidence and fold the complete repository snapshot.

Related Issues and Pull Requests

Fixes #196

Changes

  • Add inclusive since filtering, descending update ordering and stable-ID deduplication before issue hydration; since-only scans retain complete timelines.
  • Add authoritative issue lookups and paginated pull-request closing-issue references with stable subject identity checks.
  • Add migration 026 for persistent raw issue/PR evidence and generation-fenced dirty subjects.
  • Preserve validated issue/PR subject identities through webhook processing, including review dismissals, and commit invalidation with enqueue in one transaction.
  • Extend materialization with an optional synchronization payload that atomically commits evidence, checkpoint, full-pass age, exact-generation acknowledgements, cooldown clearing and run completion, rejecting stale cache versions.
  • Merge refreshed raw issues into retained evidence, including unpriced issues, and reload local users and repository configuration for each fold.
  • Refresh dirty PRs independently of issue update time, union old/current referenced issues, and reuse or replace shared review/diff evidence by stable ID.
  • Use a one-minute inclusive overlap from completed scan start; force full reads for a cold/incompatible cache, re-derivation, or six-hour full-repair age.

Testing

  • Test-first RED/GREEN coverage for GraphQL filtering/hydration, durable evidence, atomic invalidation enqueue, complete incremental folds, restart reuse, PR reference changes, local account/configuration changes, full repair and failure rollback.
  • Nine production mutation checks failed as expected and passed after restoring the implementation: missing filter, delta-only fold, erased dirty generations, ignored PR invalidation/re-derivation, cache-read bypass, unconditional full reads, postcommit cooldown clearing and checkpoint publication outside materialization.
  • On final rebased commit 2933004ee495802cebfa819369795a901fd738e6, an independent final reviewer ran pnpm db:migrate && pnpm test --run && pnpm lint && pnpm typecheck && pnpm build against local PostgreSQL 17; all 2,768 tests in 120 files passed, including the container-backed suites.
  • Independent task review approved the full implementation and rebase delta with no Critical or Important findings; scoped re-review confirmed the final shared-evidence regression assertion.
  • The build's three existing issue-88 Edge Runtime warnings remain.

Follow-ups / Known Limitations

The existing issue-comment subscription gap remains tracked by #231 and is outside this branch; periodic full repair remains necessary. The overlap does not establish an upstream snapshot or maximum-lag guarantee, and deleted upstream history cannot be recovered. Local folding/materialization still scales with the retained repository snapshot. No end-to-end GitHub point savings are claimed from request-count changes.

Footer

Generated by GPT-5.6 Sol (brief, orchestration), GPT-6 Astra (research, implementation, testing, review, verification)

Nitjsefnie and others added 4 commits September 8, 2026 10:27
Add inclusive incremental GitHub issue reads and authoritative dirty-subject lookup, plus a durable cache fenced by version and exact invalidation generations. Synchronization shares the materialization transaction and cooldown/run completion boundary.

Co-Authored-By: GPT-6 Astra <noreply@openai.com>
Validate issue and pull-request identities through the route and processor, including review dismissals. Persist repository-scoped dirty generations and queue updates in one transaction so failed writes cannot lose invalidations.

Co-Authored-By: GPT-6 Astra <noreply@openai.com>
…idence

Reuse completed scan-start checkpoints with a one-minute overlap, independently refresh dirty issue and pull-request dependencies, and repair fully on re-derivation or after six hours. Fold retained raw evidence with fresh local state and publish synchronization atomically.

Co-Authored-By: GPT-6 Astra <noreply@openai.com>
Observe the real complete fold input for both references and assert the persisted replacement-diff proof hash. The old regression survived omission of evidence from the second issue; the strengthened assertion catches that mutant.

Co-Authored-By: GPT-6 Astra <noreply@openai.com>
@Nitjsefnie
Nitjsefnie force-pushed the issue-196-incremental-reconciliation branch from 8643922 to 2933004 Compare September 8, 2026 09:02
@Nitjsefnie
Nitjsefnie marked this pull request as ready for review September 8, 2026 09:07
@Nitjsefnie
Nitjsefnie merged commit cfde86e into main Sep 8, 2026
2 checks passed
@Nitjsefnie
Nitjsefnie deleted the issue-196-incremental-reconciliation branch September 8, 2026 09:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Reconciliation refetches every issue every pass instead of asking what changed since the last one

1 participant