Skip to content

Pre-register the join tasks - #17

Merged
Kcodess2807 merged 1 commit into
mainfrom
join-tasks
Nov 19, 2026
Merged

Kcodess2807 merged 1 commit into
mainfrom
join-tasks

Conversation

@Kcodess2807

Copy link
Copy Markdown
Owner

Pre-registers the experiment two outside reviews asked for: does the graph add anything over replaying past statements?

  • 29 join tasks on the frozen pytest snapshot. Each needs a fact the agent was told joined to a fact GitHub records, and the question never names the entity in the middle. Example: "Which group does the author of PR #15246 sit in?"
  • Six categories: memory then GitHub, GitHub then memory, through an issue, through a file, a join after an update, and a join with one half missing (answer: unknown).
  • Gold answers: the GitHub half is read from the snapshot and asserted by make_join_tasks.py. The remembered half is invented.
  • The reading is fixed before any run: graph memory has to beat episodic recall on the join categories (sign test, p<0.05). A full-context arm is reported beside them as the ceiling.

No run yet. This PR only commits the tasks so they can't be tuned afterwards.

🤖 Generated with Claude Code

29 questions that need a remembered fact joined to a GitHub fact, and never
name the entity in the middle. The GitHub half of each gold answer is read
from the frozen pytest snapshot and asserted by the generator.

Committed before any run. The reading is fixed in the task file: graph memory
has to beat episodic recall on the join categories for the graph to count as
adding something over replaying statements.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@vercel

vercel Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
trace-rag Ready Ready Preview Nov 19, 2026 6:27am UTC

@Kcodess2807
Kcodess2807 merged commit 211c99f into main Nov 19, 2026
8 checks passed

This branch was successfully deployed

1 active deployment
Preview — 45fa8372 Deployed Nov 19, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant