What and why
demo/data/README.md documents the sample data files, but the 2-hop assets are missing: sample_qa_2hop.txt exists in the directory and demo/eval_qa_accuracy_2hop.py consumes it, yet neither is mentioned anywhere in the markdown docs. A reader browsing the data folder cannot tell what the file is for or how it differs from sample_qa.txt.
Pointers
demo/data/README.md ("Files" section)
demo/data/sample_qa_2hop.txt
demo/eval_qa_accuracy_2hop.py
Steps
- Add a bullet for
sample_qa_2hop.txt to the "Files" section, describing its format and that it holds 2-hop question/answer pairs.
- Mention
eval_qa_accuracy_2hop.py where the README discusses running the QA accuracy demo (including the full-MetaQA instructions if the 2-hop split applies there too).
- Run
tox.
Acceptance criteria
- The README documents every committed file in
demo/data/.
- The 2-hop eval harness is discoverable from the data README.
Notes
This project uses semantic versioning driven by Conventional Commits, so the PR title matters: use docs: document 2-hop sample data and eval harness.
New to the project? This issue is a good entry point, we are happy to help in the comments.
If you like the project, a star on the repo helps others find it. ⭐
What and why
demo/data/README.mddocuments the sample data files, but the 2-hop assets are missing:sample_qa_2hop.txtexists in the directory anddemo/eval_qa_accuracy_2hop.pyconsumes it, yet neither is mentioned anywhere in the markdown docs. A reader browsing the data folder cannot tell what the file is for or how it differs fromsample_qa.txt.Pointers
demo/data/README.md("Files" section)demo/data/sample_qa_2hop.txtdemo/eval_qa_accuracy_2hop.pySteps
sample_qa_2hop.txtto the "Files" section, describing its format and that it holds 2-hop question/answer pairs.eval_qa_accuracy_2hop.pywhere the README discusses running the QA accuracy demo (including the full-MetaQA instructions if the 2-hop split applies there too).tox.Acceptance criteria
demo/data/.Notes
This project uses semantic versioning driven by Conventional Commits, so the PR title matters: use
docs: document 2-hop sample data and eval harness.New to the project? This issue is a good entry point, we are happy to help in the comments.
If you like the project, a star on the repo helps others find it. ⭐