Skip to content

[Need support]: Document the 2-hop sample data and eval harness #44

Description

@TKaltofen

What and why

demo/data/README.md documents the sample data files, but the 2-hop assets are missing: sample_qa_2hop.txt exists in the directory and demo/eval_qa_accuracy_2hop.py consumes it, yet neither is mentioned anywhere in the markdown docs. A reader browsing the data folder cannot tell what the file is for or how it differs from sample_qa.txt.

Pointers

  • demo/data/README.md ("Files" section)
  • demo/data/sample_qa_2hop.txt
  • demo/eval_qa_accuracy_2hop.py

Steps

  1. Add a bullet for sample_qa_2hop.txt to the "Files" section, describing its format and that it holds 2-hop question/answer pairs.
  2. Mention eval_qa_accuracy_2hop.py where the README discusses running the QA accuracy demo (including the full-MetaQA instructions if the 2-hop split applies there too).
  3. Run tox.

Acceptance criteria

  • The README documents every committed file in demo/data/.
  • The 2-hop eval harness is discoverable from the data README.

Notes

This project uses semantic versioning driven by Conventional Commits, so the PR title matters: use docs: document 2-hop sample data and eval harness.

New to the project? This issue is a good entry point, we are happy to help in the comments.

If you like the project, a star on the repo helps others find it. ⭐

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentationgood first issueGood for newcomershelp wantedExtra attention is needed

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions