Skip to content

Support multiple positive docs for biencoder training #907

@shan-nvidia

Description

@shan-nvidia

Is your feature request related to a problem? Please describe.
In multi-hop QA, a query would have multiple positive docs. We should use all of them for biencoder training. Currently it's only using the first one in the list of positive docs.

Describe the solution you'd like
We should use all of the positive docs for a query for biencoder training. Each epoch uses a different positive document via modulo operation.

Describe alternatives you've considered
Preprocessing the source data to multiple copies with each copy having a different positive doc for a query. But this is not feasible for a good user experience.

Additional context
None.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions