Open
Description
In the jdbc/eval/retrieval-aggregator-consumer.yaml, it refers to a class edu.cmu.lti.oaqa.framework.eval.retrieval.PassageRetrievalEvalConsumer, which doesn't exist in the package nor any inherited packages. In fact, the class exists in the baseqa, but jdbc-provider doesn't depend on baseqa.
Suggestions:
- As there is no common way to load gold-standard yet, we should move the task-specific descriptor to baseqa project. Once we have a standard procedure to deal with GS, then we can put a descriptor in jdbc-provider.
- Rename all the classes related to general evaluation metrics (P/R/F/etc.) to something like GeneralEval or BasicEval, instead of PassageEval as it is actually task independent.
3, The name of the corresponding database schema might need to be changed.
Metadata
Metadata
Assignees
Labels
No labels