Skip to content

Classification results for GTR-base #123

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Classification results for GTR-base #123

wants to merge 1 commit into from

Conversation

alisartazkhan
Copy link

@alisartazkhan alisartazkhan commented Feb 17, 2025

Checklist

  • Run tests locally to make sure nothing is broken using make test.
  • Run the results files checker make pre-push.

Adding a model checklist

  • I have added model implementation to mteb/models/ directory. Instruction to add a model can be found here in the following PR ____

@alisartazkhan
Copy link
Author

Classification the following tests cause issues: filipino hate speech, Korfin, AfriSentiLangClassification

@isaac-chung
Copy link
Contributor

Hi @alisartazkhan and team, thanks for starting this effort.

Could you complete the "Adding a model checklist" part from the checklist please?

  • We'd expect the sentence transformer (ST) model to have a revision, e.g. this one. So the folder structure should not have "no revision available"
  • Since it's an ST model, please add a ModelMeta entry in this file in the mteb repo and fill out the fields.

@Samoed
Copy link
Member

Samoed commented Feb 18, 2025

Also can you add back deleted results?

@mnasser3 mnasser3 self-assigned this Feb 18, 2025
@mnasser3
Copy link

Also can you add back deleted results?

if you are talking about the three tests involving filipino hate speech, Korfin, AfriSentiLangClassification, we were not able to get .jsons on these because the datasets have issues on HF, I am not wrong. So we omitted them

@Samoed
Copy link
Member

Samoed commented Feb 18, 2025

What issues did you have with these tasks? ArguAna and CQUADDupStack, their results were previously presented in the repository, but it seems that you have deleted them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants