What's the recommend train - eval - test split ratio

[ronalhung05]

Hello everyone, I'm doing the project building an AI model to predict sentiment based on aspect (ABSA), I encountered this interesting framework, but my question is when I do the train - eval - test split, i don't know how to split them probably to give a good assessment. By that i mean, I utilize the dataset of the author 'tomaarsen/setfit-absa-semeval-restaurants' on hugging face dataset, I see only training part is fully labeled but the test is none, so I will use the training part of this to split train - eval - test, but I don't know the good ratio. Please help me thanks u guys for reading

[emilioMaddalena]

Tests sets are typically also labeled, otherwise you'll never know how well your model geenralizes. So that'd require you to save some of your labeled examples to create a test set.

That said, if by "test" you mean see what comes out of it and manually assessing the results, then your approach would be okay. By judging the model's output yourself, you're indirectly labeling the example.

Finally, if you just wanna split your labeled set into train + eval, a 80-20 ratio could be adopted. Make sure you have a balanced ratio of positive, neutral and negative examples in each split! 😄