Hey team, I have seen that the trainer does a normal cross entropy loss without accounting for class frequency. In typical real life scenarios, the 'O' (background) labels heavily outnumber the privacy or entity labels (in my dataset it is about 95% 'O'). In some of my previous projects I have used specialized loss functions like focal loss to help with this imbalance.
I would like to know how you handle such cases while training the model in this repo. I think it's not possible to have a dataset with good balance between 'O' and privacy labels, so I imagine you must be doing some specialized training, creative sampling, or have high-quality balanced data. Could you please share either some data samples, stratification strategy, or specific training procedure you recommend for handling this issue? It would be very helpful!
Hey team, I have seen that the trainer does a normal cross entropy loss without accounting for class frequency. In typical real life scenarios, the 'O' (background) labels heavily outnumber the privacy or entity labels (in my dataset it is about 95% 'O'). In some of my previous projects I have used specialized loss functions like focal loss to help with this imbalance.
I would like to know how you handle such cases while training the model in this repo. I think it's not possible to have a dataset with good balance between 'O' and privacy labels, so I imagine you must be doing some specialized training, creative sampling, or have high-quality balanced data. Could you please share either some data samples, stratification strategy, or specific training procedure you recommend for handling this issue? It would be very helpful!