Account for max_length for nutrient_extraction model

[raphael0202]

Currently, we're only using the first 512 tokens when performing nutrient extraction, due to the model context size limits. Consequently, we sometimes miss the nutrition information, such as here: https://robotoff.openfoodfacts.org/api/v1/image_predictions?model_name=nutrition_extractor&barcode=4972443146118&image_id=2.

To alleviate this issue, we can possibly:

• send consecutive chunks of 512 tokens at a time, and concatenate the results. However, this may result in incorrect prediction, especially if the end of the chunk is inside an entity.
• use heuristics to detect the beginning of the nutrition table, and send an additional prediction request with the tokens starting at the detected beginning.

[areebahmeddd]

some q's:

->can we prioritize information like looking for the nutrition table first processing them and then focus on other parts like ingredients, but this doesn't seems like a full solution?

->maybe a classic chucking could be another solution?

[raphael0202]

->can we prioritize information like looking for the nutrition table first processing them and then focus on other parts like ingredients, but this doesn't seems like a full solution?

it's tricky to know where the nutrition information is located. We have an object detection model that detects nutrition information based solely on the image, which could help us select only the tokens within the predicted bounding box. But this model is not 100% accurate.

maybe a classic chucking could be another solution?

The issue with chunking is that if the nutrition information is between two chunks, we will likely get incorrect predictions.