feat: handle parser artifacts in single-ingredient checks (#8353)#13367
Open
adityav27 wants to merge 2 commits into
Open
feat: handle parser artifacts in single-ingredient checks (#8353)#13367adityav27 wants to merge 2 commits into
adityav27 wants to merge 2 commits into
Conversation
…cts#8353) - Upgraded ingredient validation to scan for expected ingredients in O(M) time. - Categorized unparsed descriptive text as a warning rather than a hard error. - Maintained zero-false-positive strictness for actual ingredient mismatches.
github-actions
Bot
added
the
🧽 Data quality
github-project-automation
Bot
moved this to To discuss and validate
in 🍊 Open Food Facts Server issues
This was referenced Mar 29, 2026
Author
|
@teolemon could you please take a look when you have time? |
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #13367 +/- ##
==========================================
+ Coverage 49.25% 54.51% +5.25%
==========================================
Files 97 98 +1
Lines 25552 25618 +66
Branches 6101 6113 +12
==========================================
+ Hits 12585 13965 +1380
+ Misses 11338 9852 -1486
- Partials 1629 1801 +172
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
This PR resolves the 1000% spike in false positives for the
ingredients-single-ingredient-from-category-does-not-match-actual-ingredientsdata quality check.Why it was made:
Previously, if the NLP parser extracted a correct ingredient but failed to parse additional descriptive marketing text (e.g., parsing both
"en:olive-oil"and"en:unknown-manually-harvested"),DataQualityFood.pmwould blindly throw a strict error simply because the array length was> 1. It failed perfectly valid products without checking if the correct ingredient was actually present.What was changed:$O(M)$ time:
I updated the validation block to scan the parsed ingredients array in
data_quality_errors_tags.> 1(due to unparsed marketing artifacts), it now pushes todata_quality_warnings_tags. This safely flags the text for human review without invalidating the product.Large Language Models usage disclosure
I used GitHub Copilot Chat to analyze the repository structure and simulate edge cases based on the existing
dataqualityfood.ttest suite.