Delve into an end-to-end Machine Learning project to improve the quality of the Open Food Facts database
Open Food Facts’ purpose is to create the largest open-source food database in the world. To this day, it has collected over 3 millions products and their information thanks to its contributors.
Nutritional value, eco-score, product origins,… Various data that define each product and give consumers and researchers insights about what they put in their plates.
This information is provided by the community of users and contributors, who actively add products data, take pictures, and fill any missing data into the database through the , which processes each word by applying a series of character deletions, additions, and replacements to identify potential corrections.
However, this method proved to be insufficient for our use case, for several reasons:
- Special Characters and Formatting: Elements like commas, brackets, and percentage signs hold critical importance in ingredient lists, influencing product composition and allergen labeling (e.g., “salt (1.2%)”).
- Multilingual Challenges: the database contains products from all over the word with a wide variety of languages. This further complicates a basic character-based approach like Norvig’s, which is language-agnostic.
Instead, we turned to the latest advancements in Machine Learning, particularly Large Language Models (LLMs), which excel in a wide variety of Natural Language Processing (NLP) tasks, including spelling correction.
This is the path we decided to take.
Evaluate
You can’t improve what you don’t measure.
What is a good correction? And how to measure the performance of the corrector, LLM or non-LLM?
Our first step is to understand and catalog the diversity of errors the Ingredient Parser encounters.
Additionally, it’s essential to assess whether an error should even be corrected in the first place. Sometimes, trying to correct mistakes could do more harm than good:
flour, salt (1!2%)
# Is it 1.2% or 12%?...
For these reasons, we created the Spellcheck Guidelines, a set of rules that limits the corrections. These guidelines will serve us in many ways throughout the project, from the dataset generation to the model evaluation.
The guidelines was notably used to create the or , whose purpose is to identify regions of similarity.
This is a perfect analogy for our spellcheck evaluation task.
Original: "Th - cat si on the fride,"
Reference: "The - cat is on the fridge."
1 0 0 1 0 0 1
Original: "Th - cat si on the fride,"
Prediction: "Th big cat is in the fridge."
0 1 0 1 1 0 1
FN FP TP FP TP
By labeling each pair with a 0 or 1 whether the word changed or not, we can calculate how often the model correctly fixes mistakes (True Positives — TP), incorrectly changes correct words (False Positives — FP), and misses errors that should have been corrected (False Negatives — FN).
In other words, we can calculate the Precision and Recall of the Spellcheck!
We now have a robust algorithm that is capable of evaluating any Spellcheck solution!
You can find the algorithm in the , a fast in-process SQL tool capable of processing millions of rows under the second.
However, those extracted lists are not corrected yet, and manually annotating them would take too much time and resources…
However, we have access to LLMs we already evaluated on the exact task. Therefore, we prompted GPT-3.5-Turbo, the best model on our benchmark, to correct every list in respect of our guidelines.
The process took less than an hour and cost nearly 2$.
We then manually reviewed the dataset using and an , a pipeline orchestrator designed for Data science and Machine Learning projects.
The training pipeline is composed as follow:
- Configurations and hyperparameters are imported to the pipeline from config yaml files;
- The training job is launched in the cloud using ;
- The fine-tuned model is then evaluated on the benchmark using the evaluation algorithm. Depending on the model sizem this process can be extremely long. Therefore, we used for human-evaluation.
After iterating over and over between refining the data and the model training, we achieved performance comparable to proprietary LLMs on the Spellcheck task, scoring an F1-Score of 0.65.
The model, a fine-tuned and evaluation
Furthermore, we estimated the Spellcheck reduced the number of unrecognized ingredients by 11%, which is promising!
Now comes the final phase of the project: integrating the model into Open Food Facts.
Deployment & Integration
Our model is big!
7 billions parameters, which means 14 GB of memory required to run it in float16, without considering the 20% overhead factor.
Additionally, large models often mean low throughput during inference, which can make them inappropriate for real-time serving. We need GPUs with large memory to run this model in production, such as the .
The process begins by extracting data from the Open Food Facts database using DuckDB, which processes 43 GB of data in under 2 minutes!
The extracted data is then sent to a Google Bucket, triggering a Google Batch Job.
This job uses a pre-prepared Docker image containing all necessary dependencies and algorithms. To optimize the resource-intensive LLM processing, we reuse vLLM achieving impressive performances, correcting 10,000 lists of ingredients in 20 minutes only with a GPU L4!
After successful processing, the corrected data is saved in a intermediate database containing the predictions of all models in Open Food Facts, served by
Additionally, we plan to continue refining the model through iterative improvements. Its performance can be significantly enhanced by improving the quality of the training data and incorporating user feedback. This approach will allow us to fine-tune the model continuously, ensuring it remains highly effective and aligned with real-world use cases.
The model, along its datasets, can be find in the official Github repository.
Thank you for reading that far! We hope you enjoyed the reading.
If you too, you want to contribute to Open Food Facts, you can:
- Contribute to the Open Food Facts : add new products to the database or improve existing ones by simply scanning their barcodes,
- Join the Open Food Facts
on Medium, where people are continuing the conversation by highlighting and responding to this story.
↗ Original-Artikel auf towardsdatascience.com lesenVollständiger Original-BerichtAusführliche Details, Code-Beispiele & Hersteller-Stellungnahme auf towardsdatascience.com.
SOCIAL SHARE CARD GENERATOR