Skip to content

Conversation

@jacquesfize
Copy link
Contributor

@jacquesfize jacquesfize commented Oct 28, 2025

This PR propose to implement an observer matching process.

Capture d’écran du 2025-10-30 10-20-53

Basically 🔎

  • add a new step in the import process. This present every name found mapped columns with the observer" type (bib_fields.type_field) with the most similar user found in the GeoNature database. The user must verify and complete the mapping.
  • Similarity between name in the source file and the user (nom_complet property is used) in the database is based on the similarity postgresql function. A similarity threshold is set to 0.5, i.e. if a match with a similarity value < 0.5 then it is ignored.

New import workflow

This diagram shows how the user mapping works as of today developments

Diagramme sans nom

PS. There is a typo.. the t_imports field to store the observers mapping is called observermapping and not obsermapping

Things to do 🎯

  • Add matching process to the Synthese
    • change the observer field from textarea to observer
  • Add matching process to the Occhab
    • change the observer field from textarea to observer
    • Add mapping process in import_data_to_destination
  • Add matching process to Monitoring
    • change import_data_to_destination
    • change id_inventor type_widget
  • make sure observer name from different field are merged
  • Add backend test
  • Complete frontend test
  • Add the possibility to save mappings
  • Add option to disable user matching
  • Use table for the interface of mapping ?

@codecov
Copy link

codecov bot commented Oct 28, 2025

Codecov Report

❌ Patch coverage is 82.05128% with 21 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.43%. Comparing base (29d9aa6) to head (90f6868).

Files with missing lines Patch % Lines
backend/geonature/core/imports/routes/imports.py 55.17% 13 Missing ⚠️
backend/geonature/core/imports/actions.py 73.91% 6 Missing ⚠️
backend/geonature/core/imports/models.py 90.90% 1 Missing ⚠️
backend/geonature/core/imports/routes/mappings.py 85.71% 1 Missing ⚠️
Additional details and impacted files
@@                     Coverage Diff                     @@
##           feat/import-monitorings    #3775      +/-   ##
===========================================================
- Coverage                    84.51%   84.43%   -0.09%     
===========================================================
  Files                          133      134       +1     
  Lines                        10450    10553     +103     
===========================================================
+ Hits                          8832     8910      +78     
- Misses                        1618     1643      +25     
Flag Coverage Δ
pytest 84.43% <82.05%> (-0.09%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@camillemonchicourt
Copy link
Member

Add a mention to "Import" in PR and commits titles ?

@jacquesfize jacquesfize changed the title Feat/add observer control [IMPORT] add user matching step Oct 28, 2025
@camillemonchicourt
Copy link
Member

OK merci.
A voir comment laisser la possibilité de ne pas chercher à matcher les utilisateurs et de juste les importer dans le champs texte, du moins pour la synthèse.
A voir aussi si on mappe des utilisateurs, comment remplir en plus ou en complément (par exemple quand certains ne matchent pas où n'existent pas) pour quand même bien remplir le champs texte des observateurs de la synthèse...

@jacquesfize
Copy link
Contributor Author

A voir comment laisser la possibilité de ne pas chercher à matcher les utilisateurs et de juste les importer dans le champs texte, du moins pour la synthèse.

Yep je vais ajouter un paramètre donnant la possibilité de désactiver le mapping.

A voir aussi si on mappe des utilisateurs, comment remplir en plus ou en complément (par exemple quand certains ne matchent pas où n'existent pas) pour quand même bien remplir le champs texte des observateurs de la synthèse...

Pour l'instant, le champ observers_txt est rempli en plus de l'ajout des correspondances dans cor_observer_synthese

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants