Is the "Recent Changes" page a good use case for openfoodfacts-query ?

[teolemon]

Problem

https://world.openfoodfacts.org/cgi/recent_changes.pl was listing different changes on Open Food Facts but it's mostly broken. (you can look at what it looks on staging)

We should leverage events we store in openfoodfacts-query to recreate it, better

Proposed solution

• add an API to be able to get recent changes as a paginated json result
• add an html version of it that could be used as a drop-in replacement for https://world.openfoodfacts.org/cgi/recent_changes.pl (or alernatively create a webcomponent in openfoodfacts-webcomponents that use the API to display recent changes)

As a very usefull strech goal, we could also be able to filter on different facets, to for example have recent changes on beer category in germany.

The recent changes pages should expose things like:

• code (product barcode)
• userid (only use id)
• time of change
• comment
• revision number
• changed fields

(edited by @alexgarel)

[john-gom]

Maybe, although at the moment the data will be delayed by at least a day. If / when we implement an event stream then the data will be more up to date.

[alexgarel]

Now it's the case, we have an event stream and events in a table !

[alexgarel]

see also #33

[john-gom]

Currently the Redis change events we use to store events in off-query does not include the IP address of the sender, so we would need to add this in to the events generated by Product Opener.

We also store the countries_tags of the product but I assume this is just for filtering so we could use the current countries_tags for the product rather than needing to know what the countries_tags were at the time of the change.

There are a number of processes which generate changes but don't update the recent_changes_collection, such as update_all_products. However, off-query will not log second or subsequent events for the same product revision in the product history, so this is probably OK.

[john-gom]

Product Opener PR to include IP in Redis events: openfoodfacts/openfoodfacts-server#12253

[alexgarel]

@john-gom I'm not really sure if we want to add the IP as this is a personal information.
It was the case in previous implementation but I'm not sure it's a good idea to do this.

@stephanegigandet any though on why it was there before ?

[stephanegigandet]

For updates? I'm not sure. But we need ip address for scan events so that we can unique on it, look up the countries etc.

[alexgarel]

I think country resolution must be done before storage, and we can store a hash of ip address instead of the ip address.

[john-gom]

Maybe we need to document why we need the IP address, or a hash of it, before deciding?