Completion criteria:

• There is a command or function which pulls data from the Google Sheet into the Wagtail Database
• This command/function can be run periodically
• Infra has been coordinated with as needed to ensure the commands runs regularly (hourly?)

Original issue description below.

Description

Editorial is tracking unconfirmed incidents in a google sheet (#2029), and the easiest workflow for them would be for us to just pull from data from their existing Google sheet.

Directly ingesting data from Google Sheets as a production data source has some operational concerns around data validation, schema stability, and access control that we should consider. It would be good to explore the options and tradeoffs, especially around:

• Pre-publication / review step: Not sure if this is feasible given the rapid pace of changes, but it's an option to consider (e.g. export sheet to intermediate data format and approve that?).
• Intermediate validation code: What would a robust validation layer look like? Both in terms of the data and the schema in the sheet. We can also potentially add some validation in the sheet to reduce errors, e.g. enum dropdowns in the sheet could be matched with Python validation we do in the intermediate validation step.
• Rollback and alerting: if there are errors we should log it and make sure we get alerted, and rollback to the known good state.

Urgency

Medium (don't forget about this)

Impact

Important

Timing

No response

Notifications

No response