Skip to content

Make API importer only update new records #126

@tamcap

Description

@tamcap

I am currently working on some improvements to the current API importer.

However, as it is right now, we import the whole database from Durham County - despite their API providing us with a field Update_Date and syntax to filter it by Update_Date__gt=mm/dd/yyy that we can use to significantly limit our data / cpu use. We should take advantage of it.

Implementation: we will need to run a query on our db before import starts to establish the "latest" Update_Date value for each table (establishments, inspections, violations) and then incorporate it into the query string.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions