Backfill submission geocache, fixes getodk/central#1373 #1623
Conversation
brontolosone
force-pushed
the
1373_backfill-submission-geo-cache
branch
from
8cdb999 to
d17210a
Compare
brontolosone
force-pushed
the
1373_backfill-submission-geo-cache
branch
from
d17210a to
4091c60
Compare
brontolosone
force-pushed
the
1373_backfill-submission-geo-cache
branch
from
4091c60 to
2790e4d
Compare
| const path = require('path'); | ||
|
|
||
|
|
||
| function getSqlFiles(upOrDown) { |
There was a problem hiding this comment.
what do you think of moving this function into a common file? it is duplicate of what's in 20250927-01-geoextracts.js
|
|
||
|
|
||
| const up = async (db) => { | ||
| const BATCH_SIZE = 1000; |
There was a problem hiding this comment.
Why do we need to do batching during migration/upgrade-time? Server would be down in any case so I don't see any problem populating the cache in one go.
We can keep the changes in cache_all_submission_geo to support batching that would be handy down the line when we might want to run the function at run-time
| DROP FUNCTION IF EXISTS "public"."cache_all_submission_geo"() CASCADE; | ||
|
|
||
| --- create: cache_all_submission_geo(only_default_path boolean, batchsize integer) --- | ||
| CREATE FUNCTION "public"."cache_all_submission_geo"(only_default_path boolean = false, batchsize integer = 9223372036854775807) |
There was a problem hiding this comment.
can we set the default value of batchsize to NULL? I am thinking about cases where selected rows are greater than 9223372036854775807. From the docs:
LIMIT ALL is the same as omitting the LIMIT clause, as is LIMIT with a NULL argument.
2 participants
Closes getodk/central#1373
What has been done to verify that this works as intended?
Manual testing.
Why is this the best possible solution? Were any other approaches considered?
Batching is used because it gives the user some sort of progress meter, which is nice if there's a lot to process and/or their DB is slow. They'll continue to receive visual feedback that there's nothing stuck.
How does this change affect users? Describe intentional changes to behavior and behavior that could have accidentally been affected by code changes. In other words, what are the regression risks?
N/A
Does this change require updates to the API documentation? If so, please update docs/api.yaml as part of this PR.
N/A
Before submitting this PR, please make sure you have:
make testand confirmed all checks still pass OR confirm CircleCI build passes