Open
Description
BigQuery rate limits number of DML statements against a table: https://cloud.google.com/bigquery/docs/data-manipulation-language#update_delete_merge_dml_concurrency. This might result in data writes to copied_tables
and last_lsn
tables failing under heavy load. Consider moving resumption state to either the API db or source db. Or maybe even hide resumption state logic behind a trait like ResumptionStateStore
which allows users to pick a location of their own for the resumption state store (source db, API db, file etc.).
Metadata
Metadata
Assignees
Labels
No labels