Skip to content

Commit 97e5723

Browse files
committed
cloud: Load submissions into PostgreSQL only, faster
Remove loading submissions into BigQuery until we come up with a way to increase throughput (likely pulling chunks from PostgreSQL). This should help us deal with the backlog in production submission queue. We might also need to either switch to direct triggering of Cloud Functions by messages from the queue, to reduce latency, or simply switch to a persistent Cloud Run service. Concerns: #541
1 parent a37bff6 commit 97e5723

File tree

2 files changed

+2
-9
lines changed

2 files changed

+2
-9
lines changed

cloud

Lines changed: 1 addition & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -217,14 +217,7 @@ function execute_command() {
217217
fi
218218
declare -a -r bigquery_args
219219

220-
declare -r database=$(
221-
echo -n "mux: "
222-
echo -n "$psql_kcidb_db" | escape_whitespace
223-
echo -n " "
224-
echo -n "$bigquery_kcidb_db" | escape_whitespace
225-
echo -n " "
226-
echo -n "$bigquery_sample_kcidb_db" | escape_whitespace
227-
)
220+
declare -r database="$psql_kcidb_db"
228221

229222
if "$test"; then
230223
declare -r sqlite_clean_test_file="$TMPDIR/clean.sqlite3"

kcidb/cloud/functions.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -228,7 +228,7 @@ function functions_deploy() {
228228
--env-vars-file "$env_yaml_file" \
229229
--trigger-topic "${load_queue_trigger_topic}" \
230230
--memory 1024MB \
231-
--max-instances=1 \
231+
--max-instances=4 \
232232
--timeout 540
233233
# Remove the environment YAML file
234234
rm "$env_yaml_file"

0 commit comments

Comments
 (0)