Tests reliability improvements: Connection parameters extraction logic unification by pfcoperez · Pull Request #4192 · PeerDB-io/peerdb

pfcoperez · 2026-04-17T20:30:19Z

This PR unifies test DB connection parameters acquisition into common functions used across all test suites.

Ready when

Tested
Exhaustivity is confirmed.

What does this PR resolve?

Closes: https://linear.app/clickhouse/issue/DBI-674

codecov · 2026-04-17T20:47:02Z

❌ 2 Tests Failed:

Tests completed	Failed	Passed	Skipped
2225	2	2223	203

View the top 3 failed test(s) by shortest run time

github.com/PeerDB-io/peerdb/flow/e2e::TestApiMongo

Stack Traces | 0.01s run time

=== RUN   TestApiMongo
=== PAUSE TestApiMongo
=== CONT  TestApiMongo
--- FAIL: TestApiMongo (0.01s)

github.com/PeerDB-io/peerdb/flow/e2e::TestApiPg

Stack Traces | 0.01s run time

=== RUN   TestApiPg
=== PAUSE TestApiPg
=== CONT  TestApiPg
--- FAIL: TestApiPg (0.01s)

github.com/PeerDB-io/peerdb/flow/e2e::TestApiMongo/TestCancelAddCancel

Stack Traces | 37.1s run time

=== RUN   TestApiMongo/TestCancelAddCancel
=== PAUSE TestApiMongo/TestCancelAddCancel
=== CONT  TestApiMongo/TestCancelAddCancel
2026/04/22 13:53:42 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/22 13:53:42 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    cancel_table_addition_test.go:787: WaitFor wait for initial load to finish 2026-04-22 13:53:46.484090771 +0000 UTC m=+440.278878955
    cancel_table_addition_test.go:791: WaitFor t1 2026-04-22 13:53:46.484481166 +0000 UTC m=+440.279269350
    cancel_table_addition_test.go:127: WaitFor wait for pause for add e2e_test_api_vlxy6uui.t2 2026-04-22 13:53:46.50288155 +0000 UTC m=+440.297669733
    cancel_table_addition_test.go:825: WaitFor wait for stuck snapshot of t2 add table 2026-04-22 13:53:48.509614311 +0000 UTC m=+442.304402484
    cancel_table_addition_test.go:828: WaitFor waiting for initial load MV error messages for t2 2026-04-22 13:53:49.510822273 +0000 UTC m=+443.305610446
    cancel_table_addition_test.go:859: WaitFor wait for running after t2 cancel 2026-04-22 13:54:02.511374273 +0000 UTC m=+456.306162457
    cancel_table_addition_test.go:127: WaitFor wait for pause for add e2e_test_api_vlxy6uui.t2 2026-04-22 13:54:02.522889562 +0000 UTC m=+456.317677745
    cancel_table_addition_test.go:872: WaitFor wait for stuck snapshot of t2 add table again 2026-04-22 13:54:18.54253534 +0000 UTC m=+472.337323553
    cancel_table_addition_test.go:873: UNEXPECTED ERROR unable to establish connection with catalog: FATAL: terminating connection due to administrator command (SQLSTATE 57P01)
    api_test.go:49: begin tearing down postgres schema api_vlxy6uui
--- FAIL: TestApiMongo/TestCancelAddCancel (37.14s)

github.com/PeerDB-io/peerdb/flow/e2e::TestApiPg/TestTableAdditionWithoutInitialLoad

Stack Traces | 131s run time

=== RUN   TestApiPg/TestTableAdditionWithoutInitialLoad
=== PAUSE TestApiPg/TestTableAdditionWithoutInitialLoad
=== CONT  TestApiPg/TestTableAdditionWithoutInitialLoad
2026/04/22 13:51:13 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/04/22 13:51:13 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    api_test.go:2500: WaitFor wait for initial load to finish 2026-04-22 13:51:17.289874511 +0000 UTC m=+307.821344866
2026/04/22 13:51:17 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_7l2n88u0.\"original\" ORDER BY id"
2026/04/22 13:51:17 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_7l2n88u0.\"original\" ORDER BY id"
2026/04/22 13:51:17 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_7317182388686260784 CURSOR FOR SELECT id,val FROM e2e_test_api_7l2n88u0.\"original\" ORDER BY id" args=[]
2026/04/22 13:51:17 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_7l2n88u0.\"original\" ORDER BY id" channelLen=0
2026/04/22 13:51:17 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_7317182388686260784
2026/04/22 13:51:17 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_7317182388686260784 records=1 bytes=9 channelLen=0
2026/04/22 13:51:17 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_7l2n88u0.\"original\" ORDER BY id" rows=1 bytes=9 channelLen=0
2026/04/22 13:51:17 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_7317182388686260784
2026/04/22 13:51:17 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_7317182388686260784 records=0 bytes=0 channelLen=0
2026/04/22 13:51:17 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_7l2n88u0.\"original\" ORDER BY id" rows=0 bytes=0 channelLen=0
2026/04/22 13:51:17 INFO [pg_query_executor] committing transaction x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart
2026/04/22 13:51:17 INFO [pg_query_executor] committed transaction for query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_7l2n88u0.\"original\" ORDER BY id" rows=1 bytes=9 channelLen=0
    api_test.go:2510: WaitFor wait for pause for add table 2026-04-22 13:51:17.308170054 +0000 UTC m=+307.839640399
2026/04/22 13:51:17 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,c1,c2 FROM e2e_test_pgch_hjtimni3.\"test_nullable_sc_ch\" ORDER BY id"
2026/04/22 13:51:17 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,c1,c2 FROM e2e_test_pgch_hjtimni3.\"test_nullable_sc_ch\" ORDER BY id"
2026/04/22 13:51:17 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_10457237255841560904 CURSOR FOR SELECT id,c1,c2 FROM e2e_test_pgch_hjtimni3.\"test_nullable_sc_ch\" ORDER BY id" args=[]
2026/04/22 13:51:17 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,c1,c2 FROM e2e_test_pgch_hjtimni3.\"test_nullable_sc_ch\" ORDER BY id" channelLen=0
2026/04/22 13:51:17 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_10457237255841560904
2026/04/22 13:51:17 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_10457237255841560904 records=1 bytes=8 channelLen=0
2026/04/22 13:51:17 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,c1,c2 FROM e2e_test_pgch_hjtimni3.\"test_nullable_sc_ch\" ORDER BY id" rows=1 bytes=8 channelLen=0
2026/04/22 13:51:17 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_10457237255841560904
2026/04/22 13:51:17 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_10457237255841560904 records=0 bytes=0 channelLen=0
2026/04/22 13:51:17 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,c1,c2 FROM e2e_test_pgch_hjtimni3.\"test_nullable_sc_ch\" ORDER BY id" rows=0 bytes=0 channelLen=0
2026/04/22 13:51:17 INFO [pg_query_executor] committing transaction x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart
2026/04/22 13:51:17 INFO [pg_query_executor] committed transaction for query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,c1,c2 FROM e2e_test_pgch_hjtimni3.\"test_nullable_sc_ch\" ORDER BY id" rows=1 bytes=8 channelLen=0
    api_test.go:2531: WaitFor wait for table addition to finish 2026-04-22 13:51:19.317433144 +0000 UTC m=+309.848903489
2026/04/22 13:51:19 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,c1,c2 FROM e2e_test_pgch_hjtimni3.\"test_nullable_sc_ch\" ORDER BY id"
2026/04/22 13:51:19 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,c1,c2 FROM e2e_test_pgch_hjtimni3.\"test_nullable_sc_ch\" ORDER BY id"
2026/04/22 13:51:19 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_8540081903733456979 CURSOR FOR SELECT id,c1,c2 FROM e2e_test_pgch_hjtimni3.\"test_nullable_sc_ch\" ORDER BY id" args=[]
2026/04/22 13:51:19 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,c1,c2 FROM e2e_test_pgch_hjtimni3.\"test_nullable_sc_ch\" ORDER BY id" channelLen=0
2026/04/22 13:51:19 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_8540081903733456979
2026/04/22 13:51:19 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_8540081903733456979 records=1 bytes=8 channelLen=0
2026/04/22 13:51:19 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,c1,c2 FROM e2e_test_pgch_hjtimni3.\"test_nullable_sc_ch\" ORDER BY id" rows=1 bytes=8 channelLen=0
2026/04/22 13:51:19 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_8540081903733456979
2026/04/22 13:51:19 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_8540081903733456979 records=0 bytes=0 channelLen=0
2026/04/22 13:51:19 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,c1,c2 FROM e2e_test_pgch_hjtimni3.\"test_nullable_sc_ch\" ORDER BY id" rows=0 bytes=0 channelLen=0
2026/04/22 13:51:19 INFO [pg_query_executor] committing transaction x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart
2026/04/22 13:51:19 INFO [pg_query_executor] committed transaction for query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,c1,c2 FROM e2e_test_pgch_hjtimni3.\"test_nullable_sc_ch\" ORDER BY id" rows=1 bytes=8 channelLen=0
    api_test.go:2560: WaitFor test cdc of cdc_only table addition 2026-04-22 13:51:21.324980437 +0000 UTC m=+311.856450802
2026/04/22 13:51:21 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,c1,c2 FROM e2e_test_pgch_hjtimni3.\"test_nullable_sc_ch\" ORDER BY id"
2026/04/22 13:51:21 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,c1,c2 FROM e2e_test_pgch_hjtimni3.\"test_nullable_sc_ch\" ORDER BY id"
2026/04/22 13:51:21 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_2438548708861241747 CURSOR FOR SELECT id,c1,c2 FROM e2e_test_pgch_hjtimni3.\"test_nullable_sc_ch\" ORDER BY id" args=[]
2026/04/22 13:51:21 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,c1,c2 FROM e2e_test_pgch_hjtimni3.\"test_nullable_sc_ch\" ORDER BY id" channelLen=0
2026/04/22 13:51:21 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_2438548708861241747
2026/04/22 13:51:21 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_2438548708861241747 records=1 bytes=8 channelLen=0
2026/04/22 13:51:21 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,c1,c2 FROM e2e_test_pgch_hjtimni3.\"test_nullable_sc_ch\" ORDER BY id" rows=1 bytes=8 channelLen=0
2026/04/22 13:51:21 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_2438548708861241747
2026/04/22 13:51:21 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_2438548708861241747 records=0 bytes=0 channelLen=0
2026/04/22 13:51:21 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,c1,c2 FROM e2e_test_pgch_hjtimni3.\"test_nullable_sc_ch\" ORDER BY id" rows=0 bytes=0 channelLen=0
2026/04/22 13:51:21 INFO [pg_query_executor] committing transaction x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart
2026/04/22 13:51:21 INFO [pg_query_executor] committed transaction for query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,c1,c2 FROM e2e_test_pgch_hjtimni3.\"test_nullable_sc_ch\" ORDER BY id" rows=1 bytes=8 channelLen=0
    clickhouse.go:97: 
        	Error Trace:	.../flow/e2e/congen.go:44
        	            				.../flow/e2e/clickhouse.go:97
        	            				.../flow/e2e/clickhouse.go:79
        	            				.../flow/e2e/clickhouse.go:154
        	            				.../flow/e2e/test_utils.go:240
        	            				.../flow/e2e/test_utils.go:852
        	            				.../flow/e2e/test_utils.go:237
        	            				.../flow/e2e/api_test.go:2560
        	Error:      	Received unexpected error:
        	            	unable to establish connection with catalog: FATAL: terminating connection due to administrator command (SQLSTATE 57P01)
        	Test:       	TestApiPg/TestTableAdditionWithoutInitialLoad
    api_test.go:49: begin tearing down postgres schema api_7l2n88u0
2026/04/22 13:51:24 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_cuhfiqwy.\"qrepapi_api_cuhfiqwy\" ORDER BY id"
2026/04/22 13:51:24 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_cuhfiqwy.\"qrepapi_api_cuhfiqwy\" ORDER BY id"
2026/04/22 13:51:24 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_13546884399267033622 CURSOR FOR SELECT id,val FROM e2e_test_api_cuhfiqwy.\"qrepapi_api_cuhfiqwy\" ORDER BY id" args=[]
2026/04/22 13:51:24 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_cuhfiqwy.\"qrepapi_api_cuhfiqwy\" ORDER BY id" channelLen=0
2026/04/22 13:51:24 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_13546884399267033622
2026/04/22 13:51:24 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_13546884399267033622 records=1 bytes=9 channelLen=0
2026/04/22 13:51:24 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_cuhfiqwy.\"qrepapi_api_cuhfiqwy\" ORDER BY id" rows=1 bytes=9 channelLen=0
2026/04/22 13:51:24 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_13546884399267033622
2026/04/22 13:51:24 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_13546884399267033622 records=0 bytes=0 channelLen=0
2026/04/22 13:51:24 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_cuhfiqwy.\"qrepapi_api_cuhfiqwy\" ORDER BY id" rows=0 bytes=0 channelLen=0
2026/04/22 13:51:24 INFO [pg_query_executor] committing transaction x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart
2026/04/22 13:51:24 INFO [pg_query_executor] committed transaction for query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_cuhfiqwy.\"qrepapi_api_cuhfiqwy\" ORDER BY id" rows=1 bytes=9 channelLen=0
2026/04/22 13:51:24 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,c1,c2 FROM e2e_test_pgch_hjtimni3.\"test_nullable_sc_ch\" ORDER BY id"
2026/04/22 13:51:24 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,c1,c2 FROM e2e_test_pgch_hjtimni3.\"test_nullable_sc_ch\" ORDER BY id"
2026/04/22 13:51:24 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_17319086865846488778 CURSOR FOR SELECT id,c1,c2 FROM e2e_test_pgch_hjtimni3.\"test_nullable_sc_ch\" ORDER BY id" args=[]
2026/04/22 13:51:24 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,c1,c2 FROM e2e_test_pgch_hjtimni3.\"test_nullable_sc_ch\" ORDER BY id" channelLen=0
2026/04/22 13:51:24 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_17319086865846488778
2026/04/22 13:51:24 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_17319086865846488778 records=1 bytes=8 channelLen=0
2026/04/22 13:51:24 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,c1,c2 FROM e2e_test_pgch_hjtimni3.\"test_nullable_sc_ch\" ORDER BY id" rows=1 bytes=8 channelLen=0
2026/04/22 13:51:24 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_17319086865846488778
2026/04/22 13:51:24 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_17319086865846488778 records=0 bytes=0 channelLen=0
2026/04/22 13:51:24 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,c1,c2 FROM e2e_test_pgch_hjtimni3.\"test_nullable_sc_ch\" ORDER BY id" rows=0 bytes=0 channelLen=0
2026/04/22 13:51:24 INFO [pg_query_executor] committing transaction x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart
2026/04/22 13:51:24 INFO [pg_query_executor] committed transaction for query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,c1,c2 FROM e2e_test_pgch_hjtimni3.\"test_nullable_sc_ch\" ORDER BY id" rows=1 bytes=8 channelLen=0
    api_test.go:49: 
        	Error Trace:	.../flow/e2e/pg.go:147
        	            				.../flow/e2e/api_test.go:49
        	            				.../flow/e2eshared/e2eshared.go:38
        	            				.../hostedtoolcache/go/1.26.2.../src/testing/testing.go:1317
        	            				.../hostedtoolcache/go/1.26.2.../src/testing/testing.go:1667
        	            				.../hostedtoolcache/go/1.26.2.../src/testing/testing.go:2030
        	            				.../hostedtoolcache/go/1.26.2.../src/runtime/panic.go:694
        	            				.../hostedtoolcache/go/1.26.2.../src/testing/testing.go:1022
        	            				.../flow/e2e/congen.go:44
        	            				.../flow/e2e/clickhouse.go:97
        	            				.../flow/e2e/clickhouse.go:79
        	            				.../flow/e2e/clickhouse.go:154
        	            				.../flow/e2e/test_utils.go:240
        	            				.../flow/e2e/test_utils.go:852
        	            				.../flow/e2e/test_utils.go:237
        	            				.../flow/e2e/api_test.go:2560
        	            				.../hostedtoolcache/go/1.26.2.../src/reflect/value.go:586
        	            				.../hostedtoolcache/go/1.26.2.../src/reflect/value.go:369
        	            				.../flow/e2eshared/e2eshared.go:40
        	Error:      	failed to teardown postgres schema
        	Test:       	TestApiPg/TestTableAdditionWithoutInitialLoad
        	Messages:   	api_7l2n88u0: failed to drop replication slots: ERROR: replication slot "peerflow_slot_added_tables_no_initial_load_api_7l2n88u0" is active for PID 11363 (SQLSTATE 55006)
--- FAIL: TestApiPg/TestTableAdditionWithoutInitialLoad (131.40s)

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

github-actions · 2026-04-17T21:00:07Z

🔄 Flaky Test Detected

Analysis: Two different TestApiPg subtests failed across different CI matrix runs (mysql-gtid vs mysql-pos) with WaitFor timeout and teardown errors, indicating timing-dependent flakiness under high-parallelism test load rather than a real bug.
Confidence: 0.92

✅ Automatically retrying the workflow

View workflow run

github-actions · 2026-04-17T21:19:20Z

🔄 Flaky Test Detected

Analysis: The Test_Normalize_Metadata_With_Retry e2e test timed out after 180s waiting for a normalize error condition, a classic timing-sensitive failure likely caused by resource contention under the 32-parallel-test load.
Confidence: 0.78

✅ Automatically retrying the workflow

View workflow run

github-actions · 2026-04-21T11:07:50Z

🔄 Flaky Test Detected

Analysis: BigQuery E2E test Test_Schema_Change_Drop_Consecutive_Columns failed with repeated googleapi: Error 400: Unrecognized name: col_to_add errors (~100 retries over 216s), indicating a BigQuery schema propagation timing issue unrelated to the MySQL-focused PR changes.
Confidence: 0.82

✅ Automatically retrying the workflow

View workflow run

github-actions · 2026-04-21T11:25:34Z

❌ Test Failure

Analysis: The e2e test Test_MySQL_Column_Position_Shifting_DDL_Error timed out waiting for an expected DDL error message, likely because the recent commit (#4197) reclassified the MySQL error as transient/retryable, causing it to be retried indefinitely instead of surfaced as a failure.
Confidence: 0.8

⚠️ This appears to be a real bug - manual intervention needed

View workflow run

github-actions · 2026-04-21T11:32:24Z

🔄 Flaky Test Detected

Analysis: The e2e test suite hit the exact 900-second timeout ceiling (900.602s), indicating a flaky timeout failure rather than a deterministic assertion error.
Confidence: 0.9

✅ Automatically retrying the workflow

View workflow run

github-actions · 2026-04-21T11:48:43Z

🔄 Flaky Test Detected

Analysis: The test failed due to PostgreSQL catalog connection being terminated by an admin command (SQLSTATE 57P01), a transient infrastructure event unrelated to the JSON support logic under test.
Confidence: 0.92

✅ Automatically retrying the workflow

View workflow run

github-actions · 2026-04-21T12:15:08Z

🔄 Flaky Test Detected

Analysis: The test failed due to a PostgreSQL catalog connection being terminated by an administrator shutdown (SQLSTATE 57P01), which is a transient infrastructure error unrelated to the code under test.
Confidence: 0.95

✅ Automatically retrying the workflow

View workflow run

github-actions · 2026-04-21T12:38:34Z

🔄 Flaky Test Detected

Analysis: The test failed because the PostgreSQL catalog connection was forcibly terminated mid-test (FATAL: terminating connection due to administrator command, SQLSTATE 57P01), a transient CI infrastructure event unrelated to the code change.
Confidence: 0.95

✅ Automatically retrying the workflow

View workflow run

Copilot

Pull request overview

Unifies how test suites read DB connection parameters (host/port/version/credentials) by centralizing the logic and updating e2e/connector tests to use the shared helpers.

Changes:

Added shared test environment helpers for ClickHouse/MySQL/Mongo/RDS IAM auth.
Replaced duplicated env parsing in e2e and connector tests with calls into the shared helpers.
Simplified MySQL/ClickHouse host/port/version checks across suites.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
flow/internal/test_env.go	New centralized helpers for test DB env configuration and RDS IAM auth test inputs.
flow/e2e/switchboard_mysql_test.go	Uses shared MySQL version helper to decide skipping CTE test.
flow/e2e/mysql.go	Uses shared MySQL host/port/version helpers instead of local parsing functions.
flow/e2e/mongo.go	Uses shared Mongo credential helpers for admin/user connections.
flow/e2e/clickhouse_mysql_test.go	Uses shared MySQL version helper for enum ordinal behavior.
flow/e2e/clickhouse.go	Uses shared ClickHouse host/port helpers.
flow/connectors/postgres/rds_test.go	Uses shared RDS IAM auth env/config helpers.
flow/connectors/mysql/ssh_keepalive_test.go	Uses shared MySQL host/port/version helpers; removes duplicated parsing.
flow/connectors/mysql/rds_test.go	Uses shared RDS IAM auth env/config helpers.
flow/connectors/clickhouse/s3_iam_role_test.go	Uses shared ClickHouse host/port helpers.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

ilidemi · 2026-04-21T19:58:50Z

 )

 func TestAwsRDSIAMAuthConnectForPostgres(t *testing.T) {
 	t.Skip("flaky")


Would be good to validate that this and the other one still pass when not skipped (maybe it was, just going off of PR commits)

That's a good point. One green is enough, added an ephemeral commit to try it out: 84f4ae3

The test still occasionally pass:

The query compares it with a control PR #4214 created just for this purpose.

ilidemi · 2026-04-21T20:38:25Z

Should catalog.go GetAncillaryPostgresConfigFromEnv also be moved to test_env.go?

pfcoperez · 2026-04-21T20:50:18Z

Should catalog.go GetAncillaryPostgresConfigFromEnv also be moved to test_env.go?

I was tempted to do this in the same PR too but finally decided not to given that this is production code. This PR factors out everything that load tests env vars not production code. GetAncillaryPostgresConfigFromEnv is a method used multiple times in business logic.

ilidemi · 2026-04-21T22:29:39Z

GetAncillaryPostgresConfigFromEnv is a method used multiple times in business logic.

Production postgres would be peers created at runtime, right? I'm only seeing test usages

Keeping GetCatalogPostgresConfigFromEnv as is makes sense

…Decided to start the path for mongo even if they were used just in one SetupMongo function

…ter pattern

…` and `TestAwsRDSIAMAuthConnectForPostgresViaProxy`

…ling

…nection-parameters

github-actions · 2026-04-22T10:09:50Z

🔄 Flaky Test Detected

Analysis: All failures are network errors ("unexpected EOF") connecting to an external AWS RDS instance at 34.216.16.103:5432 in the IAM auth tests, consistent across all CI matrix variants and unrelated to the recent code changes.
Confidence: 0.92

✅ Automatically retrying the workflow

View workflow run

…method

pfcoperez · 2026-04-22T10:16:54Z

GetAncillaryPostgresConfigFromEnv

GetAncillaryPostgresConfigFromEnv is a method used multiple times in business logic.

Production postgres would be peers created at runtime, right? I'm only seeing test usages

Keeping GetCatalogPostgresConfigFromEnv as is makes sense

@ilidemi You are absolutely right! I did run the search for the method above it! That's embarrassing, moved in 6058d5c

github-actions · 2026-04-22T10:27:36Z

🔄 Flaky Test Detected

Analysis: All 3 failures are transient infrastructure/network errors: unexpected EOF connecting to AWS RDS, PAM auth failure via proxy, and Postgres catalog connection terminated by admin command — none are related to recent code changes.
Confidence: 0.93

✅ Automatically retrying the workflow

View workflow run

github-actions · 2026-04-22T10:39:01Z

🔄 Flaky Test Detected

Analysis: All primary failures are network errors ("unexpected EOF" connecting to external AWS RDS at 34.216.16.103:5432) across all matrix jobs, indicating the external RDS test infrastructure was temporarily unreachable rather than a code regression.
Confidence: 0.95

✅ Automatically retrying the workflow

View workflow run

github-actions · 2026-04-22T10:44:57Z

🔄 Flaky Test Detected

Analysis: All CI matrix jobs failed because the external AWS RDS instance at 34.216.16.103:5432 was unreachable ("unexpected EOF"), causing TestAwsRDSIAMAuthConnectForPostgres and TestAwsRDSIAMAuthConnectForPostgresViaProxy to fail with a network error unrelated to any code change.
Confidence: 0.95

✅ Automatically retrying the workflow

View workflow run

…Postgres` and `TestAwsRDSIAMAuthConnectForPostgresViaProxy`" This reverts commit feeec76.

github-actions · 2026-04-22T10:58:24Z

🔄 Flaky Test Detected

Analysis: Tests TestAwsRDSIAMAuthConnectForPostgres and TestAwsRDSIAMAuthConnectForPostgresViaProxy failed with network errors ("failed to receive message: unexpected EOF") when connecting to an external AWS RDS instance at 34.216.16.103:5432 — transient network connectivity failure unrelated to code changes.
Confidence: 0.95

✅ Automatically retrying the workflow

View workflow run

github-actions · 2026-04-22T11:03:49Z

🔄 Flaky Test Detected

Analysis: All failures are transient: RDS IAM auth tests hit unexpected EOF (network blip to external AWS endpoint), and MongoDB/e2e tests show repeated polling timeouts with record count mismatches (expected 2, got 1) and a ClickHouse "unknown table" error, none of which are related to the last commit.
Confidence: 0.92

✅ Automatically retrying the workflow

View workflow run

github-actions · 2026-04-22T11:16:15Z

🔄 Flaky Test Detected

Analysis: All matrix variants failed simultaneously at 11:02:56 with "unexpected EOF" connecting to the external RDS endpoint (34.216.16.103:5432), indicating a transient network outage against the AWS RDS test instance rather than a code bug.
Confidence: 0.95

✅ Automatically retrying the workflow

View workflow run

github-actions · 2026-04-22T11:21:32Z

🔄 Flaky Test Detected

Analysis: Two AWS RDS IAM auth tests failed due to transient network errors ("unexpected EOF") and PAM auth failure against a real external RDS host, which are classic infrastructure flakiness symptoms unrelated to code changes.
Confidence: 0.92

✅ Automatically retrying the workflow

View workflow run

github-actions · 2026-04-22T11:25:07Z

🔄 Flaky Test Detected

Analysis: The test failed with PostgreSQL SQLSTATE 57P01 (admin_shutdown) — the catalog DB connection was terminated by an external command mid-run, an infrastructure issue unrelated to code logic.
Confidence: 0.93

✅ Automatically retrying the workflow

View workflow run

github-actions · 2026-04-22T11:34:15Z

🔄 Flaky Test Detected

Analysis: TestAwsRDSIAMAuthConnectForPostgresViaProxy failed with a network-level "unexpected EOF" connecting to an external AWS RDS proxy (34.216.16.103:5432), indicating a transient connectivity issue rather than a code bug.
Confidence: 0.93

✅ Automatically retrying the workflow

View workflow run

github-actions · 2026-04-22T11:51:49Z

🔄 Flaky Test Detected

Analysis: TestAwsRDSIAMAuthConnectForPostgres and TestAwsRDSIAMAuthConnectForPostgresViaProxy failed with "PAM authentication failed" on an external RDS instance, indicating a transient AWS IAM credential or infrastructure issue unrelated to the recent code change.
Confidence: 0.9

✅ Automatically retrying the workflow

View workflow run

github-actions · 2026-04-22T14:02:34Z

🔄 Flaky Test Detected

Analysis: E2E tests timed out (900s limit hit in mysql-gtid) and had teardown failures across different matrix configurations, with different tests failing in each run — classic flaky CI behavior from resource contention, not a code regression.
Confidence: 0.9

✅ Automatically retrying the workflow

View workflow run

pfcoperez self-assigned this Apr 17, 2026

pfcoperez marked this pull request as ready for review April 21, 2026 10:51

pfcoperez requested review from ilidemi, jgao54 and masterashu April 21, 2026 10:52

pfcoperez marked this pull request as draft April 21, 2026 11:02

pfcoperez force-pushed the DBI-674/unified-connection-parameters branch from be414a3 to 640de99 Compare April 21, 2026 12:21

pfcoperez marked this pull request as ready for review April 21, 2026 12:21

pfcoperez requested a review from Copilot April 21, 2026 19:42

Copilot started reviewing on behalf of pfcoperez April 21, 2026 19:42 View session

Copilot AI reviewed Apr 21, 2026

View reviewed changes

Comment thread flow/internal/test_env.go Outdated

Comment thread flow/internal/test_env.go

Comment thread flow/connectors/mysql/rds_test.go

Comment thread flow/connectors/clickhouse/s3_iam_role_test.go

ilidemi approved these changes Apr 21, 2026

View reviewed changes

pfcoperez added 6 commits April 22, 2026 11:32

Connection parameters extraction logic unification

5a5788a

Colocate IAM helpers in the same package and file.

dd47750

Further use of common test helper library in MySQL/Maria tests

591968f

Use common test library for PSQL RDS parameters

0d2b17a

Use common test helper library to get MongoDB connection parameters. …

c496c3c

…Decided to start the path for mongo even if they were used just in one SetupMongo function

Expanded helper functions

9eccded

pfcoperez added 5 commits April 22, 2026 11:37

Refactor MYSQL/Maria parameter extraction using the default-as-parame…

5d93ddb

…ter pattern

TO-REVERT: Temporarily re-enable `TestAwsRDSIAMAuthConnectForPostgres…

feeec76

…` and `TestAwsRDSIAMAuthConnectForPostgresViaProxy`

PR Comments: Move MongoDB env vars validations to the common test too…

606029e

…ling

Add checks to RDS env var parameters

e63cd4c

Merge remote-tracking branch 'upstream/main' into DBI-674/unified-con…

7c49893

…nection-parameters

pfcoperez force-pushed the DBI-674/unified-connection-parameters branch from 84f4ae3 to 7c49893 Compare April 22, 2026 09:50

Moved GetAncillaryPostgresConfigFromEnv to test helpers module.

6058d5c

PR Comments: Consistent MySQL format in MySQLTestVersionIsMySQLPos …

552396b

…method

pfcoperez mentioned this pull request Apr 22, 2026

IGNORE - DO NOT MERGE: Renable RDS PSQL tests #4214

Closed

Revert "TO-REVERT: Temporarily re-enable `TestAwsRDSIAMAuthConnectFor…

639031e

…Postgres` and `TestAwsRDSIAMAuthConnectForPostgresViaProxy`" This reverts commit feeec76.

pfcoperez added 2 commits April 22, 2026 15:14

Factor out AWS credentials extraction

10c9fdb

Factor out TOXIPROXY_POSTGRES_HOST access.

e96f0a6

pfcoperez enabled auto-merge (squash) April 22, 2026 13:47

pfcoperez merged commit 58dc970 into main Apr 22, 2026
13 of 16 checks passed

pfcoperez deleted the DBI-674/unified-connection-parameters branch April 22, 2026 14:19

Uh oh!

Conversation

pfcoperez commented Apr 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Ready when

What does this PR resolve?

Uh oh!

codecov Bot commented Apr 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

❌ 2 Tests Failed:

Uh oh!

github-actions Bot commented Apr 17, 2026

🔄 Flaky Test Detected

Uh oh!

github-actions Bot commented Apr 17, 2026

🔄 Flaky Test Detected

Uh oh!

github-actions Bot commented Apr 21, 2026

🔄 Flaky Test Detected

Uh oh!

github-actions Bot commented Apr 21, 2026

❌ Test Failure

Uh oh!

github-actions Bot commented Apr 21, 2026

🔄 Flaky Test Detected

Uh oh!

github-actions Bot commented Apr 21, 2026

🔄 Flaky Test Detected

Uh oh!

github-actions Bot commented Apr 21, 2026

🔄 Flaky Test Detected

Uh oh!

github-actions Bot commented Apr 21, 2026

🔄 Flaky Test Detected

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ilidemi Apr 21, 2026

Choose a reason for hiding this comment

Uh oh!

pfcoperez Apr 22, 2026

Choose a reason for hiding this comment

Uh oh!

pfcoperez Apr 22, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ilidemi commented Apr 21, 2026

Uh oh!

pfcoperez commented Apr 21, 2026

Uh oh!

ilidemi commented Apr 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Apr 22, 2026

🔄 Flaky Test Detected

Uh oh!

pfcoperez commented Apr 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Apr 22, 2026

🔄 Flaky Test Detected

Uh oh!

github-actions Bot commented Apr 22, 2026

🔄 Flaky Test Detected

Uh oh!

github-actions Bot commented Apr 22, 2026

🔄 Flaky Test Detected

Uh oh!

pfcoperez commented Apr 17, 2026 •

edited

Loading

codecov Bot commented Apr 17, 2026 •

edited

Loading

ilidemi commented Apr 21, 2026 •

edited

Loading

pfcoperez commented Apr 22, 2026 •

edited

Loading