Integration tests fail in multiple SCT branches

[vponomaryov]

1. branch-2025.1
   Link: https://jenkins.scylladb.com/job/sct-github-PRs-scan/job/scylla-cluster-tests/job/PR-10558/3/execution/node/123/log/

12:49:44  =========================== short test summary info ============================
12:49:44  FAILED unit_tests/test_config.py::ConfigurationTests::test_31_check_network_config_azure - ValueError: Azure Image for scylla_version='2025.1.0' not found in eastus
12:49:44  FAILED unit_tests/test_config_get_version_based_on_conf.py::test_images[aws-enterprise] - ValueError: AMIs for scylla_version='enterprise:latest' not found in eu-west-1 arch=x86_64
12:49:44  FAILED unit_tests/test_gemini_thread.py::test_gemini_thread_without_cluster - concurrent.futures._base.TimeoutError: 1 (of 1) futures unfinished
12:49:44  FAILED unit_tests/test_kafka.py::test_01_kafka_cdc_source_connector - invoke.exceptions.UnexpectedExit: Encountered a bad command exit code!
12:49:44  
12:49:44  Command: 'sudo  docker exec 86268d2a3954d14df8e2f137b379387f63b978d4a9cab1ac0dba0bf0bcd15194 /bin/sh -c \'/usr/bin/cqlsh --no-color -u cassandra -p \'"\'"\'cassandra\'"\'"\'  --request-timeout=120 --connect-timeout=60  -e "CREATE TABLE IF NOT EXISTS keyspace1.standard1 (key blob PRIMARY KEY, \\"C0\\" blob, \\"C1\\" blob, \\"C2\\" blob, \\"C3\\" blob, \\"C4\\" blob) WITH  cdc = {\'"\'"\'enabled\'"\'"\': true, \'"\'"\'preimage\'"\'"\': false, \'"\'"\'postimage\'"\'"\': true, \'"\'"\'ttl\'"\'"\': 600}" 172.18.0.2\''
12:49:44  
12:49:44  Exit code: 2
12:49:44  
12:49:44  Stdout:
12:49:44  
12:49:44  
12:49:44  
12:49:44  Stderr:
12:49:44  
12:49:44  
12:49:44  Warning: Using a password on the command line interface can be insecure.
12:49:44  Recommendation: use the credentials file to securely provide the password.
12:49:44  
12:49:44  <stdin>:1:InvalidRequest: Error from server: code=2200 [Invalid query] message="Cannot create CDC log for a table keyspace1.standard1, because keyspace uses tablets. See issue #16317."
12:49:44  ==== 4 failed, 134 passed, 3 skipped, 1018 deselected in 1893.57s (0:31:33) ====

2. branch-2024.2
   Link: https://jenkins.scylladb.com/job/sct-github-PRs-scan/job/scylla-cluster-tests/job/PR-10555/2/execution/node/117/log/

13:18:08  =========================== short test summary info ============================
13:18:08  FAILED unit_tests/test_config.py::ConfigurationTests::test_31_check_network_config_azure - ValueError: Azure Image for scylla_version='2025.1.0' not found in eastus
13:18:08  FAILED unit_tests/test_config_get_version_based_on_conf.py::test_images[aws-enterprise] - ValueError: AMIs for scylla_version='enterprise:latest' not found in eu-west-1 arch=x86_64
13:18:08  FAILED unit_tests/test_kafka.py::test_01_kafka_cdc_source_connector - invoke.exceptions.UnexpectedExit: Encountered a bad command exit code!
13:18:08  
13:18:08  Command: 'sudo  docker exec c0847d00342aaf856cfd3663354da74c085f1df5a53351e496fa9a69bf82ffff /bin/sh -c \'/usr/bin/cqlsh --no-color -u cassandra -p \'"\'"\'cassandra\'"\'"\'  --request-timeout=120 --connect-timeout=60  -e "CREATE TABLE IF NOT EXISTS keyspace1.standard1 (key blob PRIMARY KEY, \\"C0\\" blob, \\"C1\\" blob, \\"C2\\" blob, \\"C3\\" blob, \\"C4\\" blob) WITH  cdc = {\'"\'"\'enabled\'"\'"\': true, \'"\'"\'preimage\'"\'"\': false, \'"\'"\'postimage\'"\'"\': true, \'"\'"\'ttl\'"\'"\': 600}" 172.18.0.2\''
13:18:08  
13:18:08  Exit code: 2
13:18:08  
13:18:08  Stdout:
13:18:08  
13:18:08  
13:18:08  
13:18:08  Stderr:
13:18:08  
13:18:08  
13:18:08  Warning: Using a password on the command line interface can be insecure.
13:18:08  Recommendation: use the credentials file to securely provide the password.
13:18:08  
13:18:08  <stdin>:1:InvalidRequest: Error from server: code=2200 [Invalid query] message="Cannot create CDC log for a table keyspace1.standard1, because keyspace uses tablets. See issue #16317."
13:18:08  ==== 3 failed, 128 passed, 3 skipped, 1018 deselected in 1523.83s (0:25:23) ====

3. branch-6.2:
   Link: https://jenkins.scylladb.com/job/sct-github-PRs-scan/job/scylla-cluster-tests/job/PR-10556/2/execution/node/117/log/

13:20:41  =========================== short test summary info ============================
13:20:41  FAILED unit_tests/test_config.py::ConfigurationTests::test_31_check_network_config_azure - ValueError: Azure Image for scylla_version='2025.1.0' not found in eastus
13:20:41  FAILED unit_tests/test_config_get_version_based_on_conf.py::test_images[aws-enterprise] - ValueError: AMIs for scylla_version='enterprise:latest' not found in eu-west-1 arch=x86_64
13:20:41  FAILED unit_tests/test_kafka.py::test_01_kafka_cdc_source_connector - invoke.exceptions.UnexpectedExit: Encountered a bad command exit code!
13:20:41  
13:20:41  Command: 'sudo  docker exec 1fefbdc2db312928c23197890c21864c21e216746c27b2151512347cc34c8790 /bin/sh -c \'/usr/bin/cqlsh --no-color -u cassandra -p \'"\'"\'cassandra\'"\'"\'  --request-timeout=120 --connect-timeout=60  -e "CREATE TABLE IF NOT EXISTS keyspace1.standard1 (key blob PRIMARY KEY, \\"C0\\" blob, \\"C1\\" blob, \\"C2\\" blob, \\"C3\\" blob, \\"C4\\" blob) WITH  cdc = {\'"\'"\'enabled\'"\'"\': true, \'"\'"\'preimage\'"\'"\': false, \'"\'"\'postimage\'"\'"\': true, \'"\'"\'ttl\'"\'"\': 600}" 172.18.0.2\''
13:20:41  
13:20:41  Exit code: 2
13:20:41  
13:20:41  Stdout:
13:20:41  
13:20:41  
13:20:41  
13:20:41  Stderr:
13:20:41  
13:20:41  
13:20:41  Warning: Using a password on the command line interface can be insecure.
13:20:41  Recommendation: use the credentials file to securely provide the password.
13:20:41  
13:20:41  <stdin>:1:InvalidRequest: Error from server: code=2200 [Invalid query] message="Cannot create CDC log for a table keyspace1.standard1, because keyspace uses tablets. See issue #16317."
13:20:41  ==== 3 failed, 131 passed, 3 skipped, 1019 deselected in 1520.44s (0:25:20) ====

4. branch-perf-v16
   Link: https://jenkins.scylladb.com/job/sct-github-PRs-scan/job/scylla-cluster-tests/job/PR-10557/2/execution/node/112/log/?consoleFull

13:21:25  =========================== short test summary info ============================
13:21:25  FAILED unit_tests/test_config.py::ConfigurationTests::test_24_convert_ami_multi_region - ValueError: Can't convert name 'amzn2-ami-hvm-2.0.20221210.1-x86_64-gp2' to AMI_id, no image found in regions ('eu-west-1', 'eu-west-2', 'us-west-2', 'us-east-1', 'eu-north-1', 'eu-central-1')
13:21:25  FAILED unit_tests/test_config.py::ConfigurationTests::test_25_convert_ami_single_region - ValueError: Can't convert name 'amzn2-ami-hvm-2.0.20221210.1-x86_64-gp2' to AMI_id, no image found in regions ('eu-west-1',)
13:21:25  FAILED unit_tests/test_config.py::ConfigurationTests::test_31_check_network_config_azure - ValueError: Azure Image for scylla_version='2025.1.0' not found in eastus
13:21:25  FAILED unit_tests/test_config_get_version_based_on_conf.py::test_scylla_repo[master-ubuntu-xenial] - assert True == False
13:21:25  FAILED unit_tests/test_config_get_version_based_on_conf.py::test_scylla_repo[master-centos] - assert True == False
13:21:25  FAILED unit_tests/test_config_get_version_based_on_conf.py::test_scylla_repo[master-debian-jessie] - assert True == False
13:21:25  FAILED unit_tests/test_config_get_version_based_on_conf.py::test_images[aws-master] - assert True == False
13:21:25  FAILED unit_tests/test_config_get_version_based_on_conf.py::test_images[aws-enterprise] - ValueError: AMIs for scylla_version='enterprise:latest' not found in eu-west-1 arch=x86_64
13:21:25  FAILED unit_tests/test_config_get_version_based_on_conf.py::test_images[gce-master] - assert True == False
13:21:25  FAILED unit_tests/test_config_get_version_based_on_conf.py::test_images[azure-master] - assert True == False
13:21:25  FAILED unit_tests/test_kafka.py::test_01_kafka_cdc_source_connector - invoke.exceptions.UnexpectedExit: Encountered a bad command exit code!
13:21:25  
13:21:25  Command: 'sudo  docker exec f8599bac1146eac39d6c459a63d611e7d2423e1bb3e823f5e012c0264c03edce /bin/sh -c \'/usr/bin/cqlsh --no-color -u cassandra -p \'"\'"\'cassandra\'"\'"\'  --request-timeout=120 --connect-timeout=60  -e "CREATE TABLE IF NOT EXISTS keyspace1.standard1 (key blob PRIMARY KEY, \\"C0\\" blob, \\"C1\\" blob, \\"C2\\" blob, \\"C3\\" blob, \\"C4\\" blob) WITH  cdc = {\'"\'"\'enabled\'"\'"\': true, \'"\'"\'preimage\'"\'"\': false, \'"\'"\'postimage\'"\'"\': true, \'"\'"\'ttl\'"\'"\': 600}" 172.18.0.2\''
13:21:25  
13:21:25  Exit code: 2
13:21:25  
13:21:25  Stdout:
13:21:25  
13:21:25  
13:21:25  
13:21:25  Stderr:
13:21:25  
13:21:25  
13:21:25  Warning: Using a password on the command line interface can be insecure.
13:21:25  Recommendation: use the credentials file to securely provide the password.
13:21:25  
13:21:25  <stdin>:1:InvalidRequest: Error from server: code=2200 [Invalid query] message="Cannot create CDC log for a table keyspace1.standard1, because keyspace uses tablets. See issue #16317."
13:21:25  === 11 failed, 125 passed, 3 skipped, 1008 deselected in 1576.74s (0:26:16) ====

5. branch-perf-v15
   Link: https://jenkins.scylladb.com/job/sct-github-PRs-scan/job/scylla-cluster-tests/job/PR-10554/2/execution/node/112/log/?consoleFull

13:34:56  =========================== short test summary info ============================
13:34:56  FAILED unit_tests/test_config.py::ConfigurationTests::test_24_convert_ami_multi_region - ValueError: Can't convert name 'amzn2-ami-hvm-2.0.20221210.1-x86_64-gp2' to AMI_id, no image found in regions ('eu-west-1', 'eu-west-2', 'us-west-2', 'us-east-1', 'eu-north-1', 'eu-central-1')
13:34:56  FAILED unit_tests/test_config.py::ConfigurationTests::test_25_convert_ami_single_region - ValueError: Can't convert name 'amzn2-ami-hvm-2.0.20221210.1-x86_64-gp2' to AMI_id, no image found in regions ('eu-west-1',)
13:34:56  FAILED unit_tests/test_config.py::ConfigurationTests::test_31_check_network_config_azure - ValueError: Azure Image for scylla_version='2025.1.0' not found in eastus
13:34:56  FAILED unit_tests/test_config_get_version_based_on_conf.py::test_scylla_repo[master-ubuntu-xenial] - assert True == False
13:34:56  FAILED unit_tests/test_config_get_version_based_on_conf.py::test_scylla_repo[master-centos] - assert True == False
13:34:56  FAILED unit_tests/test_config_get_version_based_on_conf.py::test_scylla_repo[master-debian-jessie] - assert True == False
13:34:56  FAILED unit_tests/test_config_get_version_based_on_conf.py::test_images[aws-master] - assert True == False
13:34:56  FAILED unit_tests/test_config_get_version_based_on_conf.py::test_images[aws-enterprise] - ValueError: AMIs for scylla_version='enterprise:latest' not found in eu-west-1 arch=x86_64
13:34:56  FAILED unit_tests/test_config_get_version_based_on_conf.py::test_images[gce-master] - assert True == False
13:34:56  FAILED unit_tests/test_config_get_version_based_on_conf.py::test_images[azure-master] - ValueError: Azure Image for scylla_version='master:latest' not found in eastus
13:34:56  FAILED unit_tests/test_config_get_version_based_on_conf.py::test_images[azure-branch-6.2] - ValueError: Azure Image for scylla_version='branch-6.2:latest' not found in eastus
13:34:56  FAILED unit_tests/test_config_get_version_based_on_conf.py::test_images[azure-enterprise] - ValueError: Azure Image for scylla_version='enterprise:latest' not found in eastus
13:34:56  FAILED unit_tests/test_config_get_version_based_on_conf.py::test_images[azure-branch-2024.2] - ValueError: Azure Image for scylla_version='branch-2024.2:latest' not found in eastus
13:34:56  FAILED unit_tests/test_kafka.py::test_01_kafka_cdc_source_connector - invoke.exceptions.UnexpectedExit: Encountered a bad command exit code!
13:34:56  
13:34:56  Command: 'sudo  docker exec 672090a61745179d3aaf44a879b460fa8393baf37e04d85dd5f1d6592132fedc /bin/sh -c \'/usr/bin/cqlsh --no-color -u cassandra -p \'"\'"\'cassandra\'"\'"\'  --request-timeout=120 --connect-timeout=60  -e "CREATE TABLE IF NOT EXISTS keyspace1.standard1 (key blob PRIMARY KEY, \\"C0\\" blob, \\"C1\\" blob, \\"C2\\" blob, \\"C3\\" blob, \\"C4\\" blob) WITH  cdc = {\'"\'"\'enabled\'"\'"\': true, \'"\'"\'preimage\'"\'"\': false, \'"\'"\'postimage\'"\'"\': true, \'"\'"\'ttl\'"\'"\': 600}" 172.18.0.2\''
13:34:56  
13:34:56  Exit code: 2
13:34:56  
13:34:56  Stdout:
13:34:56  
13:34:56  
13:34:56  
13:34:56  Stderr:
13:34:56  
13:34:56  
13:34:56  Warning: Using a password on the command line interface can be insecure.
13:34:56  Recommendation: use the credentials file to securely provide the password.
13:34:56  
13:34:56  <stdin>:1:InvalidRequest: Error from server: code=2200 [Invalid query] message="Cannot create CDC log for a table keyspace1.standard1, because keyspace uses tablets. See issue #16317."
13:34:56  === 14 failed, 114 passed, 3 skipped, 1005 deselected in 1491.67s (0:24:51) ====

[pehala]

Seems like #10389 needs to be backported there

[fruch]

It's one of the issues

We also have issue with some tests looking for enterprise branch image that doesn't exist anymore

And 2025.1.0 isn't found on Azure... @yaronkaikov are you familiar with that ?

Or maybe we are looking in the wrong place cause of something related to source available ?

[yaronkaikov]

It's one of the issues

We also have issue with some tests looking for enterprise branch image that doesn't exist anymore

And 2025.1.0 isn't found on Azure... @yaronkaikov are you familiar with that ?

Or maybe we are looking in the wrong place cause of something related to source available ?

The gallery in Azure is now scylla-2025.1 instead of scylla-enterprise-2025.1 so maybe this is the reason ?