Skip to content

ansible-scylla-node: increase API port wait timeout to reduce CI flakiness#533

Draft
cursor[bot] wants to merge 3 commits into
masterfrom
cursor/ci-failure-automation-9da3
Draft

ansible-scylla-node: increase API port wait timeout to reduce CI flakiness#533
cursor[bot] wants to merge 3 commits into
masterfrom
cursor/ci-failure-automation-9da3

Conversation

@cursor

@cursor cursor Bot commented Jul 19, 2026

Copy link
Copy Markdown

Summary

  • Bump wait for the API port to come up on all nodes task timeout from 300s to 600s in ansible-scylla-node/tasks/common.yml.

Context

PR #532's build (rockylinux10) Molecule job failed with:

fatal: [node2]: FAILED! => {"changed": false, "elapsed": 300, "msg": "Timeout when waiting for 127.0.0.1:10000"}
fatal: [node3]: FAILED! => {"changed": false, "elapsed": 300, "msg": "Timeout when waiting for 127.0.0.1:10000"}

This happens during the converge scenario, before any of PR #532's changes (which only touch ansible-scylla-node/tasks/upgrade/post_upgrade.yml, not exercised by this scenario) run. The CQL port already responded (after retries) but the scylla-server API port took longer than the hardcoded 300s under loaded CI runners (multiple containers competing for resources). The same class of "Timeout when waiting for " failure has also been observed on master runs for unrelated services (e.g. scylla-manager port 5080), confirming this is pre-existing CI flakiness, not a regression introduced by PR #532.

This mirrors prior fixes for the same kind of flakiness in this file (commits 34f0e15 and 10e2925, which increased retries/timeouts for adjacent readiness checks).

Test plan

  • Molecule CI green on the affected distros
Open in Web View Automation 

ManjotS and others added 3 commits July 17, 2026 13:01
Add upgrade_sstables (default true) and invoke nodetool upgradesstables
in post_upgrade once CQL is up, so rolling upgrades rewrite SSTables
to the current format (including system keyspace drifts).

Co-authored-by: Cursor <cursoragent@cursor.com>
Wait for nodetool readiness before upgradesstables, and bound runtime
via upgrade_sstables_timeout_seconds (async/poll) like cleanup/repair.

Co-authored-by: Cursor <cursoragent@cursor.com>
…iness

The 'wait for the API port to come up on all nodes' task used a
hardcoded 300s timeout. Under loaded CI runners (multiple docker
containers competing for resources) the scylla-server API can take
longer than 300s to come up even though the CQL port is already
responding, causing intermittent molecule build failures (e.g.
build (rockylinux10)) unrelated to the code under test.

Bump the timeout to 600s, consistent with prior fixes for the same
kind of flakiness in this file (e.g. 34f0e15, 10e2925).

Co-authored-by: Manjot Singh <ManjotS@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants