Fix tests not executing #1517
Merged
Fix tests not executing #1517
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
schneems
force-pushed
the
schneems/yarn-failure
branch
2 times, most recently
from
90ed4c3 to
9b6fdc5
Compare
schneems
force-pushed
the
schneems/yarn-failure
branch
3 times, most recently
from
c1517b4 to
2c386d5
Compare
Issue #1505 spells out a problem where it appears ci-queue does not fail even when the test suite is broken. I'm unsure why this happens. To debug I'm emitting the tests that run so I can audit to make sure `rspec ./spec/helpers/yarn_installer_spec.rb:6` is executed. If it's not, then I need to diagnose why. If it is, then perhaps there some ordering bug that's affecting the outcome.
schneems
force-pushed
the
schneems/yarn-failure
branch
from
2c386d5 to
2aaad67
Compare
edmorley
approved these changes
Nov 13, 2024
Merged
schneems
added a commit
that referenced
this pull request
Apr 24, 2025
* Emit log order Issue #1505 spells out a problem where it appears ci-queue does not fail even when the test suite is broken. I'm unsure why this happens. To debug I'm emitting the tests that run so I can audit to make sure `rspec ./spec/helpers/yarn_installer_spec.rb:6` is executed. If it's not, then I need to diagnose why. If it is, then perhaps there some ordering bug that's affecting the outcome. * Update CI-queue * Update redis version * Use branch of ci-queue
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fix #1505 by bypassing redis connection SSL verification.
In #1505 debugging found that trying to connect to a Redis instance with a self-signed certificate via
ci-queuewould result in no tests being run but the output returning a zero exit code (indicating success). This behavior was reported in Shopify/ci-queue#292 where they shared that it's intentional to handle flaky test workers (shopify scale is much larger than my 16 node setup here).This PR uses a branch of the ci-queue gem that disables SSL validation by default (allowing connecting to redis servers with self-signed certificates). Differences are here Shopify/ci-queue@main...schneems:ci-queue:schneems/allow-hosted-redis.
The result is that when using these problem versions #1516 is that the suite now executes correctly and reports a failure for the problem versions:
After validating that CI behavior I rebased against main which should be green. I'll work towards upstreaming that feature, but in the short term this enables our CI to correctly run tests and report failures.