Skip to content

[TT-16950] Backport test fixes for nonroot gateway and RPC storage#8037

Merged
buger merged 2 commits intorelease-5.12from
backport-16950-5.12
Apr 15, 2026
Merged

[TT-16950] Backport test fixes for nonroot gateway and RPC storage#8037
buger merged 2 commits intorelease-5.12from
backport-16950-5.12

Conversation

@probelabs
Copy link
Copy Markdown
Contributor

@probelabs probelabs Bot commented Apr 15, 2026

Problem / Task

Backport test fixes for nonroot gateway and RPC storage to release-5.12.

Changes

  • Cherry-picked 7892fa3 (TT-16693: Update RPC storage handler tests)
  • Cherry-picked 461478b (fix: create writable directories for nonroot gateway in tests)

Testing

  • CI tests should pass

vladzabolotnyi and others added 2 commits April 15, 2026 19:21
@vladzabolotnyi @buger
TT-16693: Update RPC storage handler tests (#7828)
1931e7a 
<!-- Provide a general summary of your changes in the Title above -->

[TT-16693](https://tyktech.atlassian.net/browse/TT-16693)
[TT-16695](https://tyktech.atlassian.net/browse/TT-16695)
[TT-16696](https://tyktech.atlassian.net/browse/TT-16696)

## Description
When running tests that use the StartTest helper with
`config.SlaveOptions.UseRPC` enabled, the gateway enters an "emergency
mode" and the gateway falls back to loading API Definitions and Policies
from a Redis backup instead of the master nodes.

Because the entire test suite runs against a single shared Redis
instance, tests running in parallel would often write to the same Redis
keys. This created a race condition where a test could read data written
by another test.

The data, while often valid JSON, would have an incorrect structure for
the context in which it was being read. The unmarshaling process would
succeed but produce a slice containing nil values
([]model.MergedAPI{nil}). Subsequent code that did not anticipate these
nil values would then panic, causing the test to fail unpredictably.

Subsequently, the `prepareSpecs` function iterates over this slice and
calls methods on the APIDefinition. This leads to a nil pointer
dereference panic when it encounters one of the nil entries.

## List of change
1. Update dispatcher setup at tests to assign default func handlers and
make code cleaner
2. Update the expected response values to provide better documentation
3. Add unique tags for RPC based tests to avoid fetching invalid values
populated by other tests

## How This Has Been Tested
This change is targeting flaky tests not the production code in
particular

## Types of changes

<!-- What types of changes does your code introduce? Put an `x` in all
the boxes that apply: -->

- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Refactoring or add test (improvements in base code or adds test
coverage to functionality)

## Checklist

<!-- Go over all the following points, and put an `x` in all the boxes
that apply -->
<!-- If there are no documentation updates required, mark the item as
checked. -->
<!-- Raise up any additional concerns not covered by the checklist. -->

- [x] I ensured that the documentation is up to date
- [x] I explained why this PR updates go.mod in detail with reasoning
why it's required
- [ ] I would like a code coverage CI quality gate exception and have
explained why



























<!---TykTechnologies/jira-linter starts here-->

### Ticket Details

<details>
<summary>
<a href="https://tyktech.atlassian.net/browse/TT-16693" title="TT-16693"
target="_blank">TT-16693</a>
</summary>

|         |    |
|---------|----|
| Status  | In Code Review |
| Summary | [CI GW] Failing Unit Test:
TestProcessKeySpaceChanges_UserKeyReset |

Generated at: 2026-03-10 06:40:24

</details>

<!---TykTechnologies/jira-linter ends here-->




























[TT-16693]:
https://tyktech.atlassian.net/browse/TT-16693?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
[TT-16695]:
https://tyktech.atlassian.net/browse/TT-16695?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
[TT-16696]:
https://tyktech.atlassian.net/browse/TT-16696?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

---------

Co-authored-by: Vlad Zabolotnyi <vlad.z@tyk.io>
@buger @claude
fix: create writable directories for nonroot gateway in tests (#8022)
f962a77 
- The gateway base image now runs as `nonroot` (uid 65532) instead of
root
- Test docker-compose files mount host directories (`apps`, `policies`)
into `/opt/tyk-gateway/`, but Docker creates these as root when they
don't exist, causing permission denied errors
- Adds `mkdir -p && chmod 777` for mounted directories before `docker
compose up` in three test suites: specs, tracing, and metrics

- [ ] Verify specs contract tests pass in CI (apps/policies dirs created
writable)
- [ ] Verify tracing e2e tests pass in CI
- [ ] Verify metrics e2e tests pass in CI

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@probelabs probelabs Bot requested a review from a team as a code owner April 15, 2026 19:21
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 15, 2026

🚨 Jira Linter Failed

Commit: f962a77
Failed at: 2026-04-15 19:22:09 UTC

The Jira linter failed to validate your PR. Please check the error details below:

🔍 Click to view error details 
failed to validate branch and PR title rules: branch name 'backport-16950-5.12' must contain a valid Jira ticket ID (e.g., ABC-123)

Next Steps

  • Ensure your branch name contains a valid Jira ticket ID (e.g., ABC-123)
  • Verify your PR title matches the branch's Jira ticket ID
  • Check that the Jira ticket exists and is accessible

This comment will be automatically deleted once the linter passes.

@probelabs
Copy link
Copy Markdown
Contributor Author

probelabs Bot commented Apr 15, 2026 •
edited
Loading

This PR backports two test-related fixes to the release-5.12 branch to improve the stability and correctness of the CI pipeline.

Files Changed Analysis

  • ci/tests/specs/test.sh & ci/tests/tracing/Taskfile.yml: These files are updated to create and set world-writable permissions (777) for the apps and policies directories. This ensures that when the gateway runs as a non-root user in tests, it has the necessary permissions to write API definitions and policies, preventing test failures.
  • gateway/rpc_storage_handler_test.go: This file sees significant refactoring to improve test reliability.
    • A new newDispatcher helper function is introduced to create a minimal RPC dispatcher with default handlers, reducing boilerplate code and ensuring the gateway can start cleanly for tests.
    • A generateUniqueTestTag helper is added to create unique tags for each test, isolating them and preventing data collisions in Redis.
    • Existing tests are updated to use these new helpers, making them cleaner and more robust.

Architecture & Impact Assessment

  • What this PR accomplishes: The PR enhances the test suite's reliability by fixing file permission issues for non-root gateway containers and improving test isolation for RPC storage handler tests.

  • Key technical changes introduced:

    1. CI Test Setup: Test scripts now explicitly create and set permissions for directories used by the gateway, accommodating execution by a non-root user.
    2. Test Refactoring: RPC-related tests in gateway/rpc_storage_handler_test.go have been refactored to use helper functions for creating mock RPC dispatchers and generating unique test tags for Redis, reducing code duplication and preventing test interference.

  • Affected system components: The changes are confined to the testing environment and do not affect production code. The primary impact is on the CI pipeline's stability and the developer experience when running tests locally.

Scope Discovery & Context Expansion

  • The changes highlight a broader initiative to support running the Tyk Gateway as a non-root user for enhanced security, with the test environment being updated to reflect this practice.
  • The refactoring in rpc_storage_handler_test.go suggests an ongoing effort to improve test quality by eliminating flakiness and ensuring tests are independent and idempotent. The introduction of unique tags for Redis keys is a clear indicator of fixing state-related test interference.
  • To further understand the context, one could investigate other test setup scripts and Dockerfiles to see where else the non-root user is being enforced. Additionally, reviewing other tests that interact with shared resources like Redis could identify other candidates for similar isolation improvements.
Metadata
  • Review Effort: 2 / 5
  • Primary Label: chore

Powered by Visor from Probelabs

Last updated: 2026-04-15T19:23:34.322Z | Triggered by: pr_opened | Commit: f962a77

💡 TIP: You can chat with Visor using /visor ask <your question>

@probelabs
Copy link
Copy Markdown
Contributor Author

probelabs Bot commented Apr 15, 2026 •
edited
Loading

✅ Architecture Check Passed

No architecture issues found – changes LGTM.

✅ Performance Check Passed

No performance issues found – changes LGTM.

\n\n

✅ Architecture Check Passed

No architecture issues found – changes LGTM.

\n\n

✅ Performance Check Passed

No performance issues found – changes LGTM.

\n\n

✅ Quality Check Passed

No quality issues found – changes LGTM.

Powered by Visor from Probelabs

Last updated: 2026-04-15T19:23:27.800Z | Triggered by: pr_opened | Commit: f962a77

💡 TIP: You can chat with Visor using /visor ask <your question>

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 15, 2026

API Changes

no api changes detected

@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud Bot commented Apr 15, 2026

Quality Gate Failed Quality Gate failed

Failed conditions
1 Security Hotspot
C Security Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

@buger buger merged commit c443ca8 into release-5.12 Apr 15, 2026
49 of 54 checks passed
@buger buger deleted the backport-16950-5.12 branch April 15, 2026 19:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@buger @vladzabolotnyi