Skip to content

Fix release pipelines to pick up latest Go patch TT-16060#7512

Closed
Razeen-Abdal-Rahman wants to merge 23 commits intomasterfrom
releng/master-TT-16060
Closed

Fix release pipelines to pick up latest Go patch TT-16060#7512
Razeen-Abdal-Rahman wants to merge 23 commits intomasterfrom
releng/master-TT-16060

Conversation

@Razeen-Abdal-Rahman
Copy link
Copy Markdown
Contributor

@Razeen-Abdal-Rahman Razeen-Abdal-Rahman commented Nov 5, 2025

User description

Go version for release pipelines updated to 1.24-bookworm.
Other changes from gromit carried over:

  • .github/workflows/release.yml upgrade tests (logic added for handling scenarios where no previous version is available, used for new product launches) and steps to push docker images for fips
  • ci/Dockerfile updated to fix bugs when building locally
  • ci/goreleaser/goreleaser.yml properly imports env value for fips and adds docker builds to goreleaser for local testing, these are skipped in pipeline runs
  • ci/install/post_install.sh fix reversed logic and add handling if config file doesn't exist

Description

All versions of golang in the relese.yml file have been updated from 1.24-bullseye to 1.24-bookworm

Related Issue

Jira ticket: TT-16060

Motivation and Context

This update is required so the CI pipeline can pick up the latest Go patch, which addresses newly reported CVEs in stdlib

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Refactoring or add test (improvements in base code or adds test coverage to functionality)

Checklist

  • I ensured that the documentation is up to date
  • I explained why this PR updates go.mod in detail with reasoning why it's required
  • I would like a code coverage CI quality gate exception and have explained why

Ticket Details

TT-16060
Status In Dev
Summary Fix release pipelines to pick up latest Go patch

Generated at: 2025-11-10 15:33:45

@Razeen-Abdal-Rahman Razeen-Abdal-Rahman requested a review from a team as a code owner November 5, 2025 14:46
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Nov 5, 2025

API Changes

--- prev.txt	2025-11-10 15:34:03.828324448 +0000
+++ current.txt	2025-11-10 15:33:54.183317709 +0000
@@ -128,8 +128,6 @@
 	ErrUpstreamOAuthAuthorizationTypeRequired = errors.New("upstream OAuth authorization type is required")
 	// ErrInvalidUpstreamOAuthAuthorizationType is the error to return when configured OAuth authorization type is invalid.
 	ErrInvalidUpstreamOAuthAuthorizationType = errors.New("invalid OAuth authorization type")
-	// ErrAllLoadBalancingTargetsZeroWeight is the error to return when all load balancing targets have weight 0.
-	ErrAllLoadBalancingTargetsZeroWeight = errors.New("all load balancing targets have weight 0, at least one target must have weight > 0")
 )
 var DefaultValidationRuleSet = ValidationRuleSet{
 	&RuleUniqueDataSourceNames{},
@@ -137,7 +135,6 @@
 	&RuleValidateIPList{},
 	&RuleValidateEnforceTimeout{},
 	&RuleUpstreamAuth{},
-	&RuleLoadBalancingTargets{},
 }
 var ErrAllAuthSourcesDisabled = "all auth sources are disabled for %s, at least one of header/cookie/query must be enabled"
 var ErrDuplicateDataSourceName = errors.New("duplicate data source names are not allowed")
@@ -323,21 +320,6 @@
 
 func (a *APIDefinition) GetScopeToPolicyMapping() map[string]string
 
-func (a *APIDefinition) IsBaseAPI() bool
-    IsBaseAPI returns true if this API is a base API with child versions.
-    A base API is identified by having versions defined and either no BaseID or
-    BaseID equal to its own APIID.
-
-func (a *APIDefinition) IsBaseAPIWithVersioning() bool
-    IsBaseAPIWithVersioning returns true if this API is a base API with
-    versioning explicitly enabled. This is similar to IsBaseAPI but additionally
-    requires versioning to be enabled and have a version name.
-
-func (a *APIDefinition) IsChildAPI() bool
-    IsChildAPI returns true if this API is a child API in a versioning
-    hierarchy. A child API is identified by having a BaseID that differs from
-    its own APIID.
-
 func (a *APIDefinition) Migrate() (versions []APIDefinition, err error)
 
 func (a *APIDefinition) MigrateAuthentication()
@@ -1245,14 +1227,6 @@
 
 func (r *RuleAtLeastEnableOneAuthSource) Validate(apiDef *APIDefinition, validationResult *ValidationResult)
 
-type RuleLoadBalancingTargets struct{}
-    RuleLoadBalancingTargets implements validations for load balancing target
-    configurations.
-
-func (r *RuleLoadBalancingTargets) Validate(apiDef *APIDefinition, validationResult *ValidationResult)
-    Validate validates that when load balancing is enabled, at least one target
-    has weight > 0.
-
 type RuleUniqueDataSourceNames struct{}
 
 func (r *RuleUniqueDataSourceNames) Validate(apiDef *APIDefinition, validationResult *ValidationResult)
@@ -2097,16 +2071,6 @@
     ExampleExtractor returns an example payload according to the
     openapi3.SchemaRef object.
 
-func ExtractUserServers(
-	existingServers openapi3.Servers,
-	apiDef *apidef.APIDefinition,
-	baseAPI *apidef.APIDefinition,
-	config ServerRegenerationConfig,
-	versionName string,
-) (openapi3.Servers, error)
-    ExtractUserServers extracts user provided servers from an existing OAS API
-    by regenerating what the Tyk servers should be and filtering them out.
-
 func GetDefaultSecurityProcessingMode() string
     GetDefaultSecurityProcessingMode returns the default security processing
     mode.
@@ -2123,29 +2087,10 @@
     it fills an OAS with it. To be able to make it a valid OAS, it adds some
     required fields. It returns base API and its versions if any.
 
-func ShouldUpdateChildAPIs(newAPI, oldAPI *apidef.APIDefinition) bool
-    ShouldUpdateChildAPIs checks if child APIs need server updates after a base
-    API change.
-
-    Configuration changes that trigger child API updates: - Versioning method
-    changed (url/url-param/header) - Versioning key changed (parameter/header
-    name) - Base API's listen path changed - FallbackToDefault setting changed
-    (affects fallback URL generation) - Default version changed (affects which
-    child is the default)
-
-func ShouldUpdateOldDefaultChild(
-	setDefault bool,
-	oldDefaultVersion string,
-	newDefaultVersion string,
-) bool
-    ShouldUpdateOldDefaultChild determines if the old default child API needs
-    server regeneration when creating a new child version with set_default=true.
-    This is necessary to remove the fallback URL from the old default child
-    since it's no longer the default.
-
-    Returns true when: - A new version is being set as default (setDefault=true)
-    - There was a previous default version - The default version actually
-    changed - The old default wasn't the base API itself ("Self")
+func RetainOldServerURL(oldServers, newServers openapi3.Servers) openapi3.Servers
+    RetainOldServerURL retains the first entry from old servers provided
+    tyk adds a server URL to the start of oas.Servers to add the gw URL
+    RetainOldServerURL can be used when API def is patched.
 
 func ValidateOASObject(documentBody []byte, oasVersion string) error
     ValidateOASObject validates an OAS document against a particular OAS
@@ -2850,14 +2795,6 @@
     DomainToCertificate holds a single mapping of domain name into a
     certificate.
 
-type EdgeEndpoint struct {
-	// Endpoint is the edge gateway URL (e.g., "http://edge1.example.com").
-	Endpoint string
-	// Tags are the tags associated with this edge gateway.
-	Tags []string
-}
-    EdgeEndpoint represents an edge gateway endpoint configuration.
-
 type EndpointPostPlugin struct {
 	// Enabled activates post plugin.
 	//
@@ -3800,21 +3737,6 @@
     any logic to ensure the Tyk oas API is backwards compatible with previous
     versions of the gateway should be placed in here.
 
-func (s *OAS) RegenerateServers(
-	newAPIData *apidef.APIDefinition,
-	oldAPIData *apidef.APIDefinition,
-	newBaseAPI *apidef.APIDefinition,
-	oldBaseAPI *apidef.APIDefinition,
-	config ServerRegenerationConfig,
-	versionName string,
-) error
-    RegenerateServers updates the servers section of an OAS API definition
-     1. Computes old Tyk-generated servers from oldAPIData state (if provided)
-     2. Removes old Tyk servers from the OAS spec
-     3. Generates new Tyk servers based on newAPIData configuration
-     4. Merges them: Tyk servers first, then user servers
-     5. Deduplicates by normalized URL
-
 func (s *OAS) RemoveServer(serverUrl string) error
     RemoveServer removes the server from the server list if it's
     already present. It accepts regex-based server URLs, such as
@@ -4503,17 +4425,6 @@
 func (s *Server) Fill(api apidef.APIDefinition)
     Fill fills *Server from apidef.APIDefinition.
 
-type ServerRegenerationConfig struct {
-	// Protocol is the URL scheme (http:// or https://).
-	Protocol string
-	// DefaultHost is the default gateway host (e.g., "localhost:8080").
-	DefaultHost string
-	// EdgeEndpoints contains edge gateway configurations.
-	EdgeEndpoints []EdgeEndpoint
-}
-    ServerRegenerationConfig holds the configuration required for server URL
-    regeneration.
-
 type ServiceDiscovery struct {
 	// Enabled activates Service Discovery.
 	//

@probelabs
Copy link
Copy Markdown
Contributor

probelabs Bot commented Nov 5, 2025

🔍 Code Analysis Results

This PR updates the Go version used in release pipelines to 1.24-bookworm to incorporate the latest security patches for the Go standard library.

In addition to the version bump, this PR introduces several other significant changes:

  • FIPS-compliant Build: A new FIPS build variant (tyk-gateway-fips) is introduced, enabled by the GOEXPERIMENT=boringcrypto build flag in the GoReleaser configuration. The release workflow has been updated to build and push this new Docker image.
  • CI/CD Improvements: The package upgrade tests are now more robust, handling cases where a previous package version is not available (useful for new product launches).
  • Bug Fixes: The PR includes fixes for local Docker builds (ci/Dockerfile.std) and corrects reversed logic in the post-installation script (ci/install/post_install.sh).

Files Changed Analysis

The changes are concentrated in CI/CD and build configuration files:

  • .github/workflows/release.yml: This file contains the most substantial changes (+88/-19). It updates the Go version, adds a complete build-and-push pipeline for the new FIPS Docker image, and improves the resilience of the upgrade test steps by ignoring errors if a previous version isn't found.
  • .github/workflows/plugin-compiler-build.yml: A single-line change to update the Go version to 1.24-bookworm.
  • ci/goreleaser/goreleaser.yml: Enables the FIPS build by adding the GOEXPERIMENT=boringcrypto environment variable.
  • ci/install/post_install.sh: Fixes reversed logic for service file cleanup and adds a check for the config file's existence before setting its permissions.
  • ci/Dockerfile.std & ci/images/plugin-compiler/Dockerfile: These files receive minor updates to base images and build steps. Notably, the plugin compiler Dockerfile is updated to a specific bullseye patch version, which is inconsistent with the other bookworm updates.

Architecture & Impact Assessment

  • What this PR accomplishes:

    1. Security Hardening: Mitigates vulnerabilities in the Go stdlib by updating the build toolchain.
    2. New Product Variant: Introduces a FIPS-compliant build and Docker image (tyk-gateway-fips) for use in regulated environments.
    3. CI/CD Robustness: Improves the reliability of the release pipeline by making package upgrade tests more fault-tolerant.
  • Key technical changes introduced:

    • The Go builder image is updated from 1.24-bullseye to 1.24-bookworm across release workflows.
    • FIPS-compliant builds are enabled using the GOEXPERIMENT=boringcrypto flag.
    • The GitHub Actions release workflow now builds and publishes a new tyk-gateway-fips Docker image.
    • Error handling in package installation tests is improved with || echo "..." to prevent failures on fresh installations.
  • Affected system components:

    • CI/CD Pipeline: The release and plugin-compiler-build workflows are directly modified.
    • Build Process: The goreleaser configuration is updated to produce a new FIPS build artifact.
    • Release Artifacts: All binaries will now be compiled with the updated Go version. A new Docker image, tykio/tyk-gateway-fips, will be published.
  • Component Interaction Diagram:

    graph TD
        subgraph "GitHub Actions Release Workflow"
            A[Start Build with Go 1.24-bookworm] --> B[Run GoReleaser];
            subgraph "GoReleaser Artifacts"
                B -- default --> C[Standard & EE Binaries];
                B -- GOEXPERIMENT=boringcrypto --> D[FIPS Binary];
            end
            C --> E[Build & Push Standard/EE Docker Images];
            D --> F[Build & Push FIPS Docker Image];
            E & F --> G[Run Install & Upgrade Tests on Packages];
        end
    
    Loading

Scope Discovery & Context Expansion

  • The introduction of an officially supported FIPS build is a significant product-level change. This will require corresponding updates in user-facing documentation, deployment guides, and most importantly, the tyk-charts repository to allow users to select this new image variant in Helm deployments.
  • The Docker build and push logic in .github/workflows/release.yml is now duplicated three times for the ee, fips, and std images. This increases maintenance overhead and violates the DRY (Don't Repeat Yourself) principle. Refactoring this into a reusable workflow or composite action would be beneficial.
  • There is an inconsistency in the base images used. While most of the pipeline moves to 1.24-bookworm, the plugin compiler (ci/images/plugin-compiler/Dockerfile) is updated to tykio/golang-cross:1.24.9-bullseye. This should be aligned unless there is a specific reason for the deviation.
  • The upgrade tests have been made more resilient, but they now use fixed sleep durations to wait for services to start. This can lead to flaky tests. A more robust approach would be to poll a health check endpoint.
Metadata
  • Review Effort: 3 / 5
  • Primary Label: bug

Powered by Visor from Probelabs

Last updated: 2025-11-10T15:44:51.452Z | Triggered by: synchronize | Commit: a0e1395

💡 TIP: You can chat with Visor using /visor ask <your question>

@probelabs
Copy link
Copy Markdown
Contributor

probelabs Bot commented Nov 5, 2025

🔍 Code Analysis Results

Security Issues (2)

Severity Location Issue
🟡 Warning .github/workflows/release.yml:200-232
The FIPS-compliant build steps are configured with a conditional that checks for `1.24-bullseye`, but the job matrix has been updated to use `1.24-bookworm`. This will prevent the FIPS image from being built and published.
💡 SuggestionUpdate the `if` condition in the FIPS-related steps to check for the correct matrix variable value (`1.24-bookworm`) to ensure the FIPS build and push steps are executed.
🟡 Warning ci/images/plugin-compiler/Dockerfile:2
The plugin compiler's Dockerfile is pinned to a `bullseye`-based image, while other CI jobs are being updated to `bookworm`. This inconsistency could lead to the plugin compiler being built on an older OS without the latest security patches.
💡 SuggestionTo ensure consistency and security across all build artifacts, update the base image to a `bookworm`-based one, such as `tykio/golang-cross:1.24.9-bookworm`.

✅ Architecture Check Passed

No architecture issues found – changes LGTM.

Performance Issues (1)

Severity Location Issue
🟡 Warning .github/workflows/release.yml:571
The fixed wait time for the container to start in the RPM upgrade test was increased from 2 to 5 seconds. While this might be intended to improve test stability, using a fixed `sleep` can unnecessarily slow down the CI pipeline if the service starts faster. Conversely, it can still lead to flaky tests if the service takes longer than 5 seconds to become ready under heavy load.
💡 SuggestionReplace the fixed `sleep 5` with an active health check loop. This would involve polling a health endpoint on the container until it returns a success status (e.g., using `curl` in a retry loop with a timeout). This makes the wait time adaptive, improving both pipeline speed and test reliability.

Quality Issues (3)

Severity Location Issue
🔴 Critical .github/workflows/release.yml:203-255
The newly added steps for building and pushing FIPS Docker images use an incorrect condition. The `if` condition checks for `matrix.golang_cross == '1.24-bullseye'`, but the matrix strategy has been updated to use `1.24-bookworm`. As a result, these steps will be skipped, and the FIPS images will not be built or published.
💡 SuggestionUpdate the `if` conditions for all FIPS-related steps to use the new Go version string: `if: ${{ matrix.golang_cross == '1.24-bookworm' }}`.
🟡 Warning .github/workflows/release.yml:452-511
The `docker run` commands in the `upgrade-tests-deb` (line 452) and `upgrade-tests-rpm` (line 511) jobs are missing the `--rm` flag. This will leave stopped containers on the runner after the tests, consuming resources until the job finishes.
💡 SuggestionAdd the `--rm` flag to both `docker run` commands to ensure the containers are automatically cleaned up after they exit.
🟡 Warning ci/images/plugin-compiler/Dockerfile:2
The base image in this Dockerfile is updated to `tykio/golang-cross:1.24.9-bullseye`. While pinning to a specific patch version is a good practice, the rest of the pull request focuses on migrating from `bullseye` to `bookworm` based images. This inconsistency might be an oversight and could lead to using an older OS distribution for plugin compilation than for the main product.
💡 SuggestionConsider aligning this Dockerfile with the other changes by updating the base image to a `bookworm`-based one, for example `tykio/golang-cross:1.24-bookworm`. If this image must remain on `bullseye` for specific reasons, it would be beneficial to add a comment explaining why.

✅ Dependency Check Passed

No dependency issues found – changes LGTM.

✅ Connectivity Check Passed

No connectivity issues found – changes LGTM.


Powered by Visor from Probelabs

Last updated: 2025-11-10T15:44:52.686Z | Triggered by: synchronize | Commit: a0e1395

💡 TIP: You can chat with Visor using /visor ask <your question>

Razeen-Abdal-Rahman and others added 3 commits November 5, 2025 15:59
Co-authored-by: Gromit <policy@gromit>
Co-authored-by: Gromit <policy@gromit>
@Razeen-Abdal-Rahman Razeen-Abdal-Rahman requested a review from a team as a code owner November 5, 2025 23:00
Comment thread ci/images/plugin-compiler/Dockerfile Outdated
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Nov 6, 2025

🎯 Recommended Merge Targets

Based on JIRA ticket TT-16060: Fix release pipelines to pick up latest Go patch

Fix Version: Tyk 5.10.1

Required:

  • release-5.10.1 - Exact version branch for Tyk 5.10.1 - specific patch release
  • release-5.10 - Minor version branch for 5.10.x patches - required for creating Tyk 5.10.1
  • master - Main development branch - ensures fix is in all future releases

📋 Workflow

  1. Merge this PR to master first

  2. Cherry-pick to release branches by commenting on the merged PR:

    • /release to release-5.10.1
    • /release to release-5.10
  3. Automated backport - The bot will automatically create backport PRs to the specified release branches

@sonarqubecloud
Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants