Skip to content

[TT-15825] [BE] Address inconsistencies with use of Policy identifiers#7424

Merged
shults merged 22 commits intomasterfrom
TT-15825-be-address-inconsistencies-with-use-of-policy-identifiers
Nov 12, 2025
Merged

[TT-15825] [BE] Address inconsistencies with use of Policy identifiers#7424
shults merged 22 commits intomasterfrom
TT-15825-be-address-inconsistencies-with-use-of-policy-identifiers

Conversation

@shults
Copy link
Copy Markdown
Contributor

@shults shults commented Oct 8, 2025

User description

TT-15825
Summary [BE] Address inconsistencies with use of Policy identifiers
Type Bug Bug
Status In Dev
Points N/A
Labels 2025_long_tail, AI-Complexity-Medium, AI-Priority-High, codilime_refined, customer_bug, jira_escalated, stability_theme_auth

Description

Related Issue

Motivation and Context

How This Has Been Tested

Screenshots (if appropriate)

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Refactoring or add test (improvements in base code or adds test coverage to functionality)

Checklist

  • I ensured that the documentation is up to date
  • I explained why this PR updates go.mod in detail with reasoning why it's required
  • I would like a code coverage CI quality gate exception and have explained why

PR Type

Bug fix, Tests, Documentation


Description

  • Remove explicit policy ID toggle usage

  • Standardize policy ID to policy.ID

  • Update RPC/Dashboard loaders signatures

  • Adjust tests for new behavior


Diagram Walkthrough

flowchart LR
  Config["Config: deprecate allow_explicit_policy_id"] -- "not used" --> Loaders["Policy loaders"]
  Loaders["Policy loaders"] -- "use policy.ID only" --> Dashboard["LoadPoliciesFromDashboard"]
  Loaders -- "use policy.ID only" --> RPC["LoadPoliciesFromRPC/parsePoliciesFromRPC"]
  API["handleAddOrUpdatePolicy"] -- "backfill ID from MID" --> Filesave["Save policy file"]
  Tests["Tests updated"] -- "drop allowExplicit param" --> Dashboard
  Tests -- "expect IDs via policy.ID" --> RPC
  Backup["RPC backup handlers"] -- "parsePoliciesFromRPC()" --> RPC
Loading

File Walkthrough

Relevant files
Documentation
config.go
Deprecate unused AllowExplicitPolicyID config                       

config/config.go

  • Mark AllowExplicitPolicyID as deprecated.
  • Note it's unused in codebase.
+2/-0     
Bug fix
api.go
Backfill missing policy ID from MID on save                           

gateway/api.go

  • Default newPol.ID to newPol.MID.Hex() if empty.
  • Ensure saved filename uses resolved policy ID.
+4/-0     
Enhancement
policy.go
Standardize ID handling and simplify loader APIs                 

gateway/policy.go

  • Remove allowExplicit parameter from loaders/parsers.
  • Use policy.ID as key; drop MID fallback.
  • Adjust duplicate detection to use policy.ID.
  • Update retry path to new signatures.
+8/-18   
rpc_backup_handlers.go
Update RPC backup to new parsing/decrypt API                         

gateway/rpc_backup_handlers.go

  • Call parsePoliciesFromRPC without allowExplicit.
  • Use updated crypto decrypt signature.
+2/-2     
server.go
Remove allowExplicit parameter from sync paths                     

gateway/server.go

  • Call loaders without AllowExplicitPolicyID.
  • Keep source selection logic intact.
+2/-2     
Tests
policy_test.go
Align tests with standardized policy ID handling                 

gateway/policy_test.go

  • Update tests to new loader signatures.
  • Adjust expectations to lookup by policy.ID.
  • Remove dependency on config AllowExplicitPolicyID.
+28/-37 

Ticket Details

TT-15825
Status In Code Review
Summary [BE] Address inconsistencies with use of Policy identifiers

Generated at: 2025-11-12 14:28:57

@buger
Copy link
Copy Markdown
Member

buger commented Oct 8, 2025

I'm a bot and I 👍 this PR title. 🤖

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Oct 8, 2025

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪
🧪 PR contains tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Possible Issue

When creating or updating a policy, if the incoming policy has an empty ID, the code sets newPol.ID to newPol.MID.Hex(). Ensure MID is always populated for new policies; otherwise MID may be zero-value and .Hex() could yield an empty or invalid ID, leading to filename collisions or unexpected behavior.

if newPol.ID == "" {
	newPol.ID = newPol.MID.Hex()
}
Duplicate Handling

Policies fetched from the Dashboard now strictly use p.ID as the key and skip duplicates. Validate that upstream always sets ID (and uniquely) after deprecating explicit/Mongo ID switching; otherwise blank or reused IDs will cause policies to be dropped silently.

	if _, ok := policies[p.ID]; ok {
		log.WithFields(logrus.Fields{
			"prefix":   "policy",
			"policyID": p.ID,
			"OrgID":    p.OrgID,
		}).Warning("--> Skipping policy, new item has a duplicate ID!")
		continue
	}
	policies[p.ID] = p.ToRegularPolicy()
}
Crypto API Change

crypto.Decrypt is now called with secret (string/bytes change). Confirm the function signature expects the provided type; mismatches could cause decryption failure and policy-loading errors in backup recovery.

secret := crypto.GetPaddedString(gw.GetConfig().Secret)
cryptoText, err := store.GetKey(checkKey)
listAsString := crypto.Decrypt(secret, cryptoText)

if err != nil {
	return nil, errors.New("[RPC] --> Failed to get node policy backup (" + checkKey + "): " + err.Error())
}

if policies, err := parsePoliciesFromRPC(listAsString); err != nil {
	log.WithFields(logrus.Fields{

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Oct 8, 2025

API Changes

--- prev.txt	2025-11-12 14:29:17.975838698 +0000
+++ current.txt	2025-11-12 14:29:07.912835015 +0000
@@ -6988,6 +6988,8 @@
 	// If you set this value to `true`, then the id parameter in a stored policy (or imported policy using the Dashboard API), will be used instead of the internal ID.
 	//
 	// This option should only be used when moving an installation to a new database.
+	//
+	// Deprecated. Is not used in codebase.
 	AllowExplicitPolicyID bool `json:"allow_explicit_policy_id"`
 	// This option only applies in OSS deployment when the `policies.policy_source` is either set
 	// to `file` or an empty string. If `policies.policy_path` is set, then Tyk will load policies
@@ -10130,11 +10132,11 @@
 
 func (gw *Gateway) LoadDefinitionsFromRPCBackup() ([]*APISpec, error)
 
-func (gw *Gateway) LoadPoliciesFromDashboard(endpoint, secret string, allowExplicit bool) (map[string]user.Policy, error)
+func (gw *Gateway) LoadPoliciesFromDashboard(endpoint, secret string) (map[string]user.Policy, error)
     LoadPoliciesFromDashboard will connect and download Policies from a Tyk
     Dashboard instance.
 
-func (gw *Gateway) LoadPoliciesFromRPC(store RPCDataLoader, orgId string, allowExplicit bool) (map[string]user.Policy, error)
+func (gw *Gateway) LoadPoliciesFromRPC(store RPCDataLoader, orgId string) (map[string]user.Policy, error)
 
 func (gw *Gateway) LoadPoliciesFromRPCBackup() (map[string]user.Policy, error)
 

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Oct 8, 2025

PR Code Suggestions ✨

Explore these optional code suggestions:

CategorySuggestion                                                                                                                                    Impact
Possible issue
Normalize empty IDs from dashboard

Handle empty p.ID values to avoid inserting entries under an empty key and silently
dropping multiple policies. Default empty IDs to p.MID.Hex() (or another stable
identifier) before deduplication.

gateway/policy.go [184-194]

 for _, p := range list.Message {
+	if p.ID == "" {
+		if hex := p.MID.Hex(); hex != "" {
+			p.ID = hex
+		} else {
+			p.ID = persistentmodel.NewObjectID().Hex()
+		}
+	}
 	if _, ok := policies[p.ID]; ok {
 		log.WithFields(logrus.Fields{
 			"prefix":   "policy",
 			"policyID": p.ID,
 			"OrgID":    p.OrgID,
 		}).Warning("--> Skipping policy, new item has a duplicate ID!")
 		continue
 	}
 	policies[p.ID] = p.ToRegularPolicy()
 }
Suggestion importance[1-10]: 8

__

Why: Addresses potential empty p.ID leading to empty-key map entries and lost policies; aligns with prior behavior using MID. High impact on correctness if IDs can be empty.

Medium
Derive IDs for RPC policies

Prevent empty-key insertions when parsing policies from RPC by deriving an ID if
p.ID is empty. Use p.MID.Hex() or generate a new ID to ensure map integrity and
consistent lookups.

gateway/policy.go [208-210]

 for _, p := range dbPolicyList {
+	if p.ID == "" {
+		if hex := p.MID.Hex(); hex != "" {
+			p.ID = hex
+		} else {
+			p.ID = persistentmodel.NewObjectID().Hex()
+		}
+	}
 	policies[p.ID] = p
 }
Suggestion importance[1-10]: 8

__

Why: Prevents empty-key insertions when parsing RPC data by deriving IDs, restoring robustness previously ensured by MID. Important for data integrity.

Medium
Ensure non-empty policy ID fallback

Guard against cases where newPol.MID is a zero-value and Hex() may return an empty
string, leading to an empty filename and potential overwrite or write errors.
Fallback to generating a stable unique ID (e.g., a new ObjectID) when both ID and
MID.Hex() are empty.

gateway/api.go [936-938]

 if newPol.ID == "" {
-	newPol.ID = newPol.MID.Hex()
+	if hex := newPol.MID.Hex(); hex != "" {
+		newPol.ID = hex
+	} else {
+		newPol.ID = persistentmodel.NewObjectID().Hex()
+	}
 }
Suggestion importance[1-10]: 7

__

Why: Valid enhancement to avoid empty IDs if newPol.MID is zero; improves robustness when creating filenames. Not critical but prevents edge-case failures.

Medium

@probelabs
Copy link
Copy Markdown
Contributor

probelabs Bot commented Oct 8, 2025

🔍 Code Analysis Results

This PR addresses inconsistencies in policy identifier handling by standardizing on policy.ID and deprecating the AllowExplicitPolicyID configuration. To ensure backward compatibility, policy.ID is backfilled from policy.MID if it is not explicitly set. This change simplifies policy loading logic from both the Dashboard and RPC sources.

A significant security enhancement is the introduction of the internal/osutil package, which provides a sandboxed filesystem utility to prevent path traversal vulnerabilities. This is now used by the policy management API (/tyk/policies/*) when writing and deleting policy files.

Files Changed Analysis

  • Total Files Changed: 14
  • Additions/Deletions: 725 additions, 81 deletions.
  • Notable Patterns:
    • Logic Simplification: The allowExplicit boolean parameter has been removed from LoadPoliciesFromDashboard, LoadPoliciesFromRPC, and parsePoliciesFromRPC, streamlining their function signatures and removing conditional logic.
    • Security Hardening: The new internal/osutil package provides a secure wrapper for file I/O operations, which has been applied to the policy API handlers in gateway/api.go.
    • Enhanced Testing: Substantial new tests have been added to gateway/api_test.go, gateway/policy_test.go, and gateway/server_test.go to validate the new ID handling and API behavior. A mock for RPCDataLoader was also added to support these tests.

Architecture & Impact Assessment

  • What this PR accomplishes: It refactors the gateway's policy management to use a single, consistent identifier (policy.ID), removing ambiguity and a redundant configuration flag. It also hardens the file-based policy storage against path traversal vulnerabilities.
  • Key technical changes introduced:
    1. Deprecation of AllowExplicitPolicyID: The configuration in config/config.go is marked as deprecated and its usage is removed from the codebase.
    2. ID Backfilling: The ensurePolicyId function in gateway/policy.go provides a compatibility layer by populating policy.ID from policy.MID if it's missing.
    3. Secure File I/O: The internal/osutil package introduces a Root object that scopes all file operations (write, remove, stat) to a specific base directory.
  • Affected system components:
    • Gateway Policy Loading (from Dashboard, RPC, and file sources).
    • Gateway REST API for policies (/tyk/policies).
graph TD
    subgraph "Configuration"
        A[tyk.conf: allow_explicit_policy_id] -- "Marked as Deprecated" --> B(No longer used in code)
    end

    subgraph "Policy Loading & Identification"
        C[Policy Loaders] -- "Signature simplified (no 'allowExplicit')" --> D{LoadPoliciesFromDashboard}
        C -- "Signature simplified (no 'allowExplicit')" --> E{LoadPoliciesFromRPC}
        F[ensurePolicyId helper] -- "If policy.ID is empty" --> G[policy.ID = policy.MID.Hex()]
        D -- "Uses" --> F
        E -- "Uses" --> F
    end

    subgraph "API Security Enhancement"
        H[Gateway Policy API<br>/tyk/policies] -- "File operations" --> I[New osutil package]
        I -- "Scopes FS access<br>Prevents Path Traversal" --> J[Policy JSON files]
    end

    D & E -- "Load policies into map" --> K[Policy Map (keyed by policy.ID)]
Loading

Scope Discovery & Context Expansion

The introduction of the internal/osutil package is a valuable security improvement. However, its usage is currently limited to the policy management API handlers.

Other parts of the gateway's management API that perform file I/O could also benefit from this enhancement. Specifically, the API Definition handlers in gateway/api.go (handleAddApi, handleUpdateApi, and handleDeleteAPI) still use direct, unsandboxed functions like ioutil.WriteFile and os.Remove.

Recommendation: To ensure consistent security across the gateway, the API Definition handlers should be refactored to use the new osutil.Root utility. This would extend the same path traversal protection to the /tyk/apis/* endpoints.

Metadata
  • Review Effort: 4 / 5
  • Primary Label: bug

Powered by Visor from Probelabs

Last updated: 2025-11-12T14:32:34.531Z | Triggered by: synchronize | Commit: 3d1d1b0

💡 TIP: You can chat with Visor using /visor ask <your question>

@probelabs
Copy link
Copy Markdown
Contributor

probelabs Bot commented Oct 8, 2025

🔍 Code Analysis Results

Security Issues (1)

Severity Location Issue
🟠 Error internal/osutil/osutil.go:29-41
The path traversal protection implemented in `osutil.Ensure` can be bypassed using symbolic links. The function validates that the cleaned path is prefixed with the root directory, but it does not resolve symlinks. If an attacker can create a symlink within the policies directory pointing to an arbitrary location (e.g., `/`), they could then use the policy management API to read, write, or delete files outside the intended directory, with the privileges of the Tyk gateway process. This could lead to information disclosure, denial of service, or potentially remote code execution.
💡 SuggestionStrengthen the path validation to be aware of symlinks. Before performing file operations, check the path for symlinks. For operations that act on existing files (read, delete, update), use `os.Lstat` on each component of the user-provided path to ensure none are symlinks. For creating new files, `os.OpenFile` with `O_CREATE|O_EXCL` flags can prevent overwriting an existing file or symlink. A comprehensive solution might involve resolving the real path with `filepath.EvalSymlinks` on all but the last path component and ensuring it remains within the root directory.

Architecture Issues (1)

Severity Location Issue
🟡 Warning gateway/api.go:971-975
The `handleDeletePolicy` function incorrectly returns a 500 Internal Server Error if the policy file to be deleted does not exist. According to HTTP semantics, a DELETE request on a non-existent resource should be idempotent and return a success status (e.g., 204 No Content) or a 404 Not Found, not a server error. The current implementation also introduces a potential Time-of-check to time-of-use (TOCTOU) race condition by checking for the file's existence before attempting to delete it.
💡 SuggestionRefactor the function to attempt the `Remove` operation directly and handle the `os.IsNotExist` error specifically. This removes the race condition and allows for correct HTTP status code handling.

Performance Issues (1)

Severity Location Issue
🟡 Warning gateway/api.go:945
The `newPolicyPathRoot` function is called on every request to `handleAddOrUpdatePolicy` (L945) and `handleDeletePolicy` (L966). This function performs filesystem I/O (`filepath.Abs` and `os.Stat`) to validate the policy directory. Since the policy path is configured and does not change per-request, these repeated I/O operations are inefficient, especially under a high rate of policy updates via the API.
💡 SuggestionCache the `*osutil.Root` object on the `Gateway` struct. The cached object can be initialized on startup and invalidated when the gateway's configuration is reloaded. This will avoid filesystem stats on every policy management API call.

Quality Issues (1)

Severity Location Issue
🟡 Warning gateway/api_test.go:151
The test `fails if invalid config is provided` uses a hardcoded path `/etc/hosts` to test a scenario where the policy path is a file, not a directory. This makes the test brittle as it depends on the existence and nature of this file, which is not guaranteed across all operating systems (e.g., Windows).
💡 SuggestionTo make the test more robust and platform-independent, create a temporary file within the test itself and use its path. This ensures the test environment is controlled and predictable. The `os.CreateTemp` function with `t.TempDir()` is suitable for this.

✅ Dependency Check Passed

No dependency issues found – changes LGTM.

✅ Connectivity Check Passed

No connectivity issues found – changes LGTM.


Powered by Visor from Probelabs

Last updated: 2025-11-12T14:32:35.693Z | Triggered by: synchronize | Commit: 3d1d1b0

💡 TIP: You can chat with Visor using /visor ask <your question>

@shults shults force-pushed the TT-15825-be-address-inconsistencies-with-use-of-policy-identifiers branch 3 times, most recently from 6eef7e5 to 5fc2a48 Compare October 14, 2025 12:07
@shults shults force-pushed the TT-15825-be-address-inconsistencies-with-use-of-policy-identifiers branch 2 times, most recently from acb0367 to efc831e Compare October 21, 2025 17:57
@shults shults requested a review from a team as a code owner October 28, 2025 13:22
Comment thread internal/osutil/osutil.go Dismissed
@shults shults closed this Oct 28, 2025
@shults shults reopened this Oct 28, 2025
@shults shults force-pushed the TT-15825-be-address-inconsistencies-with-use-of-policy-identifiers branch from 56c8e93 to 311fddc Compare October 29, 2025 08:14
@shults shults force-pushed the TT-15825-be-address-inconsistencies-with-use-of-policy-identifiers branch from e7ee806 to a0b1126 Compare October 29, 2025 14:53
Comment thread gateway/api.go
Comment thread gateway/api.go Outdated
Comment thread gateway/api.go
Comment thread gateway/policy.go
Comment thread gateway/policy.go
Comment thread gateway/api.go Outdated
Comment thread gateway/api.go
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Nov 12, 2025

🎯 Recommended Merge Targets

Based on JIRA ticket TT-15825: [BE] Address inconsistencies with use of Policy identifiers

Fix Version: Tyk 5.11.0

⚠️ Warning: Expected release branches not found in repository

Required:

  • master - No matching release branches found. Fix will be included in future releases.

Fix Version: Tyk 5.8.9

Required:

  • release-5.8 - Minor version branch for 5.8.x patches - required for creating Tyk 5.8.9
  • master - Main development branch - ensures fix is in all future releases

📋 Workflow

  1. Merge this PR to master first

  2. Cherry-pick to release branches by commenting on the merged PR:

    • /release to release-5.8
  3. Automated backport - The bot will automatically create backport PRs to the specified release branches

@shults shults enabled auto-merge (squash) November 12, 2025 14:15
@shults shults force-pushed the TT-15825-be-address-inconsistencies-with-use-of-policy-identifiers branch from fa5b341 to 17c8a1d Compare November 12, 2025 14:27
@sonarqubecloud
Copy link
Copy Markdown

@shults shults disabled auto-merge November 12, 2025 16:13
@shults shults merged commit 39e1e56 into master Nov 12, 2025
48 of 49 checks passed
@shults shults deleted the TT-15825-be-address-inconsistencies-with-use-of-policy-identifiers branch November 12, 2025 16:14
@shults
Copy link
Copy Markdown
Contributor Author

shults commented Nov 12, 2025

/release to release-5.8

@shults
Copy link
Copy Markdown
Contributor Author

shults commented Nov 12, 2025

/release to release-5.11

@probelabs
Copy link
Copy Markdown
Contributor

probelabs Bot commented Nov 12, 2025

❌ Cherry-pick failed. Please check the workflow logs.

@probelabs
Copy link
Copy Markdown
Contributor

probelabs Bot commented Nov 12, 2025

⚠️ Cherry-pick encountered conflicts. A draft PR was created: #7536

shults added a commit that referenced this pull request Nov 12, 2025
#7424)

<details open>
<summary><a href="https://tyktech.atlassian.net/browse/TT-15825"
title="TT-15825" target="_blank">TT-15825</a></summary>
  <br />
  <table>
    <tr>
      <th>Summary</th>
<td>[BE] Address inconsistencies with use of Policy identifiers</td>
    </tr>
    <tr>
      <th>Type</th>
      <td>
<img alt="Bug"
src="https://tyktech.atlassian.net/rest/api/2/universal_avatar/view/type/issuetype/avatar/10303?size=medium"
/>
        Bug
      </td>
    </tr>
    <tr>
      <th>Status</th>
      <td>In Dev</td>
    </tr>
    <tr>
      <th>Points</th>
      <td>N/A</td>
    </tr>
    <tr>
      <th>Labels</th>
<td><a
href="https://tyktech.atlassian.net/issues?jql=project%20%3D%20TT%20AND%20labels%20%3D%202025_long_tail%20ORDER%20BY%20created%20DESC"
title="2025_long_tail">2025_long_tail</a>, <a
href="https://tyktech.atlassian.net/issues?jql=project%20%3D%20TT%20AND%20labels%20%3D%20AI-Complexity-Medium%20ORDER%20BY%20created%20DESC"
title="AI-Complexity-Medium">AI-Complexity-Medium</a>, <a
href="https://tyktech.atlassian.net/issues?jql=project%20%3D%20TT%20AND%20labels%20%3D%20AI-Priority-High%20ORDER%20BY%20created%20DESC"
title="AI-Priority-High">AI-Priority-High</a>, <a
href="https://tyktech.atlassian.net/issues?jql=project%20%3D%20TT%20AND%20labels%20%3D%20codilime_refined%20ORDER%20BY%20created%20DESC"
title="codilime_refined">codilime_refined</a>, <a
href="https://tyktech.atlassian.net/issues?jql=project%20%3D%20TT%20AND%20labels%20%3D%20customer_bug%20ORDER%20BY%20created%20DESC"
title="customer_bug">customer_bug</a>, <a
href="https://tyktech.atlassian.net/issues?jql=project%20%3D%20TT%20AND%20labels%20%3D%20jira_escalated%20ORDER%20BY%20created%20DESC"
title="jira_escalated">jira_escalated</a>, <a
href="https://tyktech.atlassian.net/issues?jql=project%20%3D%20TT%20AND%20labels%20%3D%20stability_theme_auth%20ORDER%20BY%20created%20DESC"
title="stability_theme_auth">stability_theme_auth</a></td>
    </tr>
  </table>
</details>
<!--
  do not remove this marker as it will break jira-lint's functionality.
  added_by_jira_lint
-->

---

<!-- Provide a general summary of your changes in the Title above -->

<!-- Describe your changes in detail -->

<!-- This project only accepts pull requests related to open issues. -->
<!-- If suggesting a new feature or change, please discuss it in an
issue first. -->
<!-- If fixing a bug, there should be an issue describing it with steps
to reproduce. -->
<!-- OSS: Please link to the issue here. Tyk: please create/link the
JIRA ticket. -->

<!-- Why is this change required? What problem does it solve? -->

<!-- Please describe in detail how you tested your changes -->
<!-- Include details of your testing environment, and the tests -->
<!-- you ran to see how your change affects other areas of the code,
etc. -->
<!-- This information is helpful for reviewers and QA. -->

<!-- What types of changes does your code introduce? Put an `x` in all
the boxes that apply: -->

- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [x] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Refactoring or add test (improvements in base code or adds test
coverage to functionality)

<!-- Go over all the following points, and put an `x` in all the boxes
that apply -->
<!-- If there are no documentation updates required, mark the item as
checked. -->
<!-- Raise up any additional concerns not covered by the checklist. -->

- [ ] I ensured that the documentation is up to date
- [ ] I explained why this PR updates go.mod in detail with reasoning
why it's required
- [ ] I would like a code coverage CI quality gate exception and have
explained why

___

Bug fix, Tests, Documentation

___

- Remove explicit policy ID toggle usage

- Standardize policy ID to `policy.ID`

- Update RPC/Dashboard loaders signatures

- Adjust tests for new behavior

___

```mermaid
flowchart LR
  Config["Config: deprecate allow_explicit_policy_id"] -- "not used" --> Loaders["Policy loaders"]
  Loaders["Policy loaders"] -- "use policy.ID only" --> Dashboard["LoadPoliciesFromDashboard"]
  Loaders -- "use policy.ID only" --> RPC["LoadPoliciesFromRPC/parsePoliciesFromRPC"]
  API["handleAddOrUpdatePolicy"] -- "backfill ID from MID" --> Filesave["Save policy file"]
  Tests["Tests updated"] -- "drop allowExplicit param" --> Dashboard
  Tests -- "expect IDs via policy.ID" --> RPC
  Backup["RPC backup handlers"] -- "parsePoliciesFromRPC()" --> RPC
```

<details> <summary><h3> File Walkthrough</h3></summary>

<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Documentation</strong></td><td><table>
<tr>
  <td>
    <details>
<summary><strong>config.go</strong><dd><code>Deprecate unused
AllowExplicitPolicyID config</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

config/config.go

<ul><li>Mark <code>AllowExplicitPolicyID</code> as deprecated.<br> <li>
Note it's unused in codebase.</ul>

</details>

  </td>
<td><a
href="https://github.com/TykTechnologies/tyk/pull/7424/files#diff-fe44f09c4d5977b5f5eaea29170b6a0748819c9d02271746a20d81a5f3efca17">+2/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></td></tr><tr><td><strong>Bug fix</strong></td><td><table>
<tr>
  <td>
    <details>
<summary><strong>api.go</strong><dd><code>Backfill missing policy ID
from MID on save</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

gateway/api.go

<ul><li>Default <code>newPol.ID</code> to <code>newPol.MID.Hex()</code>
if empty.<br> <li> Ensure saved filename uses resolved policy ID.</ul>

</details>

  </td>
<td><a
href="https://github.com/TykTechnologies/tyk/pull/7424/files#diff-644cda3aeb4ac7f325359e85fcddb810f100dd5e6fa480b0d9f9363a743c4e05">+4/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></td></tr><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
  <td>
    <details>
<summary><strong>policy.go</strong><dd><code>Standardize ID handling and
simplify loader APIs</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

gateway/policy.go

<ul><li>Remove <code>allowExplicit</code> parameter from
loaders/parsers.<br> <li> Use <code>policy.ID</code> as key; drop MID
fallback.<br> <li> Adjust duplicate detection to use
<code>policy.ID</code>.<br> <li> Update retry path to new
signatures.</ul>

</details>

  </td>
<td><a
href="https://github.com/TykTechnologies/tyk/pull/7424/files#diff-ec674104322b26b82def55e9be32117753ab66e7840490481eb6eb4c15bc35e7">+8/-18</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>rpc_backup_handlers.go</strong><dd><code>Update RPC
backup to new parsing/decrypt API</code>&nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></summary>
<hr>

gateway/rpc_backup_handlers.go

<ul><li>Call <code>parsePoliciesFromRPC</code> without
allowExplicit.<br> <li> Use updated crypto decrypt signature.</ul>

</details>

  </td>
<td><a
href="https://github.com/TykTechnologies/tyk/pull/7424/files#diff-69d9cb8df2bd4296a8e5e5d769009a09bd61ca65b7dbcbf29751af92698bd9ce">+2/-2</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>server.go</strong><dd><code>Remove allowExplicit
parameter from sync paths</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

gateway/server.go

<ul><li>Call loaders without <code>AllowExplicitPolicyID</code>.<br>
<li> Keep source selection logic intact.</ul>

</details>

  </td>
<td><a
href="https://github.com/TykTechnologies/tyk/pull/7424/files#diff-4652d1bf175a0be8f5e61ef7177c9666f23e077d8626b73ac9d13358fa8b525b">+2/-2</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></td></tr><tr><td><strong>Tests</strong></td><td><table>
<tr>
  <td>
    <details>
<summary><strong>policy_test.go</strong><dd><code>Align tests with
standardized policy ID handling</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

gateway/policy_test.go

<ul><li>Update tests to new loader signatures.<br> <li> Adjust
expectations to lookup by <code>policy.ID</code>.<br> <li> Remove
dependency on config <code>AllowExplicitPolicyID</code>.</ul>

</details>

  </td>
<td><a
href="https://github.com/TykTechnologies/tyk/pull/7424/files#diff-40d701767204255c38c7dd64939d6bb8df621640c4bddfe5f56080380476a18a">+28/-37</a>&nbsp;
</td>

</tr>
</table></td></tr></tr></tbody></table>

</details>

___

<!---TykTechnologies/jira-linter starts here-->

<details>
<summary>
<a href="https://tyktech.atlassian.net/browse/TT-15825" title="TT-15825"
target="_blank">TT-15825</a>
</summary>

|         |    |
|---------|----|
| Status  | In Code Review |
| Summary | [BE] Address inconsistencies with use of Policy identifiers
|

Generated at: 2025-11-12 14:28:57

</details>

<!---TykTechnologies/jira-linter ends here-->

---------

Co-authored-by: Radosław Krawczyk <radoslawkrawczyk@ST11253-radkra.local>
(cherry picked from commit 39e1e56)
shults added a commit that referenced this pull request Nov 12, 2025
…use of Policy identifiers (#7424) (#7536)

Cherry-pick of `39e1e5660b67c3b62e499bcc3fa5939b1a44e4cb` from `master`
to `release-5.8` requires manual resolution.

  **Conflicts detected:** 3
   - gateway/api_test.go
  
  Tips:
- Check out this branch locally and run: `git cherry-pick -x
39e1e56`
- Resolve conflicts (including submodules if any), then push back to
this branch.
  
Original commit:
39e1e56
  
<!---TykTechnologies/jira-linter starts here-->

### Ticket Details

<details>
<summary>
<a href="https://tyktech.atlassian.net/browse/TT-15825" title="TT-15825"
target="_blank">TT-15825</a>
</summary>

|         |    |
|---------|----|
| Status  | In Code Review |
| Summary | [BE] Address inconsistencies with use of Policy identifiers
|

Generated at: 2025-11-12 17:33:09

</details>

<!---TykTechnologies/jira-linter ends here-->

---------

Co-authored-by: Tyk Bot <bot@tyk.io>
Co-authored-by: Yaroslav Kotsur <yarikkotsur@gmail.com>
Co-authored-by: Radosław Krawczyk <radoslawkrawczyk@ST11253-radkra.local>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants