Support pipPackage in build.yaml the same way gomodule#1782
Merged
renuka-fernando merged 27 commits intomainfrom Apr 28, 2026
Merged
Support pipPackage in build.yaml the same way gomodule#1782renuka-fernando merged 27 commits intomainfrom
renuka-fernando merged 27 commits intomainfrom
Conversation
Add a new prompt-compressor policy (sample implementation, policy-definition, requirements, and __init__) plus a default policy and a Python executor unit test. Include the policy in build manifests (filePath and version v0.1.0). Bump versions for multiple existing policies in build-manifest.yaml/build.yaml, add a Postman collection for the gateway API, and apply small updates to generated API code, the gateway runtime Dockerfile, and Makefile.
Add support for direct pip specifications (git+, VCS and PEP 508 "name @ url" forms) in the gateway builder. Introduces sanitizePipSpec, isDirectPipSpec and fetchDirectPipPackage and wires detection into FetchPipPackage so VCS/file/URL-based packages are fetched via `pip wheel --no-deps` and extracted. Error messages and logs sanitize credentials before emitting them. Adds unit tests for direct-spec detection and a git-backed package download test. Update python-deps Dockerfile stage to install git so ephemeral builds can fetch VCS packages. Add a new sample policy package prompt-compressor (pyproject.toml, README, and __version__ in __init__.py).
Switch prompt-compressor packaging from a local filePath to a pipPackage pointing at the git repo (prompt-compressor-v0.1.0) in gateway/build.yaml and gateway/build-manifest.yaml. Bump subscription-validation policy version v1.0.1 -> v1.0.2 in build-manifest.yaml. Update gateway/gateway-runtime/Dockerfile to upgrade pip, setuptools and wheel before installing requirements so PEP 621 pyproject.toml and recent git-installed packages are supported.
Add robust pip package discovery: parse indexed and VCS pip specs into structured refs, support major-only version specifiers (~=major.0) and resolve them to exact tags for VCS refs, sanitize git URLs (remove credentials) and handle direct pip targets. Split fetching into indexed and VCS flows, add runPipCommand helper, read resolved version from wheel METADATA, and preserve the resolved exact pip spec for lockfile generation. Update generator to record resolved PipSpec into build manifest when available, add many unit tests (including VCS resolution and wheel METADATA reading), and tweak build.yaml to point the sample policy at the updated repo ref.
When multiple Python candidates share the same policy name, prefer a discovered pip package whose OriginalPipSpec exactly matches the manifest entry. Adds OriginalPipSpec to DiscoveredPolicy and sets it during discovery, updates the generator to first try matching OriginalPipSpec for python pip packages and fall back to any python candidate if no exact spec match is found. Includes a unit test to verify selecting the correct pip candidate and updates build/build-manifest entries for the prompt-compressor pip specs/versions.
Add support for Go-style short pip URLs and minor-only version resolution for Python policies. Implements short-URL expansion (e.g. github.com/org/repo/path@v1 -> full git+ URL with subdirectory and tag), generalized VCS ref classification (exact / minor-only / major-only), and resolution of partial refs via git ls-remote. Update indexed package parsing to accept ~=N.M.0 (minor-only) and rename the range flag to IsVersionRange. Tests and generator expectations updated; build.yaml and build-manifest.yaml examples adjusted. Also add feature-implementation-plan.md documenting design and verification plan.
Prompt compress
Delete Python sample/default policy files (prompt-compressor and python-* sample policies) and remove the feature implementation plan document. Update gateway/build.yaml to remove the prompt-compressor pip entry and adjust gateway/build-manifest.yaml policy versions (several v1.x.y entries downgraded to earlier patch versions). Also include a small doc comment tweak in generated API file (annotations example). These changes clean up sample policy artifacts and keep manifests consistent with the current packaged policies.
Rename the Python SDK package from wso2_gateway_policy_sdk to apip_sdk_core and update generator, tests and copy paths accordingly. Update sdk-python package layout (files renamed/moved) and adjust python-executor tests to expect apip_sdk_core. Bump multiple policy versions in gateway/build-manifest.yaml. Remove the large gateway-controller Postman collection (postman_collection.json). Other related python-executor and runtime test updates applied to reflect the SDK rename.
Prefer pip candidates when resolving python policies by requiring IsPipPackage, and add a test to ensure non-pip local candidates are skipped. Reject unsupported named direct pip references ("name @ url") with a clear error. Introduce buildExactIndexedPipSpec to construct exact indexed pip specs (including index URL) and avoid leaking raw index URLs in logs by sanitizing the logged value. Add unit tests for the new behaviors and update the PythonSourceDir comment to reflect extracted pip policies.
Add standalone apip-sdk-core Python package and packaging metadata for an initial 0.1.0 release. Files added: README (detailed usage, examples and API reference), CHANGELOG, LICENSE (Apache-2.0), .gitignore, tests, and package pyproject.toml updates (classifiers, license file, URLs, sdist/wheel settings, and dev extras). Update package root (__init__) to re-export versioned policy symbols and expose __version__ from package metadata. Add basic smoke tests for the public API and headers behavior. Remove the top-level py.typed marker (package-level typed marker expected in src/apip_sdk_core).
Co-authored-by: Renuka Piyumal Fernando <renukapiyumal@gmail.com>
Point README license badge and license reference to the repository LICENSE on GitHub instead of a local file. Also remove the author's email from the authors list in pyproject.toml (metadata cleanup/privacy). No functional code changes.
Delete sdk-python/LICENSE and remove license references from sdk-python/pyproject.toml: remove the license = { file = "LICENSE" } setting and the "LICENSE" entry from [tool.hatch.build.targets.sdist]. Other package metadata and included files remain unchanged.
Add a new gateway policy manifest (gateway/gateway-controller/default-policies/prompt-compressor.yaml) that compresses prompt text before upstream LLM calls. The policy targets a single string via jsonPath (default $.messages[0].content), supports ordered compression rules (upperTokenLimit, type: ratio|token, value) including a -1 fallback, and supports selective compression tags. Adjust sdk-python/src/apip_sdk_core/__init__.py to build __all__ by converting policy.__all__ to a list and extending it with "policy" and "__version__" to ensure consistent exports.
Update discovery tests to use resolvePipExecutable's args and skip early if setuptools or wheel are not available (run `pip show setuptools` and `pip show wheel`). This ensures the test environment can build Python packages. Also modify prompt-compressor.yaml to require at least one rule with `upperTokenLimit: -1` by adding a `contains` constraint and `minContains: 1`, enforcing a catch-all fallback rule.
…-support Python Policy Version Support
Contributor
|
Warning Rate limit exceeded
To keep reviews running without waiting, you can enable usage-based add-on for your organization. This allows additional reviews beyond the hourly cap. Account admins can enable it under billing. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (8)
📝 WalkthroughWalkthroughThe changes migrate the Python SDK from Sequence Diagram(s)sequenceDiagram
participant BM as Build Manifest Generator
participant Resolver as FetchPipPackage Resolver
participant Parser as Pip Spec Parser
participant Git as Git (ls-remote / fetch)
participant PipIdx as Pip Index / pip
participant Wheel as Wheel Extraction
BM->>Resolver: request resolution(spec)
Resolver->>Parser: classify spec (indexed / VCS / short form)
alt VCS (git+)
Parser-->>Resolver: VCS spec struct
Resolver->>Git: git ls-remote --tags (if range)
Git-->>Resolver: tag list / resolved ref
Resolver->>Git: fetch at resolved ref (with subdirectory)
Git-->>Wheel: provide wheel/artifact
else Indexed package
Parser-->>Resolver: Indexed spec struct
Resolver->>PipIdx: run pip to download wheel (resolve range -> exact)
PipIdx-->>Wheel: wheel file
end
Wheel->>Resolver: extract Version + top-level module(s)
Resolver->>BM: return resolved exact spec, PipSpec, OriginalPipSpec, PythonSourceDir
BM->>BM: select preferred candidate (match manifest pipPackage if present)
BM->>BM: write build-manifest.yaml with resolved pipPackage entries
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 5
🧹 Nitpick comments (1)
gateway/gateway-controller/default-policies/prompt-compressor.yaml (1)
53-57: Consider adding value range constraints.The
valuefield lacks minimum/maximum constraints. Forratiomode, values should logically be in the range(0, 1]. Adding schema-level validation would prevent invalid configurations at definition time rather than runtime.♻️ Optional: Add range constraints
value: type: number + minimum: 0 + exclusiveMinimum: true description: | Rule value. For "ratio", use a retained-size ratio such as 0.8. For "token", use the target retained token estimate.Note: If
tokenmode permits values greater than 1, a conditional schema or runtime validation may be more appropriate.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@gateway/gateway-controller/default-policies/prompt-compressor.yaml` around lines 53 - 57, Add schema-level validation for the rule "value" to prevent invalid configurations: for ratio mode, enforce 0 < value <= 1 by adding exclusiveMinimum: 0 and maximum: 1 (or use exclusiveMaximum if you want <1); for token mode allow larger integers or omit the upper bound. Prefer using a conditional schema (if: {properties:{mode:{const:"ratio"}}} then: {properties:{value:{exclusiveMinimum:0, maximum:1}}}) so the "value" constraints apply only when the rule's "mode" is "ratio"; refer to the "value" field and "mode" property in the prompt-compressor policy schema to locate where to add these constraints.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@gateway/gateway-builder/internal/buildfile/generator.go`:
- Around line 208-210: The generated manifest becomes sensitive when you persist
resolved pip specs (when found.IsPipPackage && found.PipSpec != "" and you
assign entry.PipPackage = found.PipSpec), so either redact credentials from
found.PipSpec before assigning to entry.PipPackage or ensure the file is created
with restrictive permissions; update the code that writes the manifest (the
os.WriteFile call that writes build-manifest.yaml) to use mode 0o600 instead of
0o644, or implement a redaction step that strips private index/VCS credentials
from found.PipSpec prior to assignment and serialization.
In `@gateway/gateway-builder/internal/discovery/python_module.go`:
- Around line 327-335: The wheel invocation currently forces build isolation off
by including "--no-build-isolation" in the args slice used by runPipCommand;
remove that flag from the args (the slice built just before calling
runPipCommand in the function that creates wheelDir) so pip can perform build
isolation and automatically install PEP 517 build-system.requires dependencies
when building the wheel; keep the other flags ("wheel", "--no-deps",
resolvedSpec, "-w", wheelDir) intact and call runPipCommand as before.
In `@gateway/gateway-controller/default-policies/prompt-compressor.yaml`:
- Around line 30-34: The schema uses the draft-2019-09 keyword "minContains"
which gojsonschema v1.2.0 does not support, so either remove the "minContains:
1" constraint from the "contains" block (the properties targeting
"upperTokenLimit") or upgrade the JSON Schema validator to one that supports
draft-2019-09; locate the "contains" section and the "upperTokenLimit" const in
prompt-compressor.yaml and either delete the "minContains: 1" line or update the
project's jsonschema library (or switch to a validator) so "minContains" is
honored during validation.
In `@sdk-python/src/apip_sdk_core/__init__.py`:
- Line 23: The __all__.extend call is adding names in non-alphabetical order
causing RUF022; update the __all__.extend usage so the added names are sorted
alphabetically (i.e., ensure "__version__" comes before "policy") or wrap the
added sequence in a sorted operation before extending; modify the __all__.extend
invocation in this file to use the sorted ordering of the symbols to satisfy
Ruff.
---
Nitpick comments:
In `@gateway/gateway-controller/default-policies/prompt-compressor.yaml`:
- Around line 53-57: Add schema-level validation for the rule "value" to prevent
invalid configurations: for ratio mode, enforce 0 < value <= 1 by adding
exclusiveMinimum: 0 and maximum: 1 (or use exclusiveMaximum if you want <1); for
token mode allow larger integers or omit the upper bound. Prefer using a
conditional schema (if: {properties:{mode:{const:"ratio"}}} then:
{properties:{value:{exclusiveMinimum:0, maximum:1}}}) so the "value" constraints
apply only when the rule's "mode" is "ratio"; refer to the "value" field and
"mode" property in the prompt-compressor policy schema to locate where to add
these constraints.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: fad79597-be2c-4a54-ab29-90eb858c01a3
📒 Files selected for processing (40)
gateway/gateway-builder/internal/buildfile/generator.gogateway/gateway-builder/internal/buildfile/generator_test.gogateway/gateway-builder/internal/discovery/discovery_test.gogateway/gateway-builder/internal/discovery/manifest.gogateway/gateway-builder/internal/discovery/python_module.gogateway/gateway-builder/internal/discovery/python_module_test.gogateway/gateway-builder/internal/policyengine/generator.gogateway/gateway-builder/internal/policyengine/policyengine_test.gogateway/gateway-builder/pkg/types/policy.gogateway/gateway-controller/default-policies/prompt-compressor.yamlgateway/gateway-controller/default-policies/python-body-buffer-modifier.yamlgateway/gateway-controller/default-policies/python-header-modifier.yamlgateway/gateway-controller/default-policies/python-streaming-logger.yamlgateway/gateway-runtime/Dockerfilegateway/gateway-runtime/python-executor/executor/__init__.pygateway/gateway-runtime/python-executor/executor/execution_tracker.pygateway/gateway-runtime/python-executor/executor/instance_store.pygateway/gateway-runtime/python-executor/executor/policy_loader.pygateway/gateway-runtime/python-executor/executor/server.pygateway/gateway-runtime/python-executor/executor/translator.pygateway/gateway-runtime/python-executor/tests/__init__.pygateway/gateway-runtime/python-executor/tests/test_execution_tracker.pygateway/gateway-runtime/python-executor/tests/test_sdk_types.pygateway/gateway-runtime/python-executor/tests/test_server.pygateway/gateway-runtime/python-executor/tests/test_translator.pysdk-python/.gitignoresdk-python/CHANGELOG.mdsdk-python/README.mdsdk-python/pyproject.tomlsdk-python/src/apip_sdk_core/__init__.pysdk-python/src/apip_sdk_core/policy/__init__.pysdk-python/src/apip_sdk_core/policy/v1alpha2/__init__.pysdk-python/src/apip_sdk_core/policy/v1alpha2/actions.pysdk-python/src/apip_sdk_core/policy/v1alpha2/policy.pysdk-python/src/apip_sdk_core/policy/v1alpha2/types.pysdk-python/src/apip_sdk_core/py.typedsdk-python/src/wso2_gateway_policy_sdk/__init__.pysdk-python/src/wso2_gateway_policy_sdk/py.typedsdk-python/tests/__init__.pysdk-python/tests/test_public_api.py
💤 Files with no reviewable changes (4)
- gateway/gateway-controller/default-policies/python-streaming-logger.yaml
- gateway/gateway-controller/default-policies/python-header-modifier.yaml
- sdk-python/src/wso2_gateway_policy_sdk/init.py
- gateway/gateway-controller/default-policies/python-body-buffer-modifier.yaml
Remove the minContains entry from gateway/gateway-controller/default-policies/prompt-compressor.yaml's parameter schema. Removal is purely cleanup with no functional change. Also update sdk-python/src/apip_sdk_core/__init__.py to extend __all__ with "__version__" before "policy" to correct the exported symbol ordering.
Replace raw pip spec values with sanitizePipSpec(...) in error messages to avoid leaking sensitive data (e.g. tokens in VCS/git+ direct references). Updates multiple validation/failure paths in gateway/gateway-builder/internal/discovery/python_module.go to use sanitized output when including the spec in formatted errors.
…-support Python policy version support
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
♻️ Duplicate comments (1)
gateway/gateway-builder/internal/discovery/python_module.go (1)
197-211:⚠️ Potential issue | 🟠 MajorSanitize the remaining short-URL error paths.
These returns still interpolate the raw spec, so malformed credentialed inputs can be echoed back in errors. Please pass the remaining
pipPackage/specvalues throughsanitizePipSpec(...)before returning them.Minimal fix
- return nil, fmt.Errorf("failed to expand short URL %q: %w", pipPackage, err) + return nil, fmt.Errorf("failed to expand short URL %q: %w", sanitizePipSpec(pipPackage), err) } @@ - return nil, fmt.Errorf("unrecognized pipPackage format: %q", pipPackage) + return nil, fmt.Errorf("unrecognized pipPackage format: %q", sanitizePipSpec(pipPackage)) @@ - return "", fmt.Errorf("short URL must contain '@version': %s", spec) + return "", fmt.Errorf("short URL must contain '@version': %s", sanitizePipSpec(spec)) @@ - "short URL must have at least 4 path segments (host/org/repo/subdir), got %d: %s", + "short URL must have at least 4 path segments (host/org/repo/subdir), got %d: %s", len(segments), - spec, + sanitizePipSpec(spec), )As per coding guidelines, "validate safety without exposing sensitive context."
Also applies to: 407-425
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@gateway/gateway-builder/internal/discovery/python_module.go` around lines 197 - 211, The error paths currently include raw pip specs (pipPackage and expanded) in returned errors; sanitize them before interpolating by wrapping values with sanitizePipSpec(...) wherever they are embedded in fmt.Errorf or logs. Specifically, in the expandShortURL error return and in the final unrecognized-format fmt.Errorf replace pipPackage with sanitizePipSpec(pipPackage), and likewise use sanitizePipSpec(expanded) when composing any error or log messages after calling expandShortURL; also apply the same sanitization to the analogous error/return sites in the 407-425 block. Ensure fetchVCSPipPackage continues to receive the original expanded value but never expose raw specs directly in error strings—always pass sanitized variants to fmt.Errorf or log calls.
🧹 Nitpick comments (1)
gateway/gateway-controller/default-policies/prompt-compressor.yaml (1)
43-57: Add mode-specific validation forvalue.Lines [52]-[56] allow any numeric
value, so invalid inputs can pass schema validation. Add conditional constraints soratioandtokenvalues are bounded appropriately.Suggested refactor
items: type: object additionalProperties: false properties: @@ value: type: number description: | Rule value. For "ratio", use a retained-size ratio such as 0.8. For "token", use the target retained token estimate. + allOf: + - if: + properties: + type: + const: ratio + then: + properties: + value: + exclusiveMinimum: 0 + maximum: 1 + - if: + properties: + type: + const: token + then: + properties: + value: + exclusiveMinimum: 0 required: - upperTokenLimit - type - value🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@gateway/gateway-controller/default-policies/prompt-compressor.yaml` around lines 43 - 57, The schema currently allows any numeric value for the "value" field; add JSON Schema conditional validation so "value" is constrained by the selected compression mode: use an if/then/else (or oneOf with schemas) keyed on the mode property (the string enum with values "ratio" and "token") so that when mode is "ratio" the value is a number between 0 and 1 (e.g., minimum: 0, maximum: 1) and when mode is "token" the value is an integer >= 1 (type: integer, minimum: 1); update the required list to ensure both the mode field (the string enum) and value are required so the conditionals always apply.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@gateway/gateway-builder/internal/discovery/python_module.go`:
- Around line 308-320: The short-URL path (e.g., "policy/v1" or "policy/v1.2")
is not recognized by classifyVCSRef, so those refs skip resolveVCSVersion and
remain unresolved; fix by normalizing the GitRef before classification — extract
the final segment after the last '/' (or otherwise strip the repo path) and pass
that to classifyVCSRef (or prepend a leading "v" when appropriate), then if
classifyVCSRef indicates major/minor call resolveVCSVersion and
rebuildVCSPipSpec as you already do; apply this normalization wherever
classifyVCSRef is used for VCS refs (including the occurrences around the shown
block and the other spots noted).
- Around line 240-252: The code uses pip download which can leave only sdists
and cause findWheelFile to fail; update the args construction in the block that
creates args := []string{"download", "--no-deps", downloadSpec, "-d",
downloadDir} to use "wheel" instead of "download" (keep the same flags like
--no-deps, -d and any --index-url handling), and ensure the subsequent
runPipCommand call (runPipCommand(args, 5*time.Minute, downloadSpec,
ref.IndexURL, "download")) is updated to reflect the wheel operation (adjust the
last argument from "download" to "wheel" or equivalent) so that pip builds a
wheel for findWheelFile to locate.
In `@gateway/gateway-controller/default-policies/prompt-compressor.yaml`:
- Around line 38-43: The schema for the YAML field upperTokenLimit currently
allows any integer; update the schema for upperTokenLimit (the integer type
under the policy prompt compressor) to disallow values less than -1 by adding a
minimum constraint of -1 (so valid values are >= -1) and ensure the description
still documents -1 as the fallback rule; adjust any validation logic that reads
upperTokenLimit to enforce this minimum if runtime checks exist.
---
Duplicate comments:
In `@gateway/gateway-builder/internal/discovery/python_module.go`:
- Around line 197-211: The error paths currently include raw pip specs
(pipPackage and expanded) in returned errors; sanitize them before interpolating
by wrapping values with sanitizePipSpec(...) wherever they are embedded in
fmt.Errorf or logs. Specifically, in the expandShortURL error return and in the
final unrecognized-format fmt.Errorf replace pipPackage with
sanitizePipSpec(pipPackage), and likewise use sanitizePipSpec(expanded) when
composing any error or log messages after calling expandShortURL; also apply the
same sanitization to the analogous error/return sites in the 407-425 block.
Ensure fetchVCSPipPackage continues to receive the original expanded value but
never expose raw specs directly in error strings—always pass sanitized variants
to fmt.Errorf or log calls.
---
Nitpick comments:
In `@gateway/gateway-controller/default-policies/prompt-compressor.yaml`:
- Around line 43-57: The schema currently allows any numeric value for the
"value" field; add JSON Schema conditional validation so "value" is constrained
by the selected compression mode: use an if/then/else (or oneOf with schemas)
keyed on the mode property (the string enum with values "ratio" and "token") so
that when mode is "ratio" the value is a number between 0 and 1 (e.g., minimum:
0, maximum: 1) and when mode is "token" the value is an integer >= 1 (type:
integer, minimum: 1); update the required list to ensure both the mode field
(the string enum) and value are required so the conditionals always apply.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 1fb5511e-7216-45bb-9227-8a22194ca4fd
📒 Files selected for processing (3)
gateway/gateway-builder/internal/discovery/python_module.gogateway/gateway-controller/default-policies/prompt-compressor.yamlsdk-python/src/apip_sdk_core/__init__.py
✅ Files skipped from review due to trivial changes (1)
- sdk-python/src/apip_sdk_core/init.py
Add detection and discovery support for Python policies using the industry-standard src layout and pyproject.toml. DetectRuntime now recognizes pyproject.toml and treats such projects as Python. Local Python packages are built into wheels, extracted, and validated via a new discoverLocalPythonPackagePolicy flow; flat/extracted module directories are handled by buildSimplePythonDiscoveredPolicy. Adjust validation and source collection to use PythonSourceDir (and fall back to the project root for requirements.txt when needed). Added tests for runtime detection and discovering a src-layout package policy, and minor doc/comment tweaks.
Make sanitizePipSpec exported as SanitizePipSpec with documentation and use it throughout the python discovery code to redact credentials in pip specs and logs. Update the build manifest generator to call discovery.SanitizePipSpec when writing PipPackage entries so private index and VCS credentials are replaced with a <redacted-credentials> placeholder. Add tests to verify credential redaction for both private index URLs and VCS specs.
…-support Support Python src-layout package policies
Krishanx92
reviewed
Apr 28, 2026
Krishanx92
approved these changes
Apr 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Fix #1732
Support more reliable builder-side handling of Python
pipPackagepolicies and align generated Python executor artifacts with the renamed SDK package. This is needed so Python policies referenced through direct/VCS specs, short URLs, and partial versions resolve deterministically and the generated runtime uses the updated SDK name consistently.Goals
Improve Python policy discovery and lockfile generation in the gateway builder.
Ensure pip-based policies are matched back to the correct manifest entry using the original pip spec.
Rename the Python SDK package from
wso2_gateway_policy_sdktoapip_sdk_coreand update generator/runtime expectations accordingly.Approach
Extended builder discovery to parse indexed and VCS pip specs, support short Git-style URLs, resolve major/minor-only refs to exact versions, and preserve the resolved exact pip spec in the generated manifest.
Updated manifest generation to prefer matching Python pip policies by the original
pipPackagevalue when multiple candidates share the same policy name.Updated the policy engine generator, SDK package layout, python-executor import resolution, and related tests to use
apip_sdk_coreinstead ofwso2_gateway_policy_sdk.Related PR: #1736