Remove config templating in favour of explicit secret references#1657
Remove config templating in favour of explicit secret references#1657
Conversation
nscuro
force-pushed
the
secret-refs
branch
3 times, most recently
from
ced415c to
1ed6173
Compare
Coverage summary from CodacySee diff coverage on Codacy
Coverage variation details
Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: Diff coverage details
Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: See your quality gate settings Change summary preferences |
There was a problem hiding this comment.
Pull request overview
This PR removes configuration templating in favor of explicit secret references, addressing usability concerns with expression-based secret handling. The new approach uses the x-secret-ref JSON schema annotation to mark properties as secret references, which are validated at configuration update time and transparently resolved at runtime.
Changes:
- Replaced expression/templating-based secret references with explicit
x-secret-refJSON schema annotations - Added pagination and filtering support to secret metadata listing operations
- Changed secret listing permission from
SECRET_MANAGEMENT_READtoSYSTEM_CONFIGURATION_READ
Reviewed changes
Copilot reviewed 48 out of 49 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| secret-management/** | Added pagination support with ListSecretsRequest and getSecretMetadata method to SecretManager interface |
| plugin/runtime/** | Added resolveSecretRefs method to resolve x-secret-ref annotations and validate secret existence |
| apiserver/** | Removed ConfigTemplateRenderer and related templating classes; updated PluginManager to use direct secret resolution |
| api/** | Updated OpenAPI specs to add getSecretMetadata endpoint, pagination support, and changed permissions |
| migration/** | Added migration to delete the SECRET_MANAGEMENT_READ permission |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
nscuro
force-pushed
the
secret-refs
branch
2 times, most recently
from
23d8621 to
3a704ce
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 50 out of 51 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
After extensive testing and some initial user feedback, it became clear that expressions / templating provides horrible UX, because: * It's not clear where expressions can be used. * Expressions are only evaluated at runtime, so users don't get feedback when they reference secrets that don't exist. * Non-technical users struggle with expression syntax. Additionally, we don't really *need* the flexibility expressions give us. We just need a way to safely reference managed secrets. This change introduces support for the `x-secret-ref` JSON schema annotation. Properties annotated with it will be treated as secret references. When extensions retrieve their runtime config, secret references are transparently resolved. When extension configs are updated via REST API, it's validated that all referenced secrets exist. Secret managers now support pagination and filtering for the `listSecrets` operation. This is used to deliver a convenient dropdown with search-as-you-type in the UI. Note that it is expected that not all providers can support pagination natively, in which case they'll need to emulate the desired behaviour, which is what the `env` provider does. Listing secret metadata no longer requires the `SECRET_MANAGEMENT_READ` permission, but the `SYSTEM_CONFIGURATION_READ` permission. This is because users who maintain configuration are actually the ones that need to know which secrets they can use. Signed-off-by: nscuro <nscuro@protonmail.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 50 out of 51 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Description
Removes config templating in favour of explicit secret references.
Addressed Issue
N/A
Additional Details
Frontend PR: DependencyTrack/hyades-frontend#388
Docs PR: DependencyTrack/hyades#1992
After extensive testing and some initial user feedback, it became clear that expressions / templating provides horrible UX, because:
Additionally, we don't really need the flexibility expressions give us. We just need a way to safely reference managed secrets.
This change introduces support for the
x-secret-refJSON schema annotation. Properties annotated with it will be treated as secret references.When extensions retrieve their runtime config, secret references are transparently resolved.
When extension configs are updated via REST API, it's validated that all referenced secrets exist.
Secret managers now support pagination and filtering for the
listSecretsoperation. This is used to deliver a convenient dropdown with search-as-you-type in the UI. Note that it is expected that not all providers can support pagination natively, in which case they'll need to emulate the desired behaviour, which is what theenvprovider does.Listing secret metadata no longer requires the
SECRET_MANAGEMENT_READpermission, but theSYSTEM_CONFIGURATION_READpermission. This is because users who maintain configuration are actually the ones that need to know which secrets they can use.Checklist
This PR fixes a defect, and I have provided tests to verify that the fix is effectiveThis PR introduces changes to the database model, and I have updated the migration changelog accordingly