feat: marketplace catalog service - #395
Merged
Merged
Conversation
ChielTimmermans
force-pushed
the
split/3-marketplace-catalog-service
branch
from
August 25, 2026 13:51
2fb24cb to
1250025
Compare
ChielTimmermans
changed the title
ChielTimmermans
force-pushed
the
split/3-marketplace-catalog-service
branch
5 times, most recently
from
August 27, 2026 13:25
45500b2 to
9de068d
Compare
ChielTimmermans
marked this pull request as ready for review
August 27, 2026 20:27
ChielTimmermans
force-pushed
the
split/3-marketplace-catalog-service
branch
from
August 28, 2026 14:43
9de068d to
136373b
Compare
PR Environment - Torn DownThe environment for this PR has been removed. |
ChielTimmermans
force-pushed
the
split/3-marketplace-catalog-service
branch
2 times, most recently
from
August 28, 2026 15:24
974f821 to
afcfc7a
Compare
Implements all five catalog.v1.CatalogService RPCs against the schema from stack 2/3, and swaps the health-only main.go from stack 1/3 for the real one: database pool, schema version assertion and the Connect handler. The service is anonymous and read-only. It authenticates nothing and holds no JWT secret; every access decision is an RLS policy evaluated under the fun_marketplace_catalog_api role, so the tests exercise the real role rather than a superuser. PluginVersionListByPluginID and PluginLatestPublishedDefinition join through appstore.plugins so the plugin's own policy gates version history and manifest bytes. Without that join a restricted or soft-deleted listing keeps serving its versions, publish dates, definition hashes, release notes and manifest to anyone holding the id. The check cannot live in plugin_definitions' policy instead: plugins' policy already references plugin_definitions, and mutual reference makes Postgres reject every query against either table with SQLSTATE 42P17. Any future query that reads plugin_definitions without going through plugins reopens this; rls_test.go pins the behaviour. The ILIKE search is deliberate rather than full-text: the catalog is small and unpaginated, and FTS would cost a tsvector column plus a trigger. Search terms are escaped before binding so % and _ match literally rather than as wildcards. PluginSort carries no FEATURED member: appstore.plugins has no featured column, so the ordering it named could not be expressed and UNSPECIFIED now sorts as NAME. ListPluginsRequest.sort reclaims field 30 from the removed featured_only rather than reserving it; catalog.v1 has no deployed consumer.
ChielTimmermans
force-pushed
the
split/3-marketplace-catalog-service
branch
from
August 28, 2026 15:27
afcfc7a to
2141908
Compare
jos-
previously approved these changes
Aug 31, 2026
jos-
left a comment
jos-
left a comment
Contributor
There was a problem hiding this comment.
LGTM, but 3 small comments from Claude, see below.
- /readyz wrote the raw pgx Ping error into the response body. The probe is on the outer mux, which the Ingress routes publicly, and this service has no JWT layer in front of it, so host, port, database and role were readable by any anonymous caller while the database was down. The body is now a fixed "database unavailable" and the detail goes to the log. The handler moved into readyz() so a test can pin it. - accessLabel treated the RBAC verbs that grant privileges without touching the object as read-only. bind, escalate, impersonate, approve, sign and use now read as "Read and write", so a plugin that can escalate a ClusterRole is not shown to the storefront as read-only. - GetPlugin returned Internal when a manifest failed to parse, losing the whole detail page over one field from a newer plugin-sdk. It now logs and serves the column-backed listing with capabilities and permissions empty. - latest_version_id was ordered by published DESC in two identical subqueries; only PluginList got the id DESC tiebreaker, so on a tie the grid card and the detail page could name different versions as latest. PluginGetByID matches. - Pins the soft-deleted half of the manifest guard. plugin_definitions' policy checks only its own deleted and published, so a soft-deleted plugin's rows stay readable at the table level and only the JOIN through appstore.plugins holds the manifest back; that was tested for RESTRICTED listings but not deleted ones. - Migrates the h2c handler to net/http's Server.Protocols and drops the x/net/http2 imports. The Upgrade: h2c handshake goes away with it, which nothing here uses — the ingress speaks prior-knowledge h2c. - Marks the four organization.v1.PluginService RPCs the catalog replaces as deprecated: ListPlugins, GetPluginDetail, ListPluginDefinitions and GetPluginDefinition. GetPluginDefinition is in isPublicEndpoint and filters neither visibility nor published, so it serves draft and RESTRICTED manifests anonymously; its caller plugin-controller has no user JWT, so the move needs a credentialed path. - Enables marketplaceCatalogApi in values-local.yaml and copies plugin-sdk/pluginruntime in both Dockerfiles, without which the image does not build.
ChielTimmermans
force-pushed
the
split/3-marketplace-catalog-service
branch
3 times, most recently
from
September 1, 2026 06:44
fea7515 to
4f54f53
Compare
catalog.v1 exposed definition_hash but no way to fetch the bytes it hashes, on the premise that manifests were organization-api's business. That premise died with the previous commit: organization.v1's GetPluginDefinition is anonymous, filters neither visibility nor published, and is now deprecated — so the bytes needed a home. GetPluginDefinition returns the stored manifest verbatim plus its hash, and takes either key: plugin_id, which the storefront holds having just listed the plugin, or the (organization_name, plugin_name) pair a PluginInstallation carries. plugin-controller reads the CR, never ListPlugins, so without the name form it could not call this at all. Both lookups join through appstore.plugins, and the name one through tenant.organizations as well, so a RESTRICTED or taken-down listing is unreachable either way; unknown version, unknown publisher and hidden listing all answer NotFound, as GetPlugin already does. It serves unpublished versions deliberately. plugin-controller installs definitions that were never published — PutPluginDefinition has never set published — and it holds no user JWT, so migration 036 relaxes the three catalog policies: plugin_definitions_select_catalog to `deleted IS NULL`, plugins_select_catalog to `deleted IS NULL AND visibility = 'public'`, and organizations_select_catalog to drop its published requirement, without which a publisher owning only drafts could not be resolved by name. The consequence is that the storefront's own rules leave RLS and become explicit in the queries: an EXISTS filter in PluginList, PluginGetByID and PublisherList, and published IS NOT NULL in PluginVersionListByPluginID and both latest_version_id subqueries. A bug in one of those six now shows an unpublished listing, where the policies used to catch it regardless of the query. Each carries a comment saying why it is there. One note for whoever picks this up: plugins_select_catalog no longer references plugin_definitions, so the mutual-reference 42P17 that forced the join-through- plugins pattern no longer applies. plugin-controller can move onto this by swapping defclient's client and URL. A listing its organization owns but has not made public stays unreachable — the catalog is anonymous and cannot tell who is asking.
ChielTimmermans
force-pushed
the
split/3-marketplace-catalog-service
branch
from
September 1, 2026 07:26
4f54f53 to
9cb0265
Compare
jos-
approved these changes
Sep 1, 2026
ChielTimmermans
added a commit
that referenced
this pull request
Sep 1, 2026
Moves every reader of organization.v1.PluginService onto catalog.v1, which is what #395 deprecated them for. organization-api keeps serving the RPCs; nothing is deleted here, so the two can be deployed in either order. plugin-controller fetches definitions from marketplace-catalog-api by (organization, plugin) name — the spelling a PluginInstallation carries. ORGANIZATION_API_URL becomes MARKETPLACE_CATALOG_API_URL, defaulting to externalUrls.marketplace. That rename reaches two deployments: the chart's controller, and the shoot-side one cluster-worker provisions, whose PLUGIN_ORGANIZATION_API_URL becomes PLUGIN_MARKETPLACE_CATALOG_API_URL — a shoot controller would otherwise come up without the URL it now requires. The local sandbox relay (`just plugins sandbox-catalog`) bridges the catalog's NodePort instead of organization-api's. FUN-19's cross-cluster constraint is unchanged: the controller runs outside the management cluster and needs an externally-routable URL. The console reads the storefront from catalog.v1 over its own unauthenticated transport: no cookie, no Fun-Organization header. catalog.v1 returns ids where organization.v1 returned nested names, so the plugins page resolves publisher and category names once through ListPublishers and ListCategories rather than per plugin, and the detail page maps the response into a view model so its template is unchanged apart from tags, which are plain labels now. The install modal's version picker reads ListPluginVersions, which returns only published versions — organization.v1's ListPluginDefinitions returned drafts too, so a draft is no longer offered as something to install. ListPresets has no catalog equivalent and stays on organization.v1; the demo transport answers that and the catalog, and no longer stubs the browse RPCs the console stopped calling. Adds externalUrls.marketplace per environment, the console's marketplaceApiUrl config key, and that origin to CONNECT_SRC — without which the browser's CSP blocks every catalog call.
ChielTimmermans
added a commit
that referenced
this pull request
Sep 1, 2026
Moves every reader of organization.v1.PluginService onto catalog.v1, which is what #395 deprecated them for. organization-api keeps serving the RPCs; nothing is deleted here, so the two can be deployed in either order. plugin-controller fetches definitions from marketplace-catalog-api by (organization, plugin) name — the spelling a PluginInstallation carries. ORGANIZATION_API_URL becomes MARKETPLACE_CATALOG_API_URL, defaulting to externalUrls.marketplace. That rename reaches two deployments: the chart's controller, and the shoot-side one cluster-worker provisions, whose PLUGIN_ORGANIZATION_API_URL becomes PLUGIN_MARKETPLACE_CATALOG_API_URL — a shoot controller would otherwise come up without the URL it now requires. The local sandbox relay (`just plugins sandbox-catalog`) bridges the catalog's NodePort instead of organization-api's. FUN-19's cross-cluster constraint is unchanged: the controller runs outside the management cluster and needs an externally-routable URL. The console reads the storefront from catalog.v1 over its own unauthenticated transport: no cookie, no Fun-Organization header. catalog.v1 returns ids where organization.v1 returned nested names, so the plugins page resolves publisher and category names once through ListPublishers and ListCategories rather than per plugin, and the detail page maps the response into a view model so its template is unchanged apart from tags, which are plain labels now. The install modal's version picker reads ListPluginVersions, which returns only published versions — organization.v1's ListPluginDefinitions returned drafts too, so a draft is no longer offered as something to install. ListPresets has no catalog equivalent and stays on organization.v1; the demo transport answers that and the catalog, and no longer stubs the browse RPCs the console stopped calling. Adds externalUrls.marketplace per environment, the console's marketplaceApiUrl config key, and that origin to CONNECT_SRC — without which the browser's CSP blocks every catalog call.
ChielTimmermans
added a commit
that referenced
this pull request
Sep 1, 2026
Moves every reader of organization.v1.PluginService onto catalog.v1, which is what #395 deprecated them for. organization-api keeps serving the RPCs; nothing is deleted here, so the two can be deployed in either order. plugin-controller fetches definitions from marketplace-catalog-api by (organization, plugin) name — the spelling a PluginInstallation carries. ORGANIZATION_API_URL becomes MARKETPLACE_CATALOG_API_URL, defaulting to externalUrls.marketplace. That rename reaches two deployments: the chart's controller, and the shoot-side one cluster-worker provisions, whose PLUGIN_ORGANIZATION_API_URL becomes PLUGIN_MARKETPLACE_CATALOG_API_URL — a shoot controller would otherwise come up without the URL it now requires. The local sandbox relay (`just plugins sandbox-catalog`) bridges the catalog's NodePort instead of organization-api's. FUN-19's cross-cluster constraint is unchanged: the controller runs outside the management cluster and needs an externally-routable URL. The console reads the storefront from catalog.v1 over its own unauthenticated transport: no cookie, no Fun-Organization header. catalog.v1 returns ids where organization.v1 returned nested names, so the plugins page resolves publisher and category names once through ListPublishers and ListCategories rather than per plugin, and the detail page maps the response into a view model so its template is unchanged apart from tags, which are plain labels now. The install modal's version picker reads ListPluginVersions, which returns only published versions — organization.v1's ListPluginDefinitions returned drafts too, so a draft is no longer offered as something to install. ListPresets has no catalog equivalent and stays on organization.v1; the demo transport answers that and the catalog, and no longer stubs the browse RPCs the console stopped calling. Adds externalUrls.marketplace per environment, the console's marketplaceApiUrl config key, and that origin to CONNECT_SRC — without which the browser's CSP blocks every catalog call.
ChielTimmermans
added a commit
that referenced
this pull request
Sep 1, 2026
Moves the storefront readers of organization.v1.PluginService onto catalog.v1, which is what #395 deprecated them for. organization-api keeps serving the RPCs; nothing is deleted here, so the two can be deployed in either order. plugin-controller fetches definitions from marketplace-catalog-api by (organization, plugin) name — the spelling a PluginInstallation carries. ORGANIZATION_API_URL becomes MARKETPLACE_CATALOG_API_URL, defaulting to the in-cluster marketplace-catalog-api Service as the old value did. That rename reaches two deployments: the chart's controller, and the shoot-side one cluster-worker provisions, whose PLUGIN_ORGANIZATION_API_URL becomes PLUGIN_MARKETPLACE_CATALOG_API_URL — a shoot controller would otherwise come up without the URL it now requires. Setting the old pluginController.organizationApiUrl key now fails the render rather than silently falling back to the default. The local sandbox relay (`just plugins sandbox-catalog`) bridges the catalog's NodePort instead of organization-api's. The console's plugins page and plugin detail page read catalog.v1 over their own unauthenticated transport: no cookie, no Fun-Organization header. catalog.v1 returns ids where organization.v1 returned nested names, so publisher and category names are resolved once through ListPublishers and ListCategories rather than per plugin, and the detail page maps the response into a view model so its template is unchanged apart from tags, which are plain labels now. The install modal's version picker reads ListPluginVersions, which returns only published versions — organization.v1's ListPluginDefinitions returned drafts too, so a draft is no longer offered as something to install. Still on organization.v1, deliberately: ListPresets, which has no catalog equivalent, and the three cluster-side views (cluster-plugins, cluster-details, shared-plugins-form) that list plugins to name what is installed — the catalog only returns PUBLIC listings, so moving them would hide an organization's own private plugins from its clusters. Adds externalUrls.marketplace per environment, the console's marketplaceApiUrl config key, and that origin to CONNECT_SRC — without which the browser's CSP blocks every catalog call. Regenerates db/fundament.sql from the model: #395 relaxed the three catalog policies in fundament.dbm and in migration 036 but left the schema of record carrying the old published-only predicates, which the next generated migration would have reverted.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.