Skip to content

Commit cf85600

Browse files
feat(registries): user-added registries + provenance/trust model + registry add-source CLI (MCP-866) (#573)
* feat(registries): MCP-866 trust/provenance model foundation (config + merge) Foundation layer for user-added registries. CLI add-source, server-add-time stamping/enforcement, REST/MCP surface, and docs follow in subsequent commits. - Add registry provenance trust tags (official/trusted vs custom/unverified) to both config and registries RegistryEntry, with IsTrusted() helpers. - DefaultRegistries are tagged official/trusted. - SetRegistriesFromConfig recomputes provenance AUTHORITATIVELY by ID: a shipped-default ID is always official; anything user-added is custom/unverified — a user cannot self-assert trust via config. - ServerConfig gains SourceRegistryID + SourceRegistryProvenance so a server's origin is recorded for the approval/quarantine view. - Config validation rejects skip_quarantine for servers sourced from a custom/unverified registry (quarantine-always; no user allowlist). - RegistriesLocked enterprise stub knob (doc + add-source rejection only). - Tests: provenance JSON round-trip, authoritative merge recompute (incl. rejecting self-asserted trust), skip_quarantine rejection/allowance. config + registries suites green with -race; golangci-lint 0 issues. Refs MCP-866. Co-Authored-By: Paperclip <noreply@paperclip.ing> * feat(registries): add-source CLI + REST/MCP surface + quarantine enforcement (MCP-866) Builds on the provenance foundation to let users add their own MCP registry sources, always tagged custom/unverified so their servers can never escape quarantine. There is no allowlist a user can add themselves into. - `mcpproxy registry add-source <https-url> [--protocol|--id|--name]`: daemon-first CLI that adds a generic modelcontextprotocol/registry v0.1 endpoint. Writes cfg.Registries copy-on-write via UpdateConfig + persists, and rebuilds the effective catalog so the source is immediately searchable. - Server keystone (add_from_registry): stamp SourceRegistryID/Provenance onto the derived ServerConfig from the resolved registry; a custom/unverified source forces Quarantined=true and SkipQuarantine=false regardless of the global default (CN-002 extended). - New add-source op (add_registry_source.go): pure URL→entry derivation (https validation, id-from-host slug, v0.1 servers-url derivation) + guardrails (registries_locked, no shadowing a built-in id, no duplicate). Stable cross-surface error codes: invalid_registry_url / registries_locked / registry_shadows_builtin / duplicate_registry. - REST POST /api/v1/registries; cliclient.AddRegistrySource; provenance + trusted surfaced in list_registries across runtime REST + MCP so a UI can show the one-time third-party-registry warning. - Docs: docs/registries.md trust model + add-source + registries_locked stub. - OpenAPI regenerated. TDD: add-source derivation/validation unit tests, custom-origin quarantine-always keystone tests, and a registries integration test proving a user-added v0.1 endpoint is searchable AND tagged custom/unverified. Local: go build ./..., config/registries/server/httpapi/cliclient/contracts/cmd suites green (-race on the pure-logic packages), binary API + MCP e2e green, golangci-lint 0 issues, approval-hash stability canary green. Related MCP-866 Co-Authored-By: Paperclip <noreply@paperclip.ing> * fix(storage): persist registry origin/provenance on UpstreamRecord (MCP-866) CI caught the new ServerConfig fields tripping the storage field-coverage canary (TestSaveServerSyncFieldCoverage) — they were unpersisted. Persist them so a server's registry origin survives a restart; otherwise a reloaded custom-origin server would lose its provenance and the skip_quarantine guard plus the approval/quarantine view would silently stop working. - Add SourceRegistryID + SourceRegistryProvenance to UpstreamRecord. - Carry them through every config<->record conversion (async saveServerSync, Manager.SaveUpstreamServer, GetUpstreamServer, ListUpstreamServers, ListQuarantinedUpstreamServers). - Extend the field-coverage canary's expectedFields; add a save->reload round-trip test (incl. via the quarantine listing). Fixes the Unit Tests / E2E / Build Binaries CI failures on #573 (all ran go test ./... and hit the same storage canary). storage suite green with -race; go build ./... clean; gofmt clean. Related MCP-866 Co-Authored-By: Paperclip <noreply@paperclip.ing> * fix(api): surface registry provenance + trusted in GET /api/v1/registries (MCP-866) Add Provenance and Trusted fields to contracts.Registry so the REST API surfaces the trust tag for each registry. Copy from the internal registry entry in handleListRegistries. Provenance is derived authoritatively at merge time (MCP-866): - Built-in defaults get provenance=official/trusted, trusted=true - User-added registries get provenance=custom/unverified, trusted=false Regenerated swagger.yaml. Test asserts a custom registry shows provenance=custom/unverified and trusted=false. Related #573 --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
1 parent e776e3b commit cf85600

28 files changed

Lines changed: 1173 additions & 25 deletions

cmd/mcpproxy/registry_cmd.go

Lines changed: 91 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -21,12 +21,15 @@ import (
2121

2222
// Registry command flags (spec 070).
2323
var (
24-
registryConfigPath string
25-
registrySearchTag string
26-
registryLimit int
27-
registryAddName string
28-
registryAddEnv []string
29-
registryAddEnabled bool
24+
registryConfigPath string
25+
registrySearchTag string
26+
registryLimit int
27+
registryAddName string
28+
registryAddEnv []string
29+
registryAddEnabled bool
30+
registryAddSourceProto string // MCP-866
31+
registryAddSourceID string
32+
registryAddSourceName string
3033
)
3134

3235
// GetRegistryCommand builds the `registry` command group (spec 070): a single
@@ -53,12 +56,79 @@ Typical flow:
5356
mcpproxy registry add pulse weather-mcp # add it (quarantined)
5457
mcpproxy upstream approve weather-mcp # approve once you trust it
5558
56-
'registry add' talks to the running mcpproxy daemon. 'list' and 'search' use the
57-
daemon when available and otherwise read the registries directly.`,
59+
Add your own registry source (any official modelcontextprotocol/registry v0.1 endpoint):
60+
mcpproxy registry add-source https://registry.example.com # custom/unverified
61+
62+
'registry add' and 'registry add-source' talk to the running mcpproxy daemon.
63+
'list' and 'search' use the daemon when available and otherwise read the
64+
registries directly.`,
5865
}
5966

6067
cmd.PersistentFlags().StringVarP(&registryConfigPath, "config", "c", "", "Path to MCP configuration file")
61-
cmd.AddCommand(newRegistryListCmd(), newRegistrySearchCmd(), newRegistryAddCmd())
68+
cmd.AddCommand(newRegistryListCmd(), newRegistrySearchCmd(), newRegistryAddCmd(), newRegistryAddSourceCmd())
69+
return cmd
70+
}
71+
72+
func newRegistryAddSourceCmd() *cobra.Command {
73+
cmd := &cobra.Command{
74+
Use: "add-source <https-url>",
75+
Short: "Add a custom MCP registry source (quarantine-always)",
76+
Long: `Add your own MCP server registry — any https endpoint implementing the
77+
official modelcontextprotocol/registry v0.1 protocol (the same protocol shipped
78+
by Copilot/VS Code/Azure).
79+
80+
The added source is ALWAYS tagged custom/unverified: there is no allowlist you
81+
can add yourself into. Every server you discover and add through a custom source
82+
lands quarantined and can never skip quarantine — review and approve it once you
83+
trust it:
84+
mcpproxy registry search <query> -r <id>
85+
mcpproxy registry add <id> <serverId>
86+
mcpproxy upstream approve <name>`,
87+
Args: cobra.ExactArgs(1),
88+
RunE: func(_ *cobra.Command, args []string) error {
89+
sourceURL := args[0]
90+
91+
cfg, err := loadRegistryConfig()
92+
if err != nil {
93+
return outputError(clioutput.NewStructuredError(clioutput.ErrCodeConfigNotFound, err.Error()).
94+
WithRecoveryCommand("mcpproxy doctor"), clioutput.ErrCodeConfigNotFound)
95+
}
96+
97+
// add-source MUST go through the daemon: the registry list lives on the
98+
// runtime config snapshot and is updated copy-on-write via UpdateConfig.
99+
if !shouldUseUpstreamDaemon(cfg.DataDir) {
100+
return outputError(clioutput.NewStructuredError(clioutput.ErrCodeConnectionFailed,
101+
"adding a registry source requires a running mcpproxy daemon").
102+
WithGuidance("Start the daemon, then retry").
103+
WithRecoveryCommand("mcpproxy serve"), clioutput.ErrCodeConnectionFailed)
104+
}
105+
106+
ctx, cancel := registryContext()
107+
defer cancel()
108+
109+
client := cliclient.NewClient(socket.DetectSocketPath(cfg.DataDir), nil)
110+
reg, err := client.AddRegistrySource(ctx, sourceURL, registryAddSourceProto, registryAddSourceID, registryAddSourceName)
111+
if err != nil {
112+
return registryAddErrorOutput(err)
113+
}
114+
115+
outputFormat := ResolveOutputFormat()
116+
if outputFormat == "json" || outputFormat == "yaml" {
117+
formatter, _ := GetOutputFormatter()
118+
out, _ := formatter.Format(reg)
119+
fmt.Println(out)
120+
return nil
121+
}
122+
123+
fmt.Printf("✅ Added registry source '%s' (%s)\n", reg.ID, reg.Provenance)
124+
fmt.Printf("⚠️ This is a third-party, unverified registry — its servers are always quarantined until you approve them.\n")
125+
fmt.Printf(" Search it with: mcpproxy registry search <query> -r %s\n", reg.ID)
126+
return nil
127+
},
128+
}
129+
cmd.Flags().StringVar(&registryAddSourceProto, "protocol", "", "Registry protocol (default: modelcontextprotocol/registry)")
130+
cmd.Flags().StringVar(&registryAddSourceID, "id", "", "Override the derived registry id")
131+
cmd.Flags().StringVar(&registryAddSourceName, "name", "", "Override the registry display name")
62132
return cmd
63133
}
64134

@@ -270,6 +340,18 @@ func registryAddErrorOutput(err error) error {
270340
case "registry_not_found", "server_not_found":
271341
return outputError(clioutput.NewStructuredError(clioutput.ErrCodeServerNotFound, addErr.Message).
272342
WithGuidance("Check the ids with 'mcpproxy registry list' and 'mcpproxy registry search'"), clioutput.ErrCodeServerNotFound)
343+
case "invalid_registry_url":
344+
return outputError(clioutput.NewStructuredError(clioutput.ErrCodeInvalidInput, addErr.Message).
345+
WithGuidance("Provide an https URL, e.g. https://registry.example.com"), clioutput.ErrCodeInvalidInput)
346+
case "registries_locked":
347+
return outputError(clioutput.NewStructuredError(clioutput.ErrCodeOperationFailed, addErr.Message).
348+
WithGuidance("Registry additions are disabled by policy (registries_locked)"), clioutput.ErrCodeOperationFailed)
349+
case "registry_shadows_builtin":
350+
return outputError(clioutput.NewStructuredError(clioutput.ErrCodeInvalidInput, addErr.Message).
351+
WithGuidance("Choose a different --id; built-in registry ids cannot be replaced"), clioutput.ErrCodeInvalidInput)
352+
case "duplicate_registry":
353+
return outputError(clioutput.NewStructuredError(clioutput.ErrCodeOperationFailed, addErr.Message).
354+
WithGuidance("A registry with that id already exists; pass a different --id"), clioutput.ErrCodeOperationFailed)
273355
default:
274356
return outputError(clioutput.NewStructuredError(clioutput.ErrCodeOperationFailed, addErr.Message), clioutput.ErrCodeOperationFailed)
275357
}

docs/registries.md

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,61 @@ key is configured it is sent on every request to that registry as an
2323
User-configured registries in `mcp_config.json` (`registries: [...]`) are **merged**
2424
with these defaults (keyed by ID); a custom entry never drops the shipped set.
2525

26+
## Trust model & user-added registries
27+
28+
Every registry carries a **provenance** tag:
29+
30+
| Provenance | Meaning |
31+
|---|---|
32+
| `official/trusted` | A shipped, built-in default (the five above). |
33+
| `custom/unverified` | Any registry the user added at runtime, or any non-default ID in `mcp_config.json`. |
34+
35+
Trust is **derived, not asserted** — it comes solely from whether the registry ID
36+
is one of the shipped defaults. Writing `"provenance": "official/trusted"` into a
37+
custom `mcp_config.json` entry has no effect; mcpproxy recomputes provenance on
38+
every merge. **There is no allowlist a user can add themselves into.**
39+
40+
Consequences for `custom/unverified` registries:
41+
42+
- Servers discovered through them are **always quarantined** on add, regardless of
43+
the global quarantine default — and they can **never** set `skip_quarantine`
44+
(enforced in config validation *and* at server-add time). A server's origin is
45+
recorded on its config as `source_registry_id` / `source_registry_provenance`
46+
and surfaced in the approval/quarantine view.
47+
- The `list_registries` output (MCP, REST, CLI) includes `provenance` and a
48+
`trusted` boolean so a UI can show a one-time third-party-registry warning.
49+
50+
### Adding your own registry source
51+
52+
`mcpproxy registry add-source` adds any https endpoint that implements the official
53+
`modelcontextprotocol/registry` v0.1 protocol (the same protocol Copilot / VS Code /
54+
Azure ship):
55+
56+
```bash
57+
mcpproxy registry add-source https://registry.example.com
58+
mcpproxy registry add-source https://registry.example.com --id acme --name "Acme Corp"
59+
```
60+
61+
The ID is derived from the host when omitted; `--protocol` defaults to
62+
`modelcontextprotocol/registry`. The source is always tagged `custom/unverified`.
63+
This requires a running daemon — the registry list is updated copy-on-write on the
64+
runtime config snapshot and persisted to `mcp_config.json`.
65+
66+
Equivalent surfaces:
67+
68+
- **REST:** `POST /api/v1/registries` with `{ "url": "https://…", "protocol": "…", "id": "…", "name": "…" }`.
69+
- **CLI:** `mcpproxy registry add-source <https-url>`.
70+
71+
Errors share a stable code across surfaces: `invalid_registry_url` (400),
72+
`registries_locked` (403), `registry_shadows_builtin` / `duplicate_registry` (409).
73+
74+
### Enterprise: `registries_locked` (stub)
75+
76+
Setting `"registries_locked": true` in `mcp_config.json` disables runtime registry
77+
additions (`registry add-source` and the REST/MCP add-source surface return
78+
`registries_locked`). Built-in defaults are unaffected. This is a forward-looking
79+
stub for enterprise policy pinning.
80+
2681
## Official v0.1 protocol
2782

2883
The official registry returns a cursor-paginated list of wrapped entries:

internal/cliclient/client.go

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1862,3 +1862,56 @@ func (c *Client) AddFromRegistry(ctx context.Context, registryID, serverID, name
18621862
}
18631863
return &apiResp.Data.Server, nil
18641864
}
1865+
1866+
// AddRegistrySource adds a user-supplied registry source via the daemon
1867+
// (MCP-866). POST /api/v1/registries → data.registry. On failure it returns a
1868+
// *RegistryAddError carrying the stable cross-surface code.
1869+
func (c *Client) AddRegistrySource(ctx context.Context, sourceURL, protocol, id, name string) (*contracts.RegistrySummary, error) {
1870+
body := contracts.AddRegistrySourceRequest{URL: sourceURL, Protocol: protocol, ID: id, Name: name}
1871+
bodyBytes, err := json.Marshal(body)
1872+
if err != nil {
1873+
return nil, fmt.Errorf("failed to marshal request: %w", err)
1874+
}
1875+
1876+
u := c.baseURL + "/api/v1/registries"
1877+
req, err := http.NewRequestWithContext(ctx, http.MethodPost, u, bytes.NewReader(bodyBytes))
1878+
if err != nil {
1879+
return nil, fmt.Errorf("failed to create request: %w", err)
1880+
}
1881+
req.Header.Set("Content-Type", "application/json")
1882+
c.prepareRequest(ctx, req)
1883+
1884+
resp, err := c.httpClient.Do(req)
1885+
if err != nil {
1886+
return nil, fmt.Errorf("failed to call add-registry-source API: %w", err)
1887+
}
1888+
defer resp.Body.Close()
1889+
1890+
respBytes, err := io.ReadAll(resp.Body)
1891+
if err != nil {
1892+
return nil, fmt.Errorf("failed to read response: %w", err)
1893+
}
1894+
1895+
var apiResp struct {
1896+
Success bool `json:"success"`
1897+
Data *contracts.AddRegistrySourceData `json:"data"`
1898+
Error string `json:"error"`
1899+
Code string `json:"code"`
1900+
RequestID string `json:"request_id"`
1901+
}
1902+
if err := json.Unmarshal(respBytes, &apiResp); err != nil {
1903+
return nil, fmt.Errorf("failed to parse response (status %d): %s", resp.StatusCode, string(respBytes))
1904+
}
1905+
1906+
if !apiResp.Success || resp.StatusCode != http.StatusOK {
1907+
msg := apiResp.Error
1908+
if msg == "" {
1909+
msg = fmt.Sprintf("API returned status %d", resp.StatusCode)
1910+
}
1911+
return nil, &RegistryAddError{Code: apiResp.Code, Message: msg, RequestID: apiResp.RequestID}
1912+
}
1913+
if apiResp.Data == nil {
1914+
return nil, fmt.Errorf("daemon returned success with no registry data")
1915+
}
1916+
return &apiResp.Data.Registry, nil
1917+
}

internal/config/config.go

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -101,6 +101,13 @@ type Config struct {
101101
// Registries configuration for MCP server discovery
102102
Registries []RegistryEntry `json:"registries,omitempty" mapstructure:"registries"`
103103

104+
// RegistriesLocked is an enterprise stub knob (MCP-866): when true, runtime
105+
// additions of custom registries (e.g. `registry add-source`, the REST/MCP
106+
// add-source surface) are rejected so an administrator can pin the discovery
107+
// sources. Built-in defaults are unaffected. Documented but otherwise inert
108+
// beyond the add-source rejection.
109+
RegistriesLocked bool `json:"registries_locked,omitempty" mapstructure:"registries-locked"`
110+
104111
// Deprecated: Features flags are unused and have no runtime effect. Kept for backward compatibility.
105112
Features *FeatureFlags `json:"features,omitempty" mapstructure:"features"`
106113

@@ -251,6 +258,16 @@ type ServerConfig struct {
251258
LauncherWaitTimeout Duration `json:"launcher_wait_timeout,omitempty" mapstructure:"launcher_wait_timeout" swaggertype:"string"`
252259
EnabledTools []string `json:"enabled_tools,omitempty" mapstructure:"enabled_tools"` // Allowlist: only these tools are exposed; mutually exclusive with disabled_tools
253260
DisabledTools []string `json:"disabled_tools,omitempty" mapstructure:"disabled_tools"` // Denylist: these tools are hidden; mutually exclusive with enabled_tools
261+
262+
// SourceRegistryID records which registry this server was added from (empty
263+
// for manually-configured servers). MCP-866: surfaced in the approval /
264+
// quarantine view so a reviewer can see a server's origin.
265+
SourceRegistryID string `json:"source_registry_id,omitempty" mapstructure:"source_registry_id"`
266+
// SourceRegistryProvenance is the trust tag of the source registry at add
267+
// time (RegistryProvenanceOfficial / RegistryProvenanceCustom). When it is
268+
// RegistryProvenanceCustom, skip_quarantine is forbidden — a custom,
269+
// unverified registry can never opt its servers out of quarantine.
270+
SourceRegistryProvenance string `json:"source_registry_provenance,omitempty" mapstructure:"source_registry_provenance"`
254271
}
255272

256273
// OAuthConfig represents OAuth configuration for a server
@@ -632,6 +649,18 @@ func (c *OutputSanitisationConfig) WouldMutate(trust string) bool {
632649
return trust == "untrusted" && (c.IsStripEnabled() || c.IsSpotlightEnabled())
633650
}
634651

652+
// Registry provenance tags (MCP-866). Trust is derived, not user-asserted: a
653+
// registry is "trusted" only when it is one of the shipped built-in defaults.
654+
// Anything a user adds at runtime (e.g. via `registry add-source`) is
655+
// "custom/unverified" and can NEVER skip quarantine — there is no allowlist a
656+
// user can append themselves into.
657+
const (
658+
// RegistryProvenanceOfficial marks a built-in, shipped-by-default registry.
659+
RegistryProvenanceOfficial = "official/trusted"
660+
// RegistryProvenanceCustom marks a user-added registry of unknown trust.
661+
RegistryProvenanceCustom = "custom/unverified"
662+
)
663+
635664
// RegistryEntry represents a registry in the configuration
636665
type RegistryEntry struct {
637666
ID string `json:"id"`
@@ -646,6 +675,19 @@ type RegistryEntry struct {
646675
// true and no key is configured, the registry is skipped/marked unavailable
647676
// rather than failing the whole search (FR-008).
648677
RequiresKey bool `json:"requires_key,omitempty"`
678+
// Provenance is the trust tag for this registry (MCP-866):
679+
// RegistryProvenanceOfficial for built-in defaults, RegistryProvenanceCustom
680+
// for user-added registries. It is authoritatively (re)computed by the
681+
// registries merge from whether the ID is a shipped default — a user cannot
682+
// claim "official/trusted" by writing it into their config.
683+
Provenance string `json:"provenance,omitempty" mapstructure:"provenance"`
684+
}
685+
686+
// IsTrusted reports whether the registry is an official, shipped-by-default
687+
// source. Trust is never granted by omission — an absent provenance tag is
688+
// untrusted.
689+
func (r *RegistryEntry) IsTrusted() bool {
690+
return r != nil && r.Provenance == RegistryProvenanceOfficial
649691
}
650692

651693
// CursorMCPConfig represents the structure for Cursor IDE MCP configuration
@@ -847,6 +889,7 @@ func DefaultRegistries() []RegistryEntry {
847889
ServersURL: "https://registry.modelcontextprotocol.io/v0.1/servers",
848890
Tags: []string{"verified", "official"},
849891
Protocol: "modelcontextprotocol/registry",
892+
Provenance: RegistryProvenanceOfficial,
850893
},
851894
{
852895
ID: "reference",
@@ -856,6 +899,7 @@ func DefaultRegistries() []RegistryEntry {
856899
ServersURL: "builtin://reference",
857900
Tags: []string{"verified", "official", "reference"},
858901
Protocol: "builtin/reference",
902+
Provenance: RegistryProvenanceOfficial,
859903
},
860904
{
861905
ID: "docker-mcp-catalog",
@@ -865,6 +909,7 @@ func DefaultRegistries() []RegistryEntry {
865909
ServersURL: "https://hub.docker.com/v2/repositories/mcp/",
866910
Tags: []string{"verified"},
867911
Protocol: "custom/docker",
912+
Provenance: RegistryProvenanceOfficial,
868913
},
869914
{
870915
ID: "pulse",
@@ -875,6 +920,7 @@ func DefaultRegistries() []RegistryEntry {
875920
Tags: []string{"verified"},
876921
Protocol: "custom/pulse",
877922
RequiresKey: true,
923+
Provenance: RegistryProvenanceOfficial,
878924
},
879925
{
880926
ID: "smithery",
@@ -885,6 +931,7 @@ func DefaultRegistries() []RegistryEntry {
885931
Tags: []string{"verified"},
886932
Protocol: "modelcontextprotocol/registry",
887933
RequiresKey: true,
934+
Provenance: RegistryProvenanceOfficial,
888935
},
889936
}
890937
}
@@ -1247,6 +1294,16 @@ func (c *Config) ValidateDetailed() []ValidationError {
12471294
Message: "enabled_tools and disabled_tools are mutually exclusive; use one or the other",
12481295
})
12491296
}
1297+
1298+
// MCP-866: a server sourced from a custom/unverified registry can NEVER
1299+
// skip quarantine. There is no allowlist a user can add themselves into,
1300+
// so an unverified third-party source must always be reviewed.
1301+
if server.SkipQuarantine && server.SourceRegistryProvenance == RegistryProvenanceCustom {
1302+
errors = append(errors, ValidationError{
1303+
Field: fieldPrefix + ".skip_quarantine",
1304+
Message: "skip_quarantine is not allowed for a server added from a custom/unverified registry",
1305+
})
1306+
}
12501307
}
12511308

12521309
// Validate DataDir exists (if specified and not empty).

0 commit comments

Comments
 (0)