feat(api-proxy): add /reflect endpoint for dynamic provider and model discovery#2253
feat(api-proxy): add /reflect endpoint for dynamic provider and model discovery#2253
Conversation
Add GET /reflect on port 10000 (management port) that returns the list
of configured API proxy endpoints along with the models supported by
each endpoint.
- fetchJson: helper to fetch and parse JSON responses from provider APIs
- extractModelIds: normalise OpenAI/Anthropic/Copilot {data:[{id}]} and
Gemini {models:[{name}]} formats into sorted string arrays
- cachedModels / resetModelCacheState: in-memory model cache populated
at startup, with reset helper for test isolation
- fetchStartupModels: fetches model lists from all configured providers
concurrently at startup (alongside validateApiKeys)
- reflectEndpoints: builds the reflection payload with per-endpoint
configured status, base_url, port, models_url and cached models
- handleManagementEndpoint: extended to serve GET /reflect
- onListenerReady: triggers fetchStartupModels in addition to validateApiKeys
- All new functions exported and covered by unit tests (306 passing)
Agent-Logs-Url: https://github.com/github/gh-aw-firewall/sessions/36f4f63e-88fd-493f-a600-7fab58452dd6
- fetchJson: add debug-level logging for network errors and timeouts to aid operator diagnostics during model discovery - extractModelIds: extract GEMINI_MODEL_NAME_PREFIX constant and use startsWith/slice for prefix stripping (clearer than regex) - Add test for Gemini model names without the models/ prefix Agent-Logs-Url: https://github.com/github/gh-aw-firewall/sessions/36f4f63e-88fd-493f-a600-7fab58452dd6
✅ Coverage Check PassedOverall Coverage
📁 Per-file Coverage Changes (1 files)
Coverage comparison generated by |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Pull request overview
Adds an api-proxy management reflection endpoint and startup model discovery so agent harnesses can programmatically determine which providers are configured and which models are available.
Changes:
- Introduces
GET /reflecton the management port (10000) returning provider endpoint metadata plus cached model lists and amodels_fetch_completereadiness flag. - Adds startup model fetching (
fetchStartupModels) and response normalization (extractModelIds), using a newfetchJsonhelper to retrieve and parse model lists. - Extends Jest coverage for the new helper/model-fetch/reflect logic.
Show a summary per file
| File | Description |
|---|---|
| containers/api-proxy/server.js | Implements fetchJson, startup model caching, reflectEndpoints(), and wires /reflect into management endpoints + startup flow. |
| containers/api-proxy/server.test.js | Adds unit tests for fetchJson, extractModelIds, fetchStartupModels, and reflectEndpoints. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 2/2 changed files
- Comments generated: 2
| const req = mod.request(reqOpts, (res) => { | ||
| if (res.statusCode < 200 || res.statusCode >= 300) { | ||
| res.resume(); | ||
| resolveOnce(null); | ||
| return; | ||
| } | ||
| const chunks = []; | ||
| res.on('data', (chunk) => chunks.push(chunk)); | ||
| res.on('end', () => { | ||
| try { | ||
| resolveOnce(JSON.parse(Buffer.concat(chunks).toString())); | ||
| } catch { | ||
| resolveOnce(null); | ||
| } | ||
| }); | ||
| res.on('error', (err) => { | ||
| logRequest('debug', 'fetch_json_error', { url: sanitizeForLog(url), error: String(err && err.message ? err.message : err) }); | ||
| resolveOnce(null); | ||
| }); | ||
| }); |
There was a problem hiding this comment.
fetchJson() can hang indefinitely if the upstream response stream closes/aborts before emitting end or error (e.g., connection drop mid-body). Unlike httpProbe(), it doesn't handle res.close/res.aborted, so the Promise may never resolve and modelFetchComplete may never flip to true. Consider adding handlers for res.on('aborted'...) and/or res.on('close'...) that resolveOnce(null) (and optionally req.destroy()) to guarantee settlement.
| // Use COPILOT_GITHUB_TOKEN for /models if available (COPILOT_API_KEY is not accepted) | ||
| const copilotToken = copilotGithubToken || copilotAuthToken; | ||
| if (copilotToken) { | ||
| fetches.push( | ||
| fetchJson(`https://${copilotTarget}/models`, { | ||
| method: 'GET', | ||
| headers: { | ||
| 'Authorization': `Bearer ${copilotToken}`, | ||
| 'Copilot-Integration-Id': copilotIntegrationId, | ||
| }, | ||
| }, TIMEOUT_MS).then((json) => { | ||
| cachedModels.copilot = extractModelIds(json); | ||
| }) | ||
| ); | ||
| } |
There was a problem hiding this comment.
fetchStartupModels() attempts to call the Copilot /models endpoint whenever copilotAuthToken is present, but this token can be derived from COPILOT_API_KEY (BYOK). Elsewhere in this file the /models endpoint is explicitly documented as requiring COPILOT_GITHUB_TOKEN and key validation skips BYOK-only mode; for consistency and to avoid a guaranteed-unhelpful request, gate the models fetch on copilotGithubToken being set (or otherwise explicitly detect when the auth token is a GitHub token).
This comment has been minimized.
This comment has been minimized.
|
@copilot address the review feedback |
- fetchJson: add res.on('close') handler so the Promise always settles
when the upstream connection drops mid-body without emitting 'end'
or 'error', preventing modelFetchComplete from hanging indefinitely
- fetchStartupModels: gate Copilot /models fetch exclusively on
copilotGithubToken (GitHub OAuth); skip when only COPILOT_API_KEY
(BYOK) is present — consistent with validateApiKeys behaviour where
BYOK-only mode is documented as having no probe endpoint
- Tests: add cases for res.close mid-body drop and BYOK-only skip
Agent-Logs-Url: https://github.com/github/gh-aw-firewall/sessions/cd6fa904-5c17-4a7e-bcd6-0f0f44061d0e
Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Done in e9f207b. Two fixes applied:
Both changes are covered by new tests. |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| const anthropicKey = ov('anthropicKey', ANTHROPIC_API_KEY); | ||
| const anthropicTarget = ov('anthropicTarget', ANTHROPIC_API_TARGET); | ||
| const copilotGithubToken = ov('copilotGithubToken', COPILOT_GITHUB_TOKEN); | ||
| const copilotAuthToken = ov('copilotAuthToken', COPILOT_AUTH_TOKEN); |
🔥 Smoke Test: Copilot BYOK — PASS
Running in BYOK offline mode ( PR: "feat(api-proxy): add /reflect endpoint for dynamic provider and model discovery" · Author: Overall: PASS
|
|
Smoke Test Results: Status: PASS
|
|
chore: optimize test-coverage-improver workflow for ~50% token reduction Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "registry.npmjs.org"See Network Configuration for more information.
|
Chroot Version Comparison Results
Overall: ❌ Not all tests passed — Python and Node.js versions differ between host and chroot.
|
🏗️ Build Test Suite Results
Overall: 8/8 ecosystems passed — ✅ PASS
|
Smoke Test Results
Overall: FAIL — Service containers are not reachable from this environment.
|
This comment has been minimized.
This comment has been minimized.
🔬 Smoke Test Results
Overall: FAIL — pre-step outputs ( PR: feat(api-proxy): add /reflect endpoint for dynamic provider and model discovery
|
Agent harnesses have no way to programmatically discover which LLM providers are configured in the api-proxy sidecar or what models each exposes — requiring hardcoded assumptions about provider availability.
Changes
New
GET /reflectmanagement endpoint (port 10000)Returns all five proxy endpoints with their configured status, base URL, port,
models_url, and a cached model list populated at startup.{ "endpoints": [ { "provider": "openai", "port": 10000, "base_url": "http://api-proxy:10000", "configured": true, "models": ["gpt-4o", "o1", "o3-mini"], "models_url": "http://api-proxy:10000/v1/models" }, { "provider": "anthropic", "port": 10001, "base_url": "http://api-proxy:10001", "configured": false, "models": null, "models_url": "http://api-proxy:10001/v1/models" }, { "provider": "copilot", "port": 10002, "base_url": "http://api-proxy:10002", "configured": true, "models": ["claude-3.5-sonnet","gpt-4o"],"models_url": "http://api-proxy:10002/models" }, { "provider": "gemini", "port": 10003, "base_url": "http://api-proxy:10003", "configured": false, "models": null, "models_url": "http://api-proxy:10003/v1beta/models" }, { "provider": "opencode", "port": 10004, "base_url": "http://api-proxy:10004", "configured": true, "models": null, "models_url": null } ], "models_fetch_complete": true }Startup model fetching (
fetchStartupModels)validateApiKeysonce all listeners are ready/v1/models(OpenAI, Anthropic),/models(Copilot),/v1beta/models(Gemini) through Squid{data:[{id}]}and Gemini-style{models:[{name:"models/..."}]}responses viaextractModelIdscachedModels;models_fetch_completeflag signals readinessfetchJsonhelperhttpProbebut returns parsed JSON instead of status codeproxyAgent(Squid) consistent with all other upstream callsdebug-level log events on network errors and timeouts for operator diagnosticsnullon any failure — model fetch errors never block proxy startup