Skip to content

fix(agent-server): qualify OpenHands verified models#3006

Closed
juanmichelini wants to merge 1 commit into
mainfrom
openhands/fix-verified-models-3005
Closed

fix(agent-server): qualify OpenHands verified models#3006
juanmichelini wants to merge 1 commit into
mainfrom
openhands/fix-verified-models-3005

Conversation

@juanmichelini

@juanmichelini juanmichelini commented Apr 29, 2026

Copy link
Copy Markdown
Collaborator
  • A human has tested these changes.

Why

The verified-models API returned bare model IDs for the OpenHands provider. Because the OpenHands provider proxies upstream model IDs, downstream consumers could treat provider-verified IDs like gpt-5.4 or claude-opus-4-5 as OpenHands-verified too.

Summary

  • qualify /api/llm/models/verified entries for the OpenHands provider as openhands/<model>
  • add a code comment explaining why OpenHands models must stay provider-qualified in the API response
  • add regression coverage for both the direct router function and the HTTP endpoint

Issue Number

Closes #3005

How to Test

  1. Run pre-commit on the touched files:
    uv run pre-commit run --files AGENTS.md openhands-agent-server/openhands/agent_server/llm_router.py tests/agent_server/test_llm_router.py
  2. Run the targeted regression tests:
    OPENHANDS_SUPPRESS_BANNER=1 uv run pytest tests/agent_server/test_llm_router.py
  3. Verify the before/after behavior with this command:
    OPENHANDS_SUPPRESS_BANNER=1 uv run python - <<'PY'
    import asyncio
    from openhands.agent_server.llm_router import list_verified_models
    from openhands.sdk.llm.utils.verified_models import VERIFIED_MODELS
    
    print('before_fix_openhands_sample=', VERIFIED_MODELS['openhands'][:5])
    response = asyncio.run(list_verified_models())
    print('after_fix_openhands_sample=', response.models['openhands'][:5])
    print('shared_provider_sample_still_raw=', response.models['openai'][:3])
    PY

Observed output:

before_fix_openhands_sample= ['claude-opus-4-5', 'claude-opus-4-5-20251101', 'claude-opus-4-6', 'claude-opus-4-7', 'claude-sonnet-4-5']
after_fix_openhands_sample= ['openhands/claude-opus-4-5', 'openhands/claude-opus-4-5-20251101', 'openhands/claude-opus-4-6', 'openhands/claude-opus-4-7', 'openhands/claude-sonnet-4-5']
shared_provider_sample_still_raw= ['gpt-5.4', 'gpt-5.2', 'gpt-5.2-codex']

Video/Screenshots

N/A — API-only change. Command output is included above.

Type

  • Bug fix
  • Feature
  • Refactor
  • Breaking change
  • Docs / chore

Notes

  • This PR was created by an AI agent (OpenHands) on behalf of the user.

@juanmichelini can click here to continue refining the PR


Agent Server images for this PR

GHCR package: https://github.com/OpenHands/agent-sdk/pkgs/container/agent-server

Variants & Base Images

Variant Architectures Base Image Docs / Tags
java amd64, arm64 eclipse-temurin:17-jdk Link
python amd64, arm64 nikolaik/python-nodejs:python3.13-nodejs22-slim Link
golang amd64, arm64 golang:1.21-bookworm Link

Pull (multi-arch manifest)

# Each variant is a multi-arch manifest supporting both amd64 and arm64
docker pull ghcr.io/openhands/agent-server:b7e419a-python

Run

docker run -it --rm \
  -p 8000:8000 \
  --name agent-server-b7e419a-python \
  ghcr.io/openhands/agent-server:b7e419a-python

All tags pushed for this build

ghcr.io/openhands/agent-server:b7e419a-golang-amd64
ghcr.io/openhands/agent-server:b7e419a-golang_tag_1.21-bookworm-amd64
ghcr.io/openhands/agent-server:b7e419a-golang-arm64
ghcr.io/openhands/agent-server:b7e419a-golang_tag_1.21-bookworm-arm64
ghcr.io/openhands/agent-server:b7e419a-java-amd64
ghcr.io/openhands/agent-server:b7e419a-eclipse-temurin_tag_17-jdk-amd64
ghcr.io/openhands/agent-server:b7e419a-java-arm64
ghcr.io/openhands/agent-server:b7e419a-eclipse-temurin_tag_17-jdk-arm64
ghcr.io/openhands/agent-server:b7e419a-python-amd64
ghcr.io/openhands/agent-server:b7e419a-nikolaik_s_python-nodejs_tag_python3.13-nodejs22-slim-amd64
ghcr.io/openhands/agent-server:b7e419a-python-arm64
ghcr.io/openhands/agent-server:b7e419a-nikolaik_s_python-nodejs_tag_python3.13-nodejs22-slim-arm64
ghcr.io/openhands/agent-server:b7e419a-golang
ghcr.io/openhands/agent-server:b7e419a-java
ghcr.io/openhands/agent-server:b7e419a-python

About Multi-Architecture Support

  • Each variant tag (e.g., b7e419a-python) is a multi-arch manifest supporting both amd64 and arm64
  • Docker automatically pulls the correct architecture for your platform
  • Individual architecture tags (e.g., b7e419a-python-amd64) are also available if needed

Fixes #3005.

Co-authored-by: openhands <openhands@all-hands.dev>
@github-actions

Copy link
Copy Markdown
Contributor

Python API breakage checks — ✅ PASSED

Result:PASSED

Action log

@github-actions

Copy link
Copy Markdown
Contributor

REST API breakage checks (OpenAPI) — ✅ PASSED

Result:PASSED

Action log

@github-actions

Copy link
Copy Markdown
Contributor

Coverage

Coverage Report •
FileStmtsMissCoverMissing
TOTAL243091108154% 
report-only-changed-files is enabled. No files were changed during this commit :)

@juanmichelini juanmichelini deleted the openhands/fix-verified-models-3005 branch May 5, 2026 02:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

All verified models are also OpenHands verified models

2 participants