Fix gpt-5-family labeling, refresh OpenAI model list, add Claude chat models - #155
Conversation
The OpenAI chat provider routed summarize() through outlines, whose OpenAI wrapper always sends max_tokens (and temperature) — the gpt-5 family rejects max_tokens (400: use max_completion_tokens), so gpt-5-mini and friends failed at label time. summarize() now calls chat.completions directly with no token-cap or sampling params, and chat() uses a sync client (it previously called an unawaited AsyncOpenAI coroutine). tiktoken falls back to the gpt-4o encoding for model names it doesn't know. Registry refresh: add gpt-5.5, gpt-5.4, gpt-5.4-mini, gpt-5.4-nano; drop legacy gpt-4 and gpt-4-turbo. All 13 OpenAI entries verified live. New AnthropicChatProvider (Messages API, lazy import, system-message hoisting, refusal handling, SDK credential fallback when no ANTHROPIC_API_KEY in .env) with claude-haiku-4-5, claude-sonnet-5, and claude-opus-4-8 registry entries; ANTHROPIC_API_KEY joins the supported keys so it appears in Settings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 88b2f3cba2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| # no local tokenizer for Claude models; label_clusters handles | ||
| # encoder=None by skipping token-based sample truncation | ||
| self.encoder = None |
There was a problem hiding this comment.
When a user labels clusters with one of the new Anthropic models and the selected samples contain long text (or samples=0), the configured --max_tokens_per_sample and --max_tokens_total limits are silently ignored: label_clusters.py only truncates in the if enc is not None branch and otherwise sends the full selected rows. Setting self.encoder = None here therefore lets Claude requests exceed the intended token budget, which can produce context-window 400s or unexpectedly large requests instead of honoring the Setup/CLI limits.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in 408278c — the provider now supplies the approximate gpt-4o tiktoken encoding (the same fallback the OpenAI provider uses for custom endpoints), so --max_tokens_per_sample/--max_tokens_total truncation applies to Claude requests too. Also live-verified all three Claude registry models end-to-end with the real key.
…155 review) encoder=None silently disabled --max_tokens_per_sample / --max_tokens_total truncation in label_clusters. Claude has no local tokenizer, so reuse the gpt-4o tiktoken encoding as an approximation — the same fallback the OpenAI provider uses for custom endpoints. Verified all three Claude registry models live end-to-end. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Integrates #151's retry boundary with the outlines-free OpenAI chat provider and the new Anthropic provider; re-locked uv.lock with both the anthropic dep and model2vec[distill]. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 participant
Why gpt-5-mini didn't work
OpenAIChatProvider.summarize()went through outlines' OpenAI wrapper, which always sendsmax_tokens— the gpt-5 family rejects that parameter (400: they only acceptmax_completion_tokens), so every gpt-5* model failed at label time. Reproduced live, then fixed by callingchat.completions.createdirectly with no token-cap or sampling params (the gpt-5 family also rejects non-defaulttemperature). Two adjacent fixes:chat()previously called an unawaitedAsyncOpenAIcoroutine (it now uses a sync client), andtiktoken.encoding_for_modelfalls back to the gpt-4o encoding for model names newer than the installed tiktoken.Registry refresh (verified against the live API)
gpt-5.5,gpt-5.4,gpt-5.4-mini,gpt-5.4-nanogpt-4andgpt-4-turbosummarize()calls — including the previously-brokengpt-5-minitext-embedding-3-small/largeare still current, no changes neededNew: Anthropic provider
AnthropicChatProvider(Messages API, lazy SDK import per repo convention) with registry entries forclaude-haiku-4-5,claude-sonnet-5, andclaude-opus-4-8. Note there is no "Haiku 5" — Haiku 4.5 is the newest Haiku tier. Details:systemparam;stop_reason: "refusal"returns an empty label with a warning instead of crashingencoder = None—label_clusters.pyalready handles this by skipping token-based sample truncationANTHROPIC_API_KEYis set in.env, the client falls back to the SDK's own credential resolution (env vars /ant auth loginprofile)ANTHROPIC_API_KEYadded to_SUPPORTED_API_KEYS, so it shows up in the Settings page automaticallyanthropicadded as a dependency (uv.lockre-locked withUV_EXTRA_INDEX_URL=https://pypi.nvidia.com uv lock --index-strategy unsafe-best-match, same as Support Model2Vec static embedding models via the existing HF path #153; diff adds only anthropic 0.116.0)Test plan
tests/test_chat_providers.py: 12 new tests with fake clients — asserts the OpenAI provider sends nomax_tokens/max_completion_tokens/temperature/top_p, registry resolution for all entries, Anthropic system-hoisting/refusal/encoder behavioruv run pytest tests/ -q— 231 passed, 2 skippeduv run ruff check latentscope/— cleanANTHROPIC_API_KEYin Settings.🤖 Generated with Claude Code