Commit 7ecb2b4
docs(tokens): correct the note claiming gateway max_output is unenforced (#109)
* docs(tokens): correct the note claiming gateway max_output is unenforced
The comment added in 3.35.2 said the gateway's max_output values are 'not
enforced: Franklin sends max_tokens 16384 to haiku today and the gateway
accepts it.' Accepting the request and honoring it are different things. The
gateway clamps with Math.min(request.max_tokens, model.maxOutput) on both
handlers and prices the quote off the clamped ceiling — the request is
accepted rather than rejected, so the clamp stays invisible until a reply is
long enough to hit it. The smoke test that produced that claim returned ten
tokens.
The conclusion the comment supports is unchanged: the static tables stay
authoritative, because the gateway's values are wrong for models we know.
Only the reasoning was wrong, and a wrong reason is how a right rule gets
reverted later.
* docs(tokens): make the correction itself precise
Two things the first pass got imprecise, in a comment whose entire job is
precision about this mechanism:
- It cited 8192 / 64000 in the present tense. Those exact values are being
corrected upstream (BlockRunAI/blockrun#266), so the comment was written
pre-stale. Reframed as a dated observation, with the durable point stated
separately: the catalog can be wrong, and a wrong value here is not
cosmetic.
- It called the clamp 'invisible' without qualification. The handler does
log 'capping to <limit>' — server-side, where no caller sees it
(chat/completions/route.ts:1343). Invisible from here, not silent
everywhere. Someone who finds that warn should not conclude the comment
is wrong again.
---------
Co-authored-by: 1bcMax <195689928+1bcMax@users.noreply.github.com>1 parent fff9c12 commit 7ecb2b4
1 file changed
Lines changed: 15 additions & 5 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
92 | 92 | | |
93 | 93 | | |
94 | 94 | | |
95 | | - | |
96 | | - | |
97 | | - | |
98 | | - | |
99 | | - | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
100 | 110 | | |
101 | 111 | | |
102 | 112 | | |
| |||
0 commit comments