Commit 6dfba08
feat: configurable NATS max_payload for long-ISL disagg serving
NATS default max_payload is 1MB, which causes request timeouts with
disaggregated serving at long ISL (65K+ tokens). The prompt data
exceeds 1MB when base64-encoded in the NATS message.
- Add infra.nats_max_payload_mb config option (default: None = NATS 1MB)
- When set, writes /tmp/nats.conf with the custom payload size
- Plumbed through schema -> do_sweep -> setup_head.py CLI arg
Recipe usage:
infra:
nats_max_payload_mb: 24 # for long-ISL disagg (65K+ tokens)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>1 parent 083c68d commit 6dfba08
3 files changed
Lines changed: 29 additions & 5 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
108 | 108 | | |
109 | 109 | | |
110 | 110 | | |
| 111 | + | |
| 112 | + | |
111 | 113 | | |
112 | 114 | | |
113 | 115 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
119 | 119 | | |
120 | 120 | | |
121 | 121 | | |
122 | | - | |
| 122 | + | |
123 | 123 | | |
124 | 124 | | |
125 | 125 | | |
126 | 126 | | |
| 127 | + | |
| 128 | + | |
127 | 129 | | |
128 | 130 | | |
129 | 131 | | |
| |||
132 | 134 | | |
133 | 135 | | |
134 | 136 | | |
135 | | - | |
| 137 | + | |
136 | 138 | | |
137 | 139 | | |
138 | 140 | | |
139 | 141 | | |
140 | 142 | | |
141 | | - | |
142 | | - | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
143 | 155 | | |
144 | 156 | | |
145 | 157 | | |
| |||
246 | 258 | | |
247 | 259 | | |
248 | 260 | | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
249 | 267 | | |
250 | 268 | | |
251 | 269 | | |
| |||
264 | 282 | | |
265 | 283 | | |
266 | 284 | | |
267 | | - | |
| 285 | + | |
268 | 286 | | |
269 | 287 | | |
270 | 288 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
865 | 865 | | |
866 | 866 | | |
867 | 867 | | |
| 868 | + | |
| 869 | + | |
| 870 | + | |
868 | 871 | | |
869 | 872 | | |
870 | 873 | | |
| 874 | + | |
871 | 875 | | |
872 | 876 | | |
873 | 877 | | |
| |||
0 commit comments