Commit b78b331
fix(export): give the fusion probe no cache, since Cache.reset() keeps its length
The probe replayed the captured kwargs after calling Cache.reset() on the
past_key_values they carry. That call does not clear anything: transformers'
DynamicLayer.reset() zeroes the key/value tensors "while preserving the
objects", so get_seq_length() stays at the captured length and the next update()
appends to it. The probe therefore attended over twice the keys the captured
attention mask was built for.
Invisible in the tests because a tiny unpadded batch reaches the layer with
attention_mask=None, where the extra keys only change values the probe discards.
A padded batch materializes a 4D mask and the forward raises "The size of tensor
a (64) must match the size of tensor b (32) at non-singleton dimension 3".
A throwaway forward run only to fire hooks has no use for a cache at all, so
pass none. The identical reset() call in the calibration replay predates this
branch (#1223) and is PR #2248's to fix.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Fridah-nv <201670829+Fridah-nv@users.noreply.github.com>1 parent 589be3a commit b78b331
1 file changed
Lines changed: 6 additions & 8 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
290 | 290 | | |
291 | 291 | | |
292 | 292 | | |
293 | | - | |
294 | | - | |
295 | | - | |
296 | | - | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
297 | 298 | | |
298 | | - | |
299 | | - | |
300 | | - | |
301 | | - | |
| 299 | + | |
302 | 300 | | |
303 | 301 | | |
304 | 302 | | |
| |||
0 commit comments