Skip to content

Commit 86b63d7

Browse files
authored
Reduce DeepSeek long-seq decoder override to 12288 (#37834)
## Background The Galaxy long-seq decoder-block job in [run #21973954244/job/63481804018](https://github.com/tenstorrent/tt-metal/actions/runs/21973954244/job/63481804018) was OOM-killed on host at `DEEPSEEK_MAX_SEQ_LEN_OVERRIDE=16384`. ## Change This PR reduces the decoder-block override only: - `DEEPSEEK_MAX_SEQ_LEN_OVERRIDE=12288` for `models/demos/deepseek_v3/tests/test_decoder_block.py` The `test_mla.py` invocation remains at `16384`, and `test_model.py` stays at `1024`. ## Why `12288` lowers peak memory for the long sequence decoder-block path while preserving the extended long-seq coverage in CI. It targets the specific failure mode seen in the run above without weakening other test matrix coverage. ## Verification A rerun was kicked off on this branch: - [![Galaxy DeepSeek long-seq](https://github.com/tenstorrent/tt-metal/actions/workflows/galaxy-deepseek-tests.yaml/badge.svg?branch=chore/reduce-deepseek-decoder-seq-len-12288&event=workflow_dispatch)](https://github.com/tenstorrent/tt-metal/actions/runs/21983835196) - Run URL: https://github.com/tenstorrent/tt-metal/actions/runs/21983835196
1 parent c987548 commit 86b63d7

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

.github/workflows/galaxy-deepseek-tests-impl.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -122,7 +122,7 @@ jobs:
122122
run: |
123123
uv pip install -r models/demos/deepseek_v3/reference/deepseek/requirements.txt
124124
DEEPSEEK_MAX_SEQ_LEN_OVERRIDE=16384 pytest models/demos/deepseek_v3/tests/test_mla.py --timeout 900 --durations=0
125-
DEEPSEEK_MAX_SEQ_LEN_OVERRIDE=16384 pytest models/demos/deepseek_v3/tests/test_decoder_block.py --timeout 2400 --durations=0
125+
DEEPSEEK_MAX_SEQ_LEN_OVERRIDE=12288 pytest models/demos/deepseek_v3/tests/test_decoder_block.py --timeout 2400 --durations=0
126126
# TODO: set DEEPSEEK_MAX_SEQ_LEN_OVERRIDE to large value if test does not get killed in CI.
127127
DEEPSEEK_MAX_SEQ_LEN_OVERRIDE=1024 pytest models/demos/deepseek_v3/tests/test_model.py --timeout 2500 --durations=0
128128
- uses: tenstorrent/tt-metal/.github/actions/slack-report@main

0 commit comments

Comments
 (0)