Error: Shardy propagation only supports ranked tensors with a static shape in Deepseek-OCR #3316
Description
Describe the bug
- Deepseek-OCR model facing this
error: Shardy propagation only supports ranked tensors with a static shape. type: 'tensor<?x2xi32, #stablehlo.bounds<1168640, ?>>'- feb16_deepseek_ocr.log
Root cause analysis
- Issue originates from this masked_scatter operation
- Return tensors before masked scatter -> pcc drop - feb16_deepseek_ocr_d1.log, modeling_deepseekocr.py
- Return tensors after masked scatter -> original error - feb16_deepseek_ocr_d2.log, modeling_deepseekocr.py
- Replicated the error with sanity - feb16_masked_scatter.log
- This PR has decomposition of masked_scatter which fixes this dynamic shape issue.
- After adding this fix , sanity passed with
PCC=1. feb16_masked_scatter_after_fix.log, decompositions.py - But issue in whole model persists. feb16_masked_scatter_after_fix.log
- Decomposition is not invoked in whole model run. Needs further investigation
Repro
git checkout kkannan/feb16_deepseekocr_dyn_issue
git submodule update --init --recursive
git lfs pull --include '*.pt'
# whole model run
pytest tests/runner/test_models.py::test_all_models_torch[deepseek/deepseek_ocr/pytorch-Ocr-single_device-inference] -svv --runxfail
# sanity run
pytest tests/torch/ops/test_masked_scatter.py -svv