Commit a79b53a
Add nearest-interpolation decomposition for floating scalar factors, downsampling, and mixed sampling (resize1d & resize2d) (#2885)
Fixes #2820
Fixes #2017
This PR adds decomposition support for **nearest** interpolation in:
- `resize2d` — supports **floating scalar factors**, **pure
downsampling**, and **mixed sampling** (upsample in one axis, downsample
in the other).
- `resize1d` — supports **floating scalar factors** and
**downsampling**.
The decomposition rewrites nearest-neighbor into two primitive
operations:
1. **Upsampling** via `repeat_interleave` (integer repetition).
2. **Downsampling** via **depthwise `conv2d`** with a pick-first kernel
(select the first element in each stride window).
Nearest-neighbor scaling with non-integer (floating) scale factors can
be represented exactly by factoring input and output sizes using their
greatest common divisor (gcd). This produces integer `up` and `down`
factors for each axis:
- `g = gcd(out_size, in_size)`
- `up = out_size / g` → how many times each source element must be
repeated
- `down = in_size / g` → how many source elements are collapsed into one
output
Algorithm (applied per spatial axis):
1. Compute `g = gcd(out_size, in_size)`.
2. `up = out_size / g` — if `up > 1`, call `repeat_interleave(...,
repeats=up, dim=axis)` to expand each element `up` times on that axis.
3. `down = in_size / g` — if `down != 1`, apply a **depthwise `conv2d`**
with:
- a kernel shaped to pick the first element in the conv window,
- stride = `down`,
- no padding, no dilation,
- `groups = C` (depthwise conv).
Why this reproduces nearest-neighbor:
- `repeat_interleave` expands each input element into an integer block
of identical values (upsample).
- The depthwise conv kernel contains zeros except a single `1` at its
top-left (first) position; with stride=`down`, the conv selects the
first element from each `down`-sized block (pick-first semantics).
Together, expand → pick-first collapses create the exact top-left
variant of nearest-neighbor for any rational scale factor.
Implementation notes:
- For 2D:
- Upsampling applied independently on H and W (code applies repeat H
then repeat W).
- Width-downsample kernel: `(C, 1, 1, down_w)`, stride `(1, down_w)`.
- Height-downsample kernel: `(C, 1, down_h, 1)`, stride `(down_h, 1)`.
- For 1D:
- Temporarily `unsqueeze` `(N, C, W)` → `(N, C, 1, W)` to reuse the same
conv2d path, then `squeeze` back to 3D.
**Input:** `(N=1, C=3, H_in=5, W_in=7)`
**Target:** `size = (H_out=9, W_out=5)`
Per-dim gcd decomposition:
- H: `g_h = gcd(9, 5) = 1` → `up_h = 9`, `down_h = 5`
- W: `g_w = gcd(5, 7) = 1` → `up_w = 5`, `down_w = 7`
```
Input: (1, 3, 5, 7)
│
▼
repeat_interleave(dim=H, repeats=up_h=9)
▼
Shape: (1, 3, 45, 7) # each row repeated 9x
│
▼
repeat_interleave(dim=W, repeats=up_w=5)
▼
Shape: (1, 3, 45, 35) # each column repeated 5x
│
▼
conv2d(depthwise, kernel=(C,1,1,down_w=7), stride=(1,7))
→ picks first column in each block of width 7
▼
Shape: (1, 3, 45, 5) # width reduced to W_out=5
│
▼
conv2d(depthwise, kernel=(C,1,down_h=5,1), stride=(5,1))
→ picks first row in each block of height 5
▼
Output: (1, 3, 9, 5) # height reduced to H_out=9
```
**Input:** `(1, 3, W_in=9)`
**Target:** `size = (W_out=7,)`
Compute:
- `g = gcd(9, 7) = 1` → `up_w = 7`, `down_w = 9`
```
Input: (1, 3, 9)
│
▼
repeat_interleave(dim=W, repeats=up_w=7)
▼
Shape: (1, 3, 63) # every element repeated 7x
│
▼
unsqueeze to 4D for conv2d:
Temp: (1, 3, 1, 63)
│
▼
conv2d(depthwise, kernel=(C,1,1,down_w=9), stride=(1,9))
→ picks first element in each block of width 9
▼
Shape: (1, 3, 1, 7)
│
▼
squeeze back to 3D:
Output: (1, 3, 7)
```
RuntimeError: TT_ASSERT @
/__w/tt-forge-fe/tt-forge-fe/forge/csrc/ops/op_resize_2d.cpp:104:
(input_h % size_h == 0) && (input_w % size_w == 0) info: info:
This occurred because the original implementation only supported integer
downsampling ratios.
- **GHOSTNET**
- PyTorch:
`forge/test/models/pytorch/vision/ghostnet/test_ghostnet.py::test_ghostnet_timm[ghostnetv2_100.in1k]`
- **DETR**
- ONNX:
`forge/test/models/onnx/vision/detr/test_detr.py::test_detr_segmentation_onnx[facebook/detr-resnet-50-panoptic]`
- PyTorch:
`forge/test/models/pytorch/vision/detr/test_detr.py::test_detr_segmentation[resnet_50_panoptic]`
`forge/test/models/pytorch/vision/detr/test_detr.py::test_detr_detection[resnet_50]`
- **SSD300_VGG16**
- ONNX:
`forge/test/models/onnx/vision/ssd300_vgg16/test_ssd300_vgg16_onnx.py::test_ssd300_vgg16[ssd300_vgg16]`
- PyTorch:
`forge/test/models/pytorch/vision/ssd300_vgg16/test_ssd300_vgg16.py::test_ssd300_vgg16[ssd300_vgg16]`
- **GHOSTNET**: Passing end-to-end, `xfail` markers removed.
- **DETR** & **SSD300_VGG16**: Resize issue resolved, but failing with
**Out of Memory (OOM)** at runtime.
- Additional fix: Corrected DETR input data format issues in
`tt_forge_models` repo.
[test_detr.log](https://github.com/user-attachments/files/22041939/test_detr.log)
[test_ghostnet_timm.log](https://github.com/user-attachments/files/22041942/test_ghostnet_timm.log)
[test_resize2d_nearest_interpolation.log](https://github.com/user-attachments/files/22041943/test_resize2d_nearest_interpolation.log)
[test_ssd300_vgg16_onnx.log](https://github.com/user-attachments/files/22041944/test_ssd300_vgg16_onnx.log)
[test_ssd300_vgg16.log](https://github.com/user-attachments/files/22041945/test_ssd300_vgg16.log)
Add Xfail Marker (#2942)
[Issue_1](#2940)
[Issue_2](#2888)
Facing Crash issue in nightly pipeline
Added Xfail Marker
cc: @nvukobratTT
Fix the albert & distilbert postprocessing and invalid config issue (#2858)
[Issue](#2870)
Due to the framework_models variable which is reused for the storing
onnx model which causes the config issue and updated the correct logic
for the albert and distilbert postprocessing and modified the regnet and
dla to follow the export script based on the tt_forge_models
The testscripts of the dla,regnet,albert,distilbert has been modified.
- [x] New/Existing tests provide coverage for changes
Fix model analysis failures (#2937)
This PR fixes multiple issues in **model analysis** and **models ops
tests**:
- Resolves configuration extraction failure for the `speecht5` model.
- Fixes failures in `cognito`, `segformer`, and `whisper` models.
- Adds generated models ops tests for `speecht5`, `cognito`, and `unet`.
- Adds `xfail` markers for known failing models ops tests.
- Updates `resize2d` models ops tests to align with the
[nearest-interpolation decomposition
uplift](#2885).
1. **Speecht5 Config Extraction Fix**
- Issue: While extracting unique ops config for `speecht5`, the
`multiply` op had an empty tuple `()` as input shape for the parameter
node type.
- Root Cause: The input shape in the TVM JSON graph was empty because it
represents a scalar tensor.
- Fix: Updated the ops config extraction function to use the actual
parameter tensor when the node shape is an empty tuple.
2. **Cognito Model Fix**
- Issue: Failure with
```
huggingface_hub.errors.HFValidationError: Repo id must use alphanumeric
chars or '-', '_', '.', '--' and '..' are forbidden, '-' and '.' cannot
start or end the name, max length is 96
```
- Root Cause: After porting model/tokenizer loading to
`tt_forge_models`, the model was also being passed into `CogitoWrapper`,
causing duplicate loading and invalid repo IDs.
- Fix:
- Removed redundant `model_utils` file for model/input sample loading.
- Removed `CogitoWrapper` from test cases.
- Applied similar fixes for the Cognito ONNX model.
3. **Whisper Model Fix**
- Issue:
```
framework_model = Wrapper(framework_model)
UnboundLocalError: local variable 'framework_model' referenced before
assignment
```
- Fix: Corrected initialization by passing the model variable directly
into the wrapper.
4. **Generated Models Ops Tests**
- Added ops tests for:
- `speecht5`
- `cognito`
- `unet`
- Since config extraction issues are now resolved.
- Added `xfail` markers for known failing models ops tests.
5. **Resize2D Models Ops Tests Update**
- Updated skipped tests with `xfail` markers for failing cases.
- Aligned with the uplift of [nearest-interpolation decomposition
patch](#2885).
Fix group and priority property recording (#2941)
This PR fixes incorrect or missing **model group** and **priority**
assignments by updating the `ModelGroup` and `ModelPriority` properties
for Qwen v3, MobileNet v2, Swin, and Falcon models.
- **From `GENERALITY → RED`**
- **From `P2 → P1`**
- `Qwen/Qwen3-Embedding-8B`
- `Qwen/Qwen3-Embedding-4B`
- `swin_v2_s`
- `Falcon_1B`
- `Falcon_3B`
- `Falcon_7B`
- `Falcon_10B`
- `mobilnet_v2`
[Deps] Upgrades Jax to 0.7 and python to 3.11 (#2918)
Fixes : [Link to Github
Issue](#2914)
- [x] Upgrade Jax to version 0.7.
- [x] Upgrade Python to version 3.11.
- [x] includes mlir uplift to latest main
- [x] New/Existing tests provide coverage for changes1 parent bc10933 commit a79b53a
File tree
75 files changed
+3536
-307
lines changed- .github
- workflows
- docs/src
- env
- forge
- csrc
- ops
- test
- forge
- test
- mlir/operators/nn
- models_ops
- models
- onnx
- text
- albert
- deepcogito
- distilbert
- vision
- dla
- regnet
- paddlepaddle/multimodal/paddleocr
- pytorch
- audio/whisper
- text
- deepcogito
- model_utils
- falcon
- qwen
- vision
- ghostnet
- mobilenet
- segformer
- swin
- unet
- third_party
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
75 files changed
+3536
-307
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
8 | 8 | | |
9 | 9 | | |
10 | 10 | | |
11 | | - | |
12 | | - | |
13 | 11 | | |
14 | 12 | | |
15 | 13 | | |
| |||
61 | 59 | | |
62 | 60 | | |
63 | 61 | | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
64 | 71 | | |
65 | 72 | | |
66 | 73 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
| 10 | + | |
10 | 11 | | |
11 | 12 | | |
12 | 13 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
31 | 31 | | |
32 | 32 | | |
33 | 33 | | |
34 | | - | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
35 | 37 | | |
36 | 38 | | |
37 | 39 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
58 | 58 | | |
59 | 59 | | |
60 | 60 | | |
61 | | - | |
| 61 | + | |
62 | 62 | | |
63 | 63 | | |
64 | 64 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
12 | 12 | | |
13 | 13 | | |
14 | 14 | | |
15 | | - | |
| 15 | + | |
16 | 16 | | |
17 | 17 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
25 | | - | |
| 25 | + | |
26 | 26 | | |
27 | 27 | | |
28 | 28 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
16 | | - | |
| 16 | + | |
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
68 | 68 | | |
69 | 69 | | |
70 | 70 | | |
71 | | - | |
| 71 | + | |
72 | 72 | | |
73 | 73 | | |
74 | 74 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
8 | 8 | | |
9 | 9 | | |
10 | 10 | | |
11 | | - | |
| 11 | + | |
12 | 12 | | |
13 | 13 | | |
14 | 14 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
38 | 38 | | |
39 | 39 | | |
40 | 40 | | |
41 | | - | |
| 41 | + | |
42 | 42 | | |
43 | 43 | | |
44 | 44 | | |
| |||
84 | 84 | | |
85 | 85 | | |
86 | 86 | | |
87 | | - | |
| 87 | + | |
88 | 88 | | |
89 | 89 | | |
90 | 90 | | |
91 | 91 | | |
92 | 92 | | |
93 | | - | |
| 93 | + | |
94 | 94 | | |
95 | 95 | | |
96 | | - | |
| 96 | + | |
97 | 97 | | |
98 | 98 | | |
99 | 99 | | |
| |||
280 | 280 | | |
281 | 281 | | |
282 | 282 | | |
283 | | - | |
| 283 | + | |
0 commit comments