Skip to content

Commit 9b76909

Browse files
regisssBhargavckvermaAISolaryee12010486
authored
Upgrade to SynapseAI v1.20 (#1838)
Signed-off-by: Urszula Golowicz <urszula.golowicz@intel.com> Co-authored-by: Bhargav <beede@habana.ai> Co-authored-by: Chetan Kumar Verma <39086835+ckvermaAI@users.noreply.github.com> Co-authored-by: Sheng Yang <yang.sheng@intel.com> Co-authored-by: Silvia Colabrese <silvia.colabrese@intel.com> Co-authored-by: Yaser Afshar <yaser.afshar@intel.com> Co-authored-by: Dmitry <dmitry.smertin@intel.com> Co-authored-by: Urszula Golowicz <urszula.golowicz@intel.com> Co-authored-by: Adam Stachowicz <105052242+astachowiczhabana@users.noreply.github.com>
1 parent 93ade6a commit 9b76909

61 files changed

Lines changed: 311 additions & 223 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/fast_tests.yml‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ jobs:
2121
uses: actions/checkout@v2
2222
- name: Pull image
2323
run: |
24-
docker pull vault.habana.ai/gaudi-docker/1.19.0/ubuntu22.04/habanalabs/pytorch-installer-2.5.1:latest
24+
docker pull vault.habana.ai/gaudi-docker/1.20.0/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest
2525
- name: Run tests
2626
run: |
2727
docker run \
@@ -36,7 +36,7 @@ jobs:
3636
--cap-add=sys_nice \
3737
--net=host \
3838
--ipc=host \
39-
vault.habana.ai/gaudi-docker/1.19.0/ubuntu22.04/habanalabs/pytorch-installer-2.5.1:latest \
39+
vault.habana.ai/gaudi-docker/1.20.0/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest \
4040
/bin/bash tests/ci/fast_tests.sh
4141
diffusers:
4242
name: Run tests for optimum.habana.diffusers
@@ -46,7 +46,7 @@ jobs:
4646
uses: actions/checkout@v2
4747
- name: Pull image
4848
run: |
49-
docker pull vault.habana.ai/gaudi-docker/1.19.0/ubuntu22.04/habanalabs/pytorch-installer-2.5.1:latest
49+
docker pull vault.habana.ai/gaudi-docker/1.20.0/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest
5050
- name: Run tests
5151
run: |
5252
docker run \
@@ -61,5 +61,5 @@ jobs:
6161
--cap-add=sys_nice \
6262
--net=host \
6363
--ipc=host \
64-
vault.habana.ai/gaudi-docker/1.19.0/ubuntu22.04/habanalabs/pytorch-installer-2.5.1:latest \
64+
vault.habana.ai/gaudi-docker/1.20.0/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest \
6565
/bin/bash tests/ci/fast_tests_diffusers.sh

‎.github/workflows/slow_tests.yml‎

Lines changed: 18 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ jobs:
1919
uses: actions/checkout@v2
2020
- name: Pull image
2121
run: |
22-
docker pull vault.habana.ai/gaudi-docker/1.19.0/ubuntu22.04/habanalabs/pytorch-installer-2.5.1:latest
22+
docker pull vault.habana.ai/gaudi-docker/1.20.0/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest
2323
- name: Run tests
2424
run: |
2525
docker run \
@@ -31,7 +31,7 @@ jobs:
3131
--cap-add=sys_nice \
3232
--net=host \
3333
--ipc=host \
34-
vault.habana.ai/gaudi-docker/1.19.0/ubuntu22.04/habanalabs/pytorch-installer-2.5.1:latest \
34+
vault.habana.ai/gaudi-docker/1.20.0/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest \
3535
/bin/bash tests/ci/example_diff_tests.sh
3636
stable-diffusion:
3737
name: Test Stable Diffusion
@@ -45,7 +45,7 @@ jobs:
4545
uses: actions/checkout@v2
4646
- name: Pull image
4747
run: |
48-
docker pull vault.habana.ai/gaudi-docker/1.19.0/ubuntu22.04/habanalabs/pytorch-installer-2.5.1:latest
48+
docker pull vault.habana.ai/gaudi-docker/1.20.0/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest
4949
- name: Run tests
5050
run: |
5151
docker run \
@@ -57,7 +57,7 @@ jobs:
5757
--cap-add=sys_nice \
5858
--net=host \
5959
--ipc=host \
60-
vault.habana.ai/gaudi-docker/1.19.0/ubuntu22.04/habanalabs/pytorch-installer-2.5.1:latest \
60+
vault.habana.ai/gaudi-docker/1.20.0/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest \
6161
/bin/bash tests/ci/slow_tests_diffusers.sh
6262
deepspeed:
6363
name: Test DeepSpeed models
@@ -72,7 +72,7 @@ jobs:
7272
uses: actions/checkout@v2
7373
- name: Pull image
7474
run: |
75-
docker pull vault.habana.ai/gaudi-docker/1.19.0/ubuntu22.04/habanalabs/pytorch-installer-2.5.1:latest
75+
docker pull vault.habana.ai/gaudi-docker/1.20.0/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest
7676
- name: Run tests
7777
run: |
7878
docker run \
@@ -84,7 +84,7 @@ jobs:
8484
--cap-add=sys_nice \
8585
--net=host \
8686
--ipc=host \
87-
vault.habana.ai/gaudi-docker/1.19.0/ubuntu22.04/habanalabs/pytorch-installer-2.5.1:latest \
87+
vault.habana.ai/gaudi-docker/1.20.0/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest \
8888
/bin/bash tests/ci/slow_tests_deepspeed.sh
8989
multi-card:
9090
name: Test multi-card models
@@ -99,7 +99,7 @@ jobs:
9999
uses: actions/checkout@v2
100100
- name: Pull image
101101
run: |
102-
docker pull vault.habana.ai/gaudi-docker/1.19.0/ubuntu22.04/habanalabs/pytorch-installer-2.5.1:latest
102+
docker pull vault.habana.ai/gaudi-docker/1.20.0/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest
103103
- name: Run tests
104104
run: |
105105
docker run \
@@ -111,7 +111,7 @@ jobs:
111111
--cap-add=sys_nice \
112112
--net=host \
113113
--ipc=host \
114-
vault.habana.ai/gaudi-docker/1.19.0/ubuntu22.04/habanalabs/pytorch-installer-2.5.1:latest \
114+
vault.habana.ai/gaudi-docker/1.20.0/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest \
115115
/bin/bash tests/ci/slow_tests_8x.sh
116116
single-card:
117117
name: Test single-card models
@@ -127,7 +127,7 @@ jobs:
127127
uses: actions/checkout@v2
128128
- name: Pull image
129129
run: |
130-
docker pull vault.habana.ai/gaudi-docker/1.19.0/ubuntu22.04/habanalabs/pytorch-installer-2.5.1:latest
130+
docker pull vault.habana.ai/gaudi-docker/1.20.0/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest
131131
- name: Run tests
132132
run: |
133133
docker run \
@@ -139,7 +139,7 @@ jobs:
139139
--cap-add=sys_nice \
140140
--net=host \
141141
--ipc=host \
142-
vault.habana.ai/gaudi-docker/1.19.0/ubuntu22.04/habanalabs/pytorch-installer-2.5.1:latest \
142+
vault.habana.ai/gaudi-docker/1.20.0/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest \
143143
/bin/bash tests/ci/slow_tests_1x.sh
144144
albert-xxl-single-card:
145145
name: Test single-card ALBERT XXL
@@ -158,7 +158,7 @@ jobs:
158158
- name: Pull image
159159
if: github.event.schedule == '0 21 * * 6'
160160
run: |
161-
docker pull vault.habana.ai/gaudi-docker/1.19.0/ubuntu22.04/habanalabs/pytorch-installer-2.5.1:latest
161+
docker pull vault.habana.ai/gaudi-docker/1.20.0/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest
162162
- name: Run test
163163
if: github.event.schedule == '0 21 * * 6'
164164
run: |
@@ -171,7 +171,7 @@ jobs:
171171
--cap-add=sys_nice \
172172
--net=host \
173173
--ipc=host \
174-
vault.habana.ai/gaudi-docker/1.19.0/ubuntu22.04/habanalabs/pytorch-installer-2.5.1:latest \
174+
vault.habana.ai/gaudi-docker/1.20.0/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest \
175175
/bin/bash tests/ci/albert_xxl_1x.sh
176176
- name: Warning
177177
if: github.event.schedule != '0 21 * * 6'
@@ -192,7 +192,7 @@ jobs:
192192
uses: actions/checkout@v2
193193
- name: Pull image
194194
run: |
195-
docker pull vault.habana.ai/gaudi-docker/1.19.0/ubuntu22.04/habanalabs/pytorch-installer-2.5.1:latest
195+
docker pull vault.habana.ai/gaudi-docker/1.20.0/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest
196196
- name: Run tests
197197
run: |
198198
docker run \
@@ -204,7 +204,7 @@ jobs:
204204
--cap-add=sys_nice \
205205
--net=host \
206206
--ipc=host \
207-
vault.habana.ai/gaudi-docker/1.19.0/ubuntu22.04/habanalabs/pytorch-installer-2.5.1:latest \
207+
vault.habana.ai/gaudi-docker/1.20.0/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest \
208208
make slow_tests_text_generation_example TOKEN=${{ secrets.TEXT_GENERATION_CI_HUB_TOKEN }}
209209
trl:
210210
name: Test TRL integration
@@ -223,7 +223,7 @@ jobs:
223223
uses: actions/checkout@v2
224224
- name: Pull image
225225
run: |
226-
docker pull vault.habana.ai/gaudi-docker/1.19.0/ubuntu22.04/habanalabs/pytorch-installer-2.5.1:latest
226+
docker pull vault.habana.ai/gaudi-docker/1.20.0/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest
227227
- name: Run tests
228228
run: |
229229
docker run \
@@ -235,7 +235,7 @@ jobs:
235235
--cap-add=sys_nice \
236236
--net=host \
237237
--ipc=host \
238-
vault.habana.ai/gaudi-docker/1.19.0/ubuntu22.04/habanalabs/pytorch-installer-2.5.1:latest \
238+
vault.habana.ai/gaudi-docker/1.20.0/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest \
239239
/bin/bash tests/ci/slow_tests_trl.sh
240240
sentence-transformers:
241241
name: Test Sentence Transformers integration
@@ -263,7 +263,7 @@ jobs:
263263
path: sentence-transformers
264264
- name: Pull image
265265
run: |
266-
docker pull vault.habana.ai/gaudi-docker/1.19.0/ubuntu22.04/habanalabs/pytorch-installer-2.5.1:latest
266+
docker pull vault.habana.ai/gaudi-docker/1.20.0/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest
267267
- name: Run tests
268268
run: |
269269
docker run \
@@ -275,5 +275,5 @@ jobs:
275275
--cap-add=sys_nice \
276276
--net=host \
277277
--ipc=host \
278-
vault.habana.ai/gaudi-docker/1.19.0/ubuntu22.04/habanalabs/pytorch-installer-2.5.1:latest \
278+
vault.habana.ai/gaudi-docker/1.20.0/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest \
279279
/bin/bash optimum-habana/tests/ci/sentence_transformers.sh

‎.github/workflows/slow_tests_gaudi2.yml‎

Lines changed: 18 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ jobs:
1717
uses: actions/checkout@v2
1818
- name: Pull image
1919
run: |
20-
docker pull vault.habana.ai/gaudi-docker/1.19.0/ubuntu22.04/habanalabs/pytorch-installer-2.5.1:latest
20+
docker pull vault.habana.ai/gaudi-docker/1.20.0/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest
2121
- name: Run tests
2222
run: |
2323
docker run \
@@ -30,7 +30,7 @@ jobs:
3030
--cap-add=sys_nice \
3131
--net=host \
3232
--ipc=host \
33-
vault.habana.ai/gaudi-docker/1.19.0/ubuntu22.04/habanalabs/pytorch-installer-2.5.1:latest \
33+
vault.habana.ai/gaudi-docker/1.20.0/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest \
3434
/bin/bash tests/ci/example_diff_tests.sh
3535
stable-diffusion:
3636
name: Test Stable Diffusion
@@ -43,7 +43,7 @@ jobs:
4343
uses: actions/checkout@v2
4444
- name: Pull image
4545
run: |
46-
docker pull vault.habana.ai/gaudi-docker/1.19.0/ubuntu22.04/habanalabs/pytorch-installer-2.5.1:latest
46+
docker pull vault.habana.ai/gaudi-docker/1.20.0/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest
4747
- name: Run tests
4848
run: |
4949
docker run \
@@ -59,7 +59,7 @@ jobs:
5959
--cap-add=sys_nice \
6060
--net=host \
6161
--ipc=host \
62-
vault.habana.ai/gaudi-docker/1.19.0/ubuntu22.04/habanalabs/pytorch-installer-2.5.1:latest \
62+
vault.habana.ai/gaudi-docker/1.20.0/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest \
6363
/bin/bash tests/ci/slow_tests_diffusers.sh ${{ secrets.TEXT_GENERATION_CI_HUB_TOKEN }}
6464
deepspeed:
6565
name: Test DeepSpeed models
@@ -72,7 +72,7 @@ jobs:
7272
uses: actions/checkout@v2
7373
- name: Pull image
7474
run: |
75-
docker pull vault.habana.ai/gaudi-docker/1.19.0/ubuntu22.04/habanalabs/pytorch-installer-2.5.1:latest
75+
docker pull vault.habana.ai/gaudi-docker/1.20.0/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest
7676
- name: Run tests
7777
run: |
7878
docker run \
@@ -88,7 +88,7 @@ jobs:
8888
--cap-add=sys_nice \
8989
--net=host \
9090
--ipc=host \
91-
vault.habana.ai/gaudi-docker/1.19.0/ubuntu22.04/habanalabs/pytorch-installer-2.5.1:latest \
91+
vault.habana.ai/gaudi-docker/1.20.0/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest \
9292
/bin/bash tests/ci/slow_tests_deepspeed.sh ${{ secrets.TEXT_GENERATION_CI_HUB_TOKEN }}
9393
fsdp:
9494
name: Test FSDP models
@@ -101,7 +101,7 @@ jobs:
101101
uses: actions/checkout@v2
102102
- name: Pull image
103103
run: |
104-
docker pull vault.habana.ai/gaudi-docker/1.19.0/ubuntu22.04/habanalabs/pytorch-installer-2.5.1:latest
104+
docker pull vault.habana.ai/gaudi-docker/1.20.0/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest
105105
- name: Run tests
106106
run: |
107107
docker run \
@@ -117,7 +117,7 @@ jobs:
117117
--cap-add=sys_nice \
118118
--net=host \
119119
--ipc=host \
120-
vault.habana.ai/gaudi-docker/1.19.0/ubuntu22.04/habanalabs/pytorch-installer-2.5.1:latest \
120+
vault.habana.ai/gaudi-docker/1.20.0/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest \
121121
make slow_tests_fsdp TOKEN=${{ secrets.TEXT_GENERATION_CI_HUB_TOKEN }}
122122
multi-card:
123123
name: Test multi-card models
@@ -130,7 +130,7 @@ jobs:
130130
uses: actions/checkout@v2
131131
- name: Pull image
132132
run: |
133-
docker pull vault.habana.ai/gaudi-docker/1.19.0/ubuntu22.04/habanalabs/pytorch-installer-2.5.1:latest
133+
docker pull vault.habana.ai/gaudi-docker/1.20.0/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest
134134
- name: Run tests
135135
run: |
136136
docker run \
@@ -146,7 +146,7 @@ jobs:
146146
--cap-add=sys_nice \
147147
--net=host \
148148
--ipc=host \
149-
vault.habana.ai/gaudi-docker/1.19.0/ubuntu22.04/habanalabs/pytorch-installer-2.5.1:latest \
149+
vault.habana.ai/gaudi-docker/1.20.0/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest \
150150
/bin/bash tests/ci/slow_tests_8x.sh ${{ secrets.TEXT_GENERATION_CI_HUB_TOKEN }}
151151
single-card:
152152
name: Test single-card models
@@ -160,7 +160,7 @@ jobs:
160160
uses: actions/checkout@v2
161161
- name: Pull image
162162
run: |
163-
docker pull vault.habana.ai/gaudi-docker/1.19.0/ubuntu22.04/habanalabs/pytorch-installer-2.5.1:latest
163+
docker pull vault.habana.ai/gaudi-docker/1.20.0/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest
164164
- name: Run tests
165165
run: |
166166
docker run \
@@ -177,7 +177,7 @@ jobs:
177177
--cap-add=sys_nice \
178178
--net=host \
179179
--ipc=host \
180-
vault.habana.ai/gaudi-docker/1.19.0/ubuntu22.04/habanalabs/pytorch-installer-2.5.1:latest \
180+
vault.habana.ai/gaudi-docker/1.20.0/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest \
181181
/bin/bash tests/ci/slow_tests_1x.sh
182182
text-generation:
183183
name: Test text-generation example
@@ -192,7 +192,7 @@ jobs:
192192
uses: actions/checkout@v2
193193
- name: Pull image
194194
run: |
195-
docker pull vault.habana.ai/gaudi-docker/1.19.0/ubuntu22.04/habanalabs/pytorch-installer-2.5.1:latest
195+
docker pull vault.habana.ai/gaudi-docker/1.20.0/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest
196196
- name: Run tests
197197
run: |
198198
docker run \
@@ -208,7 +208,7 @@ jobs:
208208
--cap-add=sys_nice \
209209
--net=host \
210210
--ipc=host \
211-
vault.habana.ai/gaudi-docker/1.19.0/ubuntu22.04/habanalabs/pytorch-installer-2.5.1:latest \
211+
vault.habana.ai/gaudi-docker/1.20.0/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest \
212212
make slow_tests_text_generation_example TOKEN=${{ secrets.TEXT_GENERATION_CI_HUB_TOKEN }}
213213
trl:
214214
name: Test TRL integration
@@ -221,7 +221,7 @@ jobs:
221221
uses: actions/checkout@v2
222222
- name: Pull image
223223
run: |
224-
docker pull vault.habana.ai/gaudi-docker/1.19.0/ubuntu22.04/habanalabs/pytorch-installer-2.5.1:latest
224+
docker pull vault.habana.ai/gaudi-docker/1.20.0/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest
225225
- name: Run tests
226226
run: |
227227
docker run \
@@ -237,7 +237,7 @@ jobs:
237237
--cap-add=sys_nice \
238238
--net=host \
239239
--ipc=host \
240-
vault.habana.ai/gaudi-docker/1.19.0/ubuntu22.04/habanalabs/pytorch-installer-2.5.1:latest \
240+
vault.habana.ai/gaudi-docker/1.20.0/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest \
241241
/bin/bash tests/ci/slow_tests_trl.sh
242242
sentence-transformers:
243243
name: Test Sentence Transformers integration
@@ -258,7 +258,7 @@ jobs:
258258
path: sentence-transformers
259259
- name: Pull image
260260
run: |
261-
docker pull vault.habana.ai/gaudi-docker/1.19.0/ubuntu22.04/habanalabs/pytorch-installer-2.5.1:latest
261+
docker pull vault.habana.ai/gaudi-docker/1.20.0/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest
262262
- name: Run tests
263263
run: |
264264
docker run \
@@ -274,5 +274,5 @@ jobs:
274274
--cap-add=sys_nice \
275275
--net=host \
276276
--ipc=host \
277-
vault.habana.ai/gaudi-docker/1.19.0/ubuntu22.04/habanalabs/pytorch-installer-2.5.1:latest \
277+
vault.habana.ai/gaudi-docker/1.20.0/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:latest \
278278
/bin/bash optimum-habana/tests/ci/sentence_transformers.sh

‎Makefile‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -93,7 +93,7 @@ slow_tests_8x: test_installs
9393

9494
# Run DeepSpeed non-regression tests
9595
slow_tests_deepspeed: test_installs
96-
python -m pip install git+https://github.com/HabanaAI/DeepSpeed.git@1.19.0
96+
python -m pip install git+https://github.com/HabanaAI/DeepSpeed.git@1.20.0
9797
python -m pytest tests/test_examples.py -v -s -k "deepspeed"
9898

9999
slow_tests_diffusers: test_installs
@@ -109,7 +109,7 @@ slow_tests_diffusers: test_installs
109109
slow_tests_text_generation_example: test_installs
110110
python -m pip install -r examples/text-generation/requirements_awq.txt
111111
BUILD_CUDA_EXT=0 python -m pip install -vvv --no-build-isolation git+https://github.com/HabanaAI/AutoGPTQ.git
112-
python -m pip install git+https://github.com/HabanaAI/DeepSpeed.git@1.19.0
112+
python -m pip install git+https://github.com/HabanaAI/DeepSpeed.git@1.20.0
113113
python -m pytest tests/test_text_generation_example.py tests/test_encoder_decoder.py -v -s --token $(TOKEN)
114114

115115
# Run image-to-text non-regression tests

0 commit comments

Comments
 (0)