Intel® Gaudi® AI Accelerators Examples for Training and Inference

• Intel® Gaudi® AI Accelerators Examples for Training and Inference
  • Model List and Performance Data
  • Computer Vision
  • Natural Language Processing
  • Audio
  • Generative Models
  • MLPerf™ Training 4.0
  • MLPerf™ Inference 4.0
  • Reporting Bugs/Feature Requests
• Community
  • Optimum Habana
  • vLLM
  • Megatron-LM
  • Megatron-DeepSpeed

Model List and Performance Data

Please visit this page for performance information.

This repository is a collection of models that have been ported to run on Intel Gaudi AI accelerators. They are intended as examples, and will be reasonably optimized for performance while still being easy to read.

See Community section for other projects with models ported and optimized for Intel Gaudi AI accelerators.

Computer Vision

Models	Framework	Validated on Gaudi	Validated on Gaudi 2	Validated on Gaudi 3
ResNet50	PyTorch	Training (compile)	Training (compile), Inference (compile)	Training (compile)*, Inference (compile)
ResNeXt101	PyTorch	-	Training (compile)	Training (compile)
UNet2D	PyTorch Lightning	Training (compile), Inference (compile)	Training (compile), Inference (compile)	-
Unet3D	PyTorch Lightning	Training (compile), Inference (compile)	Training (compile), Inference (compile)	Training (compile)**
Vision Transformer	PyTorch	Training	-	-
YOLOX	PyTorch	Training	Inference	Inference

*Disclaimer: only on 8x **Disclaimer: only functional checks done

Natural Language Processing

Models	Framework	Validated on Gaudi	Validated on Gaudi 2	Validated on Gaudi 3
BERT Pretraining	PyTorch	Training (compile)	Training (compile)	-
BERT Finetuning	PyTorch	Training, Inference	Training, Inference (compile)	Inference (compile)*
DeepSpeed BERT-1.5B, BERT-5B	PyTorch	Training	Training (compile)	-

*Disclaimer: only bf16

Audio

Models	Framework	Validated on Gaudi	Validated on Gaudi 2	Validated on Gaudi 3
Wav2Vec2ForCTC	PyTorch	Inference	Inference	-

Generative Models

Models	Framework	Validated on Gaudi	Validated on Gaudi 2	Validated on Gaudi 3
Stable Diffusion	PyTorch Lightning	Training	Training	-

MLPerf™ Training 4.0

Models	Framework	Validated on Gaudi	Validated on Gaudi 2	Validated on Gaudi 3
GPT3	PyTorch	-	Training	-
Llama 70B LoRA	PyTorch	-	Training	-

MLPerf™ Inference 4.0

Models	Framework	Validated on Gaudi	Validated on Gaudi 2	Validated on Gaudi 3
Llama 70B	PyTorch	-	Inference	-
Stable Diffusion XL	PyTorch	-	Inference	-

MLPerf™ is a trademark and service mark of MLCommons Association in the United States and other countries. All rights reserved. Unauthorized use is strictly prohibited.

Reporting Bugs/Feature Requests

We welcome you to use the GitHub issue tracker to report bugs or suggest features.

When filing an issue, please check existing open, or recently closed, issues to make sure somebody else hasn't already reported the issue. Please try to include as much information as you can. Details like these are incredibly useful:

• A reproducible test case or series of steps
• The version of our code being used
• Any modifications you've made relevant to the bug
• Anything unusual about your environment or deployment

Community

Other projects with models ported and optimized for Intel Gaudi AI accelerators.

Optimum Habana

HuggingFace models for finetuning and inference are available in Optimum Habana project https://github.com/huggingface/optimum-habana/ and as model cards at https://huggingface.co/Habana.

vLLM

Models optimized for inferece with vLLM are available in HabanaAI/vllm-fork.

Megatron-LM

LLM training models like Llama or Mixtral are available in Megatron-LM's fork: HabanaAI/Megatron-LM.

Megatron-DeepSpeed

⚠️Note that this project will be deprecated and replaced with Megatron-LM.

LLM training models like Llama or Mixtral are available in Megatron-DeepSpeed's fork: HabanaAI/Megatron-DeepSpeed.