Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
78 changes: 75 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,81 @@

TT-Forge is Tenstorrent's MLIR-based compiler. It integrates into various compiler technologies from AI/ML frameworks, to both enable running models and create custom kernel generation.

-----
# TT-Forge Quick Start Guide

## Overview
TT-Forge is Tenstorrent's ML software stack for running PyTorch, JAX, and other framework models on Tenstorrent hardware. This guide will get you up and running in minutes.

## Prerequisites
- [Hardware Setup](https://docs.tenstorrent.com/getting-started/README.html)
- Python 3.11 or higher
- pip 21.0 or higher (for pip installation)
- Docker 20.10 or higher (for Docker installation)

## Installation

### Option 1: Install via pip (Assumes you've ran tt-installer)
```bash
export VLLM_TARGET_DEVICE="empty"

pip install tt-forge --extra-index-url https://pypi.eng.aws.tenstorrent.com/
```

### Option 2: Install via Docker
Pull & Run the container:
```bash
docker run -it --rm \
--device /dev/tenstorrent \
-v /dev/hugepages-1G:/dev/hugepages-1G \
ghcr.io/tenstorrent/tt-forge:latest
```

## Verify Installation

```python
import torch
import torch_xla.core.xla_model as xm
import torch_xla.runtime as xr

def test_add():
# 1. Explicitly set the device type to Tenstorrent
# This ensures the TT PJRT plugin is loaded correctly.
xr.set_device_type("TT")

print("Initializing Tenstorrent XLA device...")

# 2. Acquire the device
device = xm.xla_device()
print(f"Targeting device: {device}")

# 3. Create tensors directly on the device
t1 = torch.tensor([10.0, 20.0], device=device)
t2 = torch.tensor([30.0, 40.0], device=device)

# 4. Perform operation (Lazy execution)
res = t1 + t2

# 5. Sync and print (Forces execution)
# The .cpu() call triggers the graph execution on the hardware
print(f"Result: {res.cpu()}")

if __name__ == "__main__":
test_add()
```

## Running Your First Model

### PyTorch Example
```bash
python tt-forge/demos/tt-xla/cnn/resnet_demo.py
```

### JAX Example
```bash
python tt-forge/demos/tt-xla/nlp/jax/gpt_demo.py
```

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think we should include a benchmark example? @odjuricicTT

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@odjuricicTT just checking in if you have any feedback

-----
# Quick Links
- [Getting Started / How to Run a Model](https://docs.tenstorrent.com/tt-forge/getting_started.html)
Expand All @@ -29,9 +104,6 @@ TT-Forge is Tenstorrent's MLIR-based compiler. It integrates into various compil
# What Is This Repo?
This repository is the central hub for the TT-Forge compiler project, bringing together its various sub-projects into a cohesive product. Here, you'll find releases, demos, model support, roadmaps, and other key resources as the project evolves. Please file any issues with questions or feedback you may have [here](https://github.com/tenstorrent/tt-forge/issues).

# Getting Started Guide
See the documentation available for individual front ends in the [Front End](#current-ai-framework-front-end-projects) section to get started running some tests. You can also try running a demo using the [TT-Forge Getting Started](https://docs.tenstorrent.com/tt-forge/) page.

# Project Goals
- Provide abstraction of many different frontend frameworks
- Generically compile many kinds of model architectures without modification and with good performance
Expand Down