|
1 | | -<div align="right"><a href="/README_cn.md">中文版</a></div> |
2 | | - |
3 | | -## <img width="30" height="30" alt="FlagTree-GitHub" src="https://github.com/user-attachments/assets/d8d24c81-6f46-4adc-94e2-b89b03afcb43" /> FlagTree |
4 | | - |
5 | | -FlagTree is an open source, unified compiler for multiple AI chips project dedicated to developing a diverse ecosystem of AI chip compilers and related tooling platforms, thereby fostering and strengthening the upstream and downstream Triton ecosystem. Currently in its initial phase, the project aims to maintain compatibility with existing adaptation solutions while unifying the codebase to rapidly implement single-repository multi-backend support. For upstream model users, it provides unified compilation capabilities across multiple backends; for downstream chip manufacturers, it offers examples of Triton ecosystem integration. |
6 | | - |
7 | | -## Latest News |
8 | | -* 2025/12/08 Added enflame backend integration, and added CI/CD. |
9 | | -* 2025/11/26 Add FlagTree_Backend_Specialization Unified Design Document [FlagTree_Backend_Specialization](reports/decoupling/). |
10 | | -* 2025/10/28 Provides offline build support (pre-downloaded dependency packages), improving the build experience when network environment is limited. See usage instructions below. |
11 | | -* 2025/09/30 Support flagtree_hints for shared memory on GPGPU. |
12 | | -* 2025/09/29 SDK storage migrated to ksyuncs, improving download stability. |
13 | | -* 2025/09/25 Support flagtree_hints for ascend backend compilation capability. |
14 | | -* 2025/09/16 Added hcu backend integration, and added CI/CD. |
15 | | -* 2025/09/09 Forked and modified llvm-project to support FLIR. |
16 | | -* 2025/09/01 Added adaptation for Paddle framework, and added CI/CD. |
17 | | -* 2025/08/16 Added adaptation for Beijing Super Cloud Computing Center. |
18 | | -* 2025/08/04 Added T*** backend integration. |
19 | | -* 2025/08/01 FLIR supports flagtree_hints for shared memory loading. |
20 | | -* 2025/07/30 Updated cambricon backend. |
21 | | -* 2025/07/25 Inspur team added adaptation for OpenAnolis OS. |
22 | | -* 2025/07/09 FLIR supports flagtree_hints for Async DMA. |
23 | | -* 2025/07/08 Added UnifiedHardware manager for multi-backend compilation. |
24 | | -* 2025/07/02 FlagGems LibTuner adapted to triton_v3.3.x version. |
25 | | -* 2025/07/02 Added S*** backend integration. |
26 | | -* 2025/06/20 FLIR began supporting MLIR extension functionality. |
27 | | -* 2025/06/06 Added tsingmicro backend integration, and added CI/CD. |
28 | | -* 2025/06/04 Added ascend backend integration, and added CI/CD. |
29 | | -* 2025/06/03 Added metax backend integration, and added CI/CD. |
30 | | -* 2025/05/22 FlagGems LibEntry adapted to triton_v3.3.x version. |
31 | | -* 2025/05/21 FLIR began supporting conversion functionality to middle layer. |
32 | | -* 2025/04/09 Added arm aipu backend integration, provided torch standard extension examples, and added CI/CD. |
33 | | -* 2025/03/26 Integrated security compliance scanning. |
34 | | -* 2025/03/19 Added klx xpu backend integration, and added CI/CD. |
35 | | -* 2025/03/19 Added mthreads backend integration, and added CI/CD. |
36 | | -* 2025/03/12 Added iluvatar backend integration, and added CI/CD. |
37 | | - |
38 | | -## Install from source |
39 | | -Installation dependencies (ensure you use the correct python3.x version): |
40 | | -```shell |
41 | | -apt install zlib1g zlib1g-dev libxml2 libxml2-dev # ubuntu |
42 | | -cd python; python3 -m pip install -r requirements.txt |
43 | | -``` |
44 | | - |
45 | | -Building and Installation (Recommended for environments with good network connectivity): |
46 | | -```shell |
47 | | -cd python |
48 | | -export FLAGTREE_BACKEND=backendxxx |
49 | | -python3 -m pip install . --no-build-isolation -v |
50 | | -``` |
51 | | - |
52 | | -## Tips for building |
53 | | - |
54 | | -Automatic dependency library downloads may be limited by network conditions. You can manually download to the cache directory ~/.flagtree (modifiable via the FLAGTREE_CACHE_DIR environment variable). No need to manually set LLVM environment variables such as LLVM_BUILD_DIR. |
55 | | -Complete build commands for each backend: |
56 | | - |
57 | | -[iluvatar](https://github.com/FlagTree/flagtree/tree/main/third_party/iluvatar/) |
58 | | -The corresponding Triton version is 3.1 |
59 | | -```shell |
60 | | -# Recommended: Use Ubuntu 20.04 |
61 | | -mkdir -p ~/.flagtree/iluvatar; cd ~/.flagtree/iluvatar |
62 | | -wget https://baai-cp-web.ks3-cn-beijing.ksyuncs.com/trans/iluvatar-llvm18-x86_64_v0.3.0.tar.gz |
63 | | -tar zxvf iluvatar-llvm18-x86_64_v0.3.0.tar.gz |
64 | | -wget https://baai-cp-web.ks3-cn-beijing.ksyuncs.com/trans/iluvatarTritonPlugin-cpython3.10-glibc2.30-glibcxx3.4.28-cxxabi1.3.12-ubuntu-x86_64_v0.3.0.tar.gz |
65 | | -tar zxvf iluvatarTritonPlugin-cpython3.10-glibc2.30-glibcxx3.4.28-cxxabi1.3.12-ubuntu-x86_64_v0.3.0.tar.gz |
66 | | -cd ${YOUR_CODE_DIR}/flagtree/python |
67 | | -export FLAGTREE_BACKEND=iluvatar |
68 | | -python3 -m pip install . --no-build-isolation -v |
69 | | -``` |
70 | | -[xpu (klx)](https://github.com/FlagTree/flagtree/tree/main/third_party/xpu/) |
71 | | -The corresponding Triton version is 3.0 |
72 | | -```shell |
73 | | -# Recommended: Use the Docker image (22GB) https://su.bcebos.com/klx-sdk-release-public/xpytorch/docker/ubuntu2004_v030/ubuntu_2004_x86_64_v30.tar |
74 | | -# Contact kunlunxin-support@baidu.com for support |
75 | | -mkdir -p ~/.flagtree/xpu; cd ~/.flagtree/xpu |
76 | | -wget https://baai-cp-web.ks3-cn-beijing.ksyuncs.com/trans/XTDK-llvm19-ubuntu2004_x86_64_v0.3.0.tar.gz |
77 | | -tar zxvf XTDK-llvm19-ubuntu2004_x86_64_v0.3.0.tar.gz |
78 | | -wget https://baai-cp-web.ks3-cn-beijing.ksyuncs.com/trans/xre-Linux-x86_64_v0.3.0.tar.gz |
79 | | -tar zxvf xre-Linux-x86_64_v0.3.0.tar.gz |
80 | | -cd ${YOUR_CODE_DIR}/flagtree/python |
81 | | -export FLAGTREE_BACKEND=xpu |
82 | | -python3 -m pip install . --no-build-isolation -v |
83 | | -``` |
84 | | -[mthreads](https://github.com/FlagTree/flagtree/tree/main/third_party/mthreads/) |
85 | | -The corresponding Triton version is 3.1 |
86 | | -```shell |
87 | | -# Recommended: Use the Dockerfile flagtree/dockerfiles/Dockerfile-ubuntu22.04-python3.10-mthreads |
88 | | -mkdir -p ~/.flagtree/mthreads; cd ~/.flagtree/mthreads |
89 | | -wget https://baai-cp-web.ks3-cn-beijing.ksyuncs.com/trans/mthreads-llvm19-glibc2.35-glibcxx3.4.30-x64_v0.4.0.tar.gz |
90 | | -tar zxvf mthreads-llvm19-glibc2.35-glibcxx3.4.30-x64_v0.4.0.tar.gz |
91 | | -wget https://baai-cp-web.ks3-cn-beijing.ksyuncs.com/trans/mthreadsTritonPlugin-cpython3.10-glibc2.35-glibcxx3.4.30-cxxabi1.3.13-ubuntu-x86_64_v0.4.0.tar.gz |
92 | | -tar zxvf mthreadsTritonPlugin-cpython3.10-glibc2.35-glibcxx3.4.30-cxxabi1.3.13-ubuntu-x86_64_v0.4.0.tar.gz |
93 | | -cd ${YOUR_CODE_DIR}/flagtree/python |
94 | | -export FLAGTREE_BACKEND=mthreads |
95 | | -python3 -m pip install . --no-build-isolation -v |
96 | | -``` |
97 | | -[aipu (arm npu)](https://github.com/FlagTree/flagtree/tree/triton_v3.3.x/third_party/aipu/) |
98 | | -The corresponding Triton version is 3.3 |
99 | | -```shell |
100 | | -# Recommended: Use Ubuntu 22.04 |
101 | | -mkdir -p ~/.flagtree/aipu; cd ~/.flagtree/aipu |
102 | | -# x64 in the simulated environment, arm64 on the ARM development board |
103 | | -wget https://baai-cp-web.ks3-cn-beijing.ksyuncs.com/trans/llvm-a66376b0-ubuntu-x64-clang16-lld16_v0.4.0.tar.gz |
104 | | -tar zxvf llvm-a66376b0-ubuntu-x64-clang16-lld16_v0.4.0.tar.gz |
105 | | -cd ${YOUR_CODE_DIR}/flagtree/ |
106 | | -git checkout -b triton_v3.3.x origin/triton_v3.3.x |
107 | | -export FLAGTREE_BACKEND=aipu |
108 | | -python3 -m pip install . --no-build-isolation -v |
109 | | -``` |
110 | | -[tsingmicro](https://github.com/FlagTree/flagtree/tree/triton_v3.3.x/third_party/tsingmicro/) |
111 | | -The corresponding Triton version is 3.3 |
112 | | -```shell |
113 | | -# Recommended: Use Ubuntu 20.04 |
114 | | -mkdir -p ~/.flagtree/tsingmicro; cd ~/.flagtree/tsingmicro |
115 | | -wget https://baai-cp-web.ks3-cn-beijing.ksyuncs.com/trans/tsingmicro-llvm21-glibc2.30-glibcxx3.4.28-python3.11-x64_v0.2.0.tar.gz |
116 | | -tar zxvf tsingmicro-llvm21-glibc2.30-glibcxx3.4.28-python3.11-x64_v0.2.0.tar.gz |
117 | | -wget https://baai-cp-web.ks3-cn-beijing.ksyuncs.com/trans/tx8_depends_release_20250814_195126_v0.2.0.tar.gz |
118 | | -tar zxvf tx8_depends_release_20250814_195126_v0.2.0.tar.gz |
119 | | -export TX8_DEPS_ROOT=~/.flagtree/tsingmicro/tx8_deps |
120 | | -cd ${YOUR_CODE_DIR}/flagtree/ |
121 | | -git checkout -b triton_v3.3.x origin/triton_v3.3.x |
122 | | -export FLAGTREE_BACKEND=tsingmicro |
123 | | -python3 -m pip install . --no-build-isolation -v |
124 | | -``` |
125 | | -[ascend](https://github.com/FlagTree/flagtree/blob/triton_v3.2.x/python/setup_tools/setup_helper.py) |
126 | | -The corresponding Triton version is 3.3, based on the aarch64 platform |
127 | | -```shell |
128 | | -# Recommended: Use the Dockerfile flagtree/dockerfiles/Dockerfile-ubuntu22.04-python3.11-ascend |
129 | | -# After registering an account at https://www.hiascend.com/developer/download/community/result?module=cann, |
130 | | -# download the cann-toolkit and cann-kernels for the corresponding platform. |
131 | | -# cann-toolkit |
132 | | -chmod +x Ascend-cann-toolkit_8.3.RC1.alpha001_linux-aarch64.run |
133 | | -./Ascend-cann-toolkit_8.3.RC1.alpha001_linux-aarch64.run --install |
134 | | -# cann-kernels for 910B (A2) |
135 | | -chmod +x Ascend-cann-kernels-910b_8.3.RC1.alpha001_linux-aarch64.run |
136 | | -./Ascend-cann-kernels-910b_8.3.RC1.alpha001_linux-aarch64.run --install |
137 | | -# cann-kernels for 910C (A3) |
138 | | -chmod +x Atlas-A3-cann-kernels_8.3.RC1.alpha001_linux-aarch64.run |
139 | | -./Atlas-A3-cann-kernels_8.3.RC1.alpha001_linux-aarch64.run --install |
140 | | -# build |
141 | | -mkdir -p ~/.flagtree/ascend; cd ~/.flagtree/ascend |
142 | | -wget https://oaitriton.blob.core.windows.net/public/llvm-builds/llvm-b5cc222d-ubuntu-arm64.tar.gz |
143 | | -tar zxvf llvm-b5cc222d-ubuntu-arm64.tar.gz |
144 | | -cd ${YOUR_CODE_DIR}/flagtree/python |
145 | | -git checkout -b triton_v3.2.x origin/triton_v3.2.x |
146 | | -export FLAGTREE_BACKEND=ascend |
147 | | -python3 -m pip install . --no-build-isolation -v |
148 | | -``` |
149 | | -[hcu](https://github.com/FlagTree/flagtree/tree/main/third_party/hcu/) |
150 | | -The corresponding Triton version is 3.0 |
151 | | -```shell |
152 | | -# Recommended: Use the Dockerfile flagtree/dockerfiles/Dockerfile-ubuntu22.04-python3.10-hcu |
153 | | -mkdir -p ~/.flagtree/hcu; cd ~/.flagtree/hcu |
154 | | -wget https://baai-cp-web.ks3-cn-beijing.ksyuncs.com/trans/hcu-llvm20-df0864e-glibc2.35-glibcxx3.4.30-ubuntu-x86_64_v0.3.0.tar.gz |
155 | | -tar zxvf hcu-llvm20-df0864e-glibc2.35-glibcxx3.4.30-ubuntu-x86_64_v0.3.0.tar.gz |
156 | | -cd ${YOUR_CODE_DIR}/flagtree/python |
157 | | -export FLAGTREE_BACKEND=hcu |
158 | | -python3 -m pip install . --no-build-isolation -v |
159 | | -``` |
160 | | -[enflame](https://github.com/FlagTree/flagtree/tree/triton_v3.3.x/third_party/enflame/) |
161 | | -The corresponding Triton version is 3.3 |
162 | | -```shell |
163 | | -# Recommended: Use the Docker image (2.4GB) https://baai-cp-web.ks3-cn-beijing.ksyuncs.com/trans/enflame-flagtree-0.3.1.tar.gz |
164 | | -mkdir -p ~/.flagtree/enflame; cd ~/.flagtree/enflame |
165 | | -wget https://baai-cp-web.ks3-cn-beijing.ksyuncs.com/trans/enflame-llvm21-d752c5b-gcc9-x64_v0.3.0.tar.gz |
166 | | -tar zxvf enflame-llvm21-d752c5b-gcc9-x64_v0.3.0.tar.gz |
167 | | -cd ${YOUR_CODE_DIR}/flagtree/python |
168 | | -export FLAGTREE_BACKEND=enflame |
169 | | -python3 -m pip install . --no-build-isolation -v |
170 | | -``` |
171 | | - |
172 | | -[nvidia](/third_party/nvidia/) |
173 | | -To build with default backends nvidia, amd, triton_shared cpu: |
174 | | -```shell |
175 | | -cd ${YOUR_LLVM_DOWNLOAD_DIR} |
176 | | -# For Triton 3.1 |
177 | | -wget https://oaitriton.blob.core.windows.net/public/llvm-builds/llvm-10dc3a8e-ubuntu-x64.tar.gz |
178 | | -tar zxvf llvm-10dc3a8e-ubuntu-x64.tar.gz |
179 | | -export LLVM_SYSPATH=${YOUR_LLVM_DOWNLOAD_DIR}/llvm-10dc3a8e-ubuntu-x64 |
180 | | -# For Triton 3.2 |
181 | | -wget https://oaitriton.blob.core.windows.net/public/llvm-builds/llvm-86b69c31-ubuntu-x64.tar.gz |
182 | | -tar zxvf llvm-86b69c31-ubuntu-x64.tar.gz |
183 | | -export LLVM_SYSPATH=${YOUR_LLVM_DOWNLOAD_DIR}/llvm-86b69c31-ubuntu-x64 |
184 | | -# For Triton 3.3 |
185 | | -wget https://oaitriton.blob.core.windows.net/public/llvm-builds/llvm-a66376b0-ubuntu-x64.tar.gz |
186 | | -tar zxvf llvm-a66376b0-ubuntu-x64.tar.gz |
187 | | -export LLVM_SYSPATH=${YOUR_LLVM_DOWNLOAD_DIR}/llvm-a66376b0-ubuntu-x64 |
188 | | -# For Triton 3.4 |
189 | | -wget https://oaitriton.blob.core.windows.net/public/llvm-builds/llvm-8957e64a-ubuntu-x64.tar.gz |
190 | | -tar zxvf llvm-8957e64a-ubuntu-x64.tar.gz |
191 | | -export LLVM_SYSPATH=${YOUR_LLVM_DOWNLOAD_DIR}/llvm-8957e64a-ubuntu-x64 |
192 | | -# For Triton 3.5 |
193 | | -wget https://oaitriton.blob.core.windows.net/public/llvm-builds/llvm-7d5de303-ubuntu-x64.tar.gz |
194 | | -tar zxvf llvm-7d5de303-ubuntu-x64.tar.gz |
195 | | -export LLVM_SYSPATH=${YOUR_LLVM_DOWNLOAD_DIR}/llvm-7d5de303-ubuntu-x64 |
196 | | -# |
197 | | -export LLVM_INCLUDE_DIRS=$LLVM_SYSPATH/include |
198 | | -export LLVM_LIBRARY_DIR=$LLVM_SYSPATH/lib |
199 | | -cd ${YOUR_CODE_DIR}/flagtree |
200 | | -cd python # For Triton 3.1, 3.2, 3.3, you need to enter the python directory to build |
201 | | -git checkout main # For Triton 3.1 |
202 | | -git checkout -b triton_v3.2.x # For Triton 3.2 |
203 | | -git checkout -b triton_v3.3.x # For Triton 3.3 |
204 | | -git checkout -b triton_v3.4.x # For Triton 3.4 |
205 | | -git checkout -b triton_v3.5.x # For Triton 3.5 |
206 | | -unset FLAGTREE_BACKEND |
207 | | -python3 -m pip install . --no-build-isolation -v |
208 | | -# If you need to build other backends afterward, you should clear LLVM-related environment variables |
209 | | -unset LLVM_SYSPATH LLVM_INCLUDE_DIRS LLVM_LIBRARY_DIR |
210 | | -``` |
211 | | - |
212 | | -## Offline Build Support: Pre-downloading Dependency Packages |
213 | | -The above introduced how dependencies can be manually downloaded for various FlagTree backends during build time to avoid network environment limitations. Since Triton builds originally come with some dependency packages, we provide pre-downloaded packages that can be manually installed in your environment to prevent getting stuck at the automatic download stage during the build process. |
214 | | -```shell |
215 | | -cd ${YOUR_CODE_DIR}/flagtree/python |
216 | | -sh README_offline_build.sh x86_64 # View readme |
217 | | -# For Triton 3.1 (x64) |
218 | | -wget https://baai-cp-web.ks3-cn-beijing.ksyuncs.com/trans/offline-build-pack-triton-3.1.x-linux-x64.zip |
219 | | -sh scripts/offline_build_unpack.sh ./offline-build-pack-triton-3.1.x-linux-x64.zip ~/.triton |
220 | | -# For Triton 3.2 (x64) |
221 | | -wget https://baai-cp-web.ks3-cn-beijing.ksyuncs.com/trans/offline-build-pack-triton-3.2.x-linux-x64.zip |
222 | | -sh scripts/offline_build_unpack.sh ./offline-build-pack-triton-3.2.x-linux-x64.zip ~/.triton |
223 | | -# For Triton 3.2 (aarch64) |
224 | | -wget https://baai-cp-web.ks3-cn-beijing.ksyuncs.com/trans/offline-build-pack-triton-3.2.x-linux-aarch64.zip |
225 | | -sh scripts/offline_build_unpack.sh ./offline-build-pack-triton-3.2.x-linux-aarch64.zip ~/.triton |
226 | | -# For Triton 3.3 (x64) |
227 | | -wget https://baai-cp-web.ks3-cn-beijing.ksyuncs.com/trans/offline-build-pack-triton-3.3.x-linux-x64.zip |
228 | | -sh scripts/offline_build_unpack.sh ./offline-build-pack-triton-3.3.x-linux-x64.zip ~/.triton |
229 | | -# After executing the above script, the original ~/.triton directory will be renamed, and a new ~/.triton directory will be created to store the pre-downloaded packages. |
230 | | -``` |
231 | | - |
232 | | -## Running tests |
233 | | - |
234 | | -After installation, you can run tests: |
235 | | -```shell |
236 | | -# nvidia |
237 | | -cd python/test |
238 | | -python3 -m pytest -s |
239 | | -# other backends |
240 | | -cd third_party/backendxxx/python/test |
241 | | -python3 -m pytest -s |
242 | | -python3 test_xxx.py |
243 | | -``` |
244 | | - |
245 | | -## Contributing |
246 | | - |
247 | | -Contributions to FlagTree development are welcome. Please refer to [CONTRIBUTING.md](/CONTRIBUTING_cn.md) for details. |
248 | | - |
249 | | -## License |
250 | | - |
251 | | -FlagTree is licensed under the [MIT license](/LICENSE). |
| 1 | +[README](https://github.com/flagos-ai/flagtree/blob/main/README.md) |
0 commit comments