How to offload to Intel integrated GPU?

[pauljurczak]

I hope it's okay to cross-post this question here (https://numba.discourse.group/t/numba-mlir-gpu-offload/1981).

I watched the presentation announced in https://numba.discourse.group/t/numba-meeting-june-6-2023-presentation-on-numba-mlir/1959/2 and installed Numba-MLIR with:

conda install numba-mlir -c dppy/label/dev -c intel -c conda-forge -c numba

A simple example:

from numba_mlir import njit
import numpy as np

@njit (parallel=True)
def foo(a, b):
    return a + b

result = foo(np.array([1,2,3]), np.array([4,5,6]))
print(result)

runs fine. There is no documentation yet and I don't know how to control where this script is run: CPU vs. GPU. What is the way to offload it to an Intel integrated GPU?

[Hardcode84]

Yes, we are unfortunately lacking documentation for GPU offload.
Some code can be found in our tests https://github.com/numba/numba-mlir/blob/main/numba_mlir/numba_mlir/mlir/tests/test_gpu.py

You will need to install dpctl library first https://github.com/IntelPython/dpctl

conda install dpctl -c dppy/label/dev -c intel

And I've constructed a simple example:

from numba_mlir import njit
import numba
import numpy as np
import dpctl.tensor as dpt

@njit(parallel=True)
def foo(a, b):
    return a + b

a = np.array([[1,2,3],[4,5,6]])
b = np.array([[1,2,3]])

print(a + b)

a = dpt.asarray(a, device="level_zero:gpu:0")
b = dpt.asarray(b, device="level_zero:gpu:0")

result = foo(a, b)
print(result)

This should offload to GPU or fail if it isn't possible.

To be really sure it was offloaded, there is an env var NUMBA_MLIR_DUMP_IR=1 which will dump IR from various stages of compilation (few megabytes of code!, redirect to somewhere), you can grep it for spirv tokens.

[pauljurczak]

Thank you for prompt response. I tried this example and got:

[[2 4 6]
 [5 7 9]]
Traceback (most recent call last):
  File "/home/joshua/numba-mlir/m.py", line 15, in <module>
    a = dpt.asarray(a, device="level_zero:gpu:0")
  File "/home/joshua/numba-mlir/.conda/lib/python3.10/site-packages/dpctl/tensor/_ctors.py", line 574, in asarray
    sycl_queue = normalize_queue_device(
  File "/home/joshua/numba-mlir/.conda/lib/python3.10/site-packages/dpctl/tensor/_device.py", line 171, in normalize_queue_device
    d = Device.create_device(d)
  File "/home/joshua/numba-mlir/.conda/lib/python3.10/site-packages/dpctl/tensor/_device.py", line 78, in create_device
    _dev = dpctl.SyclDevice(dev)
  File "dpctl/_sycl_device.pyx", line 334, in dpctl._sycl_device.SyclDevice.__cinit__
dpctl._sycl_device.SyclDeviceCreationError: Could not create a SyclDevice with the selector string 'level_zero:gpu:0'

I'm running this test on Ubuntu 22.04.2 and Intel Celeron N5105 CPU, which has integrated UHD Graphics. I also have Intel GPU drivers (runtime and dev) installed following https://dgpu-docs.intel.com/installation-guides/ubuntu/ubuntu-focal.html.

[Hardcode84]

Can you run sycl-ls from this environment? Also what the conda list output?

[pauljurczak]

I don't have sycl-ls and not sure where to get it from. apt can't find it. My Conda environment:

# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main  
_openmp_mutex             5.1                       1_gnu  
bzip2                     1.0.8                h7b6447c_0  
ca-certificates           2023.01.10           h06a4308_0    intel
dpcpp-cpp-rt              2023.1.0            intel_46305    intel
dpcpp_cpp_rt              2023.1.0            intel_46305    intel
dpctl                     0.14.3dev1      py310h76be34b_124    dppy/label/dev
icc_rt                    2023.1.0            intel_46305    intel
intel-cmplr-lib-rt        2023.1.0            intel_46305    intel
intel-cmplr-lic-rt        2023.1.0            intel_46305    intel
intel-opencl-rt           2023.1.0            intel_46305    intel
intel-openmp              2023.1.0            intel_46305    intel
intelpython               2023.1.0                      1    intel
ld_impl_linux-64          2.38                 h1181459_1  
libffi                    3.4.4                h6a678d5_0  
libgcc-ng                 11.2.0               h1234567_1  
libgomp                   11.2.0               h1234567_1  
libstdcxx-ng              11.2.0               h1234567_1  
libuuid                   1.41.5               h5eee18b_0  
llvmlite                  0.40.0rc1       py310he1b5a44_0    numba
mkl                       2023.1.0            intel_46342    intel
mkl-dpcpp                 2023.1.0            intel_46342    intel
mkl-service               2.4.0           py310h75d02e3_15    intel
mkl_fft                   1.3.5           py310h28f0b46_11    intel
mkl_random                1.2.2           py310h0b06908_51    intel
mkl_umath                 0.1.1           py310h450dca2_61    intel
ncurses                   6.4                  h6a678d5_0  
numba                     0.57.0          np1.21py3.10ha54aa7d_g4fd4e39c6_0    numba
numba-mlir                0.1.0dev0       py310h2bc3f7f_87    dppy/label/dev
numpy                     1.23.5          py310h52df89b_7    intel
numpy-base                1.23.5          py310ha03f565_7    intel
openssl                   1.1.1t               h7f8727e_0    intel
packaging                 23.0            py310h06a4308_0    intel
pip                       23.0.1          py310h06a4308_0  
python                    3.10.11              h7a1cb2a_2  
readline                  8.2                  h5eee18b_0  
setuptools                67.8.0          py310h06a4308_0  
six                       1.16.0             pyhd3eb1b0_1    intel
sqlite                    3.41.2               h5eee18b_0  
tbb                       2021.9.0            intel_43484    intel
tbb4py                    2021.9.0        py310_intel_43484    intel
tk                        8.6.12               h1ccaba5_0  
tzdata                    2023c                h04d1e81_0  
wheel                     0.38.4          py310h06a4308_0  
xz                        5.4.2                h5eee18b_0  
zlib                      1.2.13               h5eee18b_0  

[Hardcode84]

I don't have sycl-ls and not sure where to get it from.

try conda install dpcpp_linux-64 -c intel

[pauljurczak]

I've got:

(/home/joshua/numba-mlir/.conda) joshua@qvin:~/numba-mlir$ sycl-ls
[opencl:acc:0] Intel(R) FPGA Emulation Platform for OpenCL(TM), Intel(R) FPGA Emulation Device 1.2 [2023.15.3.0.20_160000]
[opencl:gpu:1] Intel(R) OpenCL HD Graphics, Intel(R) UHD Graphics [0x4e61] 3.0 [22.28.23726.1]
[opencl:cpu:2] Intel(R) OpenCL, Intel(R) Celeron(R) N5105 @ 2.00GHz 3.0 [2023.15.3.0.20_160000]

[Hardcode84]

Ok, you don't have level_zero device (some missing driver?) but still have opencl, try device="opencl:gpu:0" or device="opencl:gpu:1"

[pauljurczak]

Thanks. device="opencl:gpu:0" runs without error, but trying to get the IR fails:

(/home/joshua/numba-mlir/.conda) joshua@qvin:~/numba-mlir$ echo $NUMBA_MLIR_DUMP_IR
1
(/home/joshua/numba-mlir/.conda) joshua@qvin:~/numba-mlir$ python3 -m m.py
[[2 4 6]
 [5 7 9]]
Segmentation fault (core dumped)

IR is listed fine with the CPU example, though.

[Hardcode84]

can you add some prints around dpt.asarray calls? I have a feeling it didn't even get to the actual function.

[pauljurczak]

You are right. I have:

print(a + b)
print('GPU part 1')
a = dpt.asarray(a, device="opencl:gpu:0")
print('GPU part 2')
b = dpt.asarray(b, device="opencl:gpu:0")
print('GPU part 3')

and it got only to:

GPU part 1

[pauljurczak]

I'm not 100% sure if N5105 satisfies minimum hardware requirements. Intel® oneAPI DPC++/C++ Compiler System Requirements are Intel® UHD Graphics for 11th generation Intel processors or newer. My UHD Graphics seems to be Gen. 11, but the CPU is Gen. 10, I think.

[diptorupd]

@pauljurczak to get the level zero drivers you can do conda install -c conda-forge level-zero or if you prefer sudo apt-get install intel-level-zero-gpu level-zero. After that can you post the output for the command:
python -c "import dpctl; dpctl.lsplatform(verbosity=2)"

[pauljurczak]

Here is the output:

(/home/joshua/numba-mlir/.conda) joshua@qvin:~/numba-mlir$ python -c "import dpctl; dpctl.lsplatform(verbosity=2)"
Platform  0 ::
    Name        Intel(R) FPGA Emulation Platform for OpenCL(TM)
    Version     OpenCL 1.2 Intel(R) FPGA SDK for OpenCL(TM), Version 20.3
    Vendor      Intel(R) Corporation
    Backend     opencl
    Num Devices 1
      # 0
        Name                Intel(R) FPGA Emulation Device
        Version             2023.15.3.0.20_160000
        Filter string       opencl:accelerator:0
Platform  1 ::
    Name        Intel(R) OpenCL HD Graphics
    Version     OpenCL 3.0 
    Vendor      Intel(R) Corporation
    Backend     opencl
    Num Devices 1
      # 0
        Name                Intel(R) UHD Graphics [0x4e61]
        Version             22.28.23726.1
        Filter string       opencl:gpu:0
Platform  2 ::
    Name        Intel(R) OpenCL
    Version     OpenCL 3.0 LINUX
    Vendor      Intel(R) Corporation
    Backend     opencl
    Num Devices 1
      # 0
        Name                Intel(R) Celeron(R) N5105 @ 2.00GHz
        Version             2023.15.3.0.20_160000
        Filter string       opencl:cpu:0

[diptorupd]

Here is the output:

Is this after installing the level-zero drivers?

[pauljurczak]

Yes.

(/home/joshua/numba-mlir/.conda) joshua@qvin:~/numba-mlir$ conda list
# packages in environment at /home/joshua/numba-mlir/.conda:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                  2_kmp_llvm    conda-forge
bzip2                     1.0.8                h7b6447c_0  
ca-certificates           2023.5.7             hbcca054_0    conda-forge
dpcpp-cpp-rt              2023.1.0            intel_46305    intel
dpcpp_cpp_rt              2023.1.0            intel_46305    intel
dpcpp_impl_linux-64       2023.1.0            intel_46305    intel
dpcpp_linux-64            2023.1.0            intel_46305    intel
dpctl                     0.14.3dev1      py310h76be34b_124    dppy/label/dev
icc_rt                    2023.1.0            intel_46305    intel
intel-cmplr-lib-rt        2023.1.0            intel_46305    intel
intel-cmplr-lic-rt        2023.1.0            intel_46305    intel
intel-opencl-rt           2023.1.0            intel_46305    intel
intel-openmp              2023.1.0            intel_46305    intel
intelpython               2023.1.0                      1    intel
ld_impl_linux-64          2.38                 h1181459_1  
level-zero                1.11.0               h00ab1b0_0    conda-forge
libffi                    3.4.4                h6a678d5_0  
libgcc-ng                 13.1.0               he5830b7_0    conda-forge
libstdcxx-ng              13.1.0               hfd8a6a1_0    conda-forge
libuuid                   1.41.5               h5eee18b_0  
llvm-openmp               14.0.6               h9e868ea_0  
llvmlite                  0.40.0rc1       py310he1b5a44_0    numba
mkl                       2023.1.0            intel_46342    intel
mkl-dpcpp                 2023.1.0            intel_46342    intel
mkl-service               2.4.0           py310h75d02e3_15    intel
mkl_fft                   1.3.5           py310h28f0b46_11    intel
mkl_random                1.2.2           py310h0b06908_51    intel
mkl_umath                 0.1.1           py310h450dca2_61    intel
ncurses                   6.4                  h6a678d5_0  
numba                     0.57.0          np1.21py3.10ha54aa7d_g4fd4e39c6_0    numba
numba-mlir                0.1.0dev0       py310h2bc3f7f_87    dppy/label/dev
numpy                     1.23.5          py310h52df89b_7    intel
numpy-base                1.23.5          py310ha03f565_7    intel
openssl                   1.1.1u               hd590300_0    conda-forge
packaging                 23.0            py310h06a4308_0    intel
pip                       23.0.1          py310h06a4308_0  
python                    3.10.11              h7a1cb2a_2  
readline                  8.2                  h5eee18b_0  
setuptools                67.8.0          py310h06a4308_0  
six                       1.16.0             pyhd3eb1b0_1    intel
sqlite                    3.41.2               h5eee18b_0  
tbb                       2021.9.0            intel_43484    intel
tbb4py                    2021.9.0        py310_intel_43484    intel
tk                        8.6.12               h1ccaba5_0  
tzdata                    2023c                h04d1e81_0  
wheel                     0.38.4          py310h06a4308_0  
xz                        5.4.2                h5eee18b_0  
zlib                      1.2.13               h5eee18b_0