Add hip-kernel

[smedegaard]

The Idea

Merge a hip-kernel into main to support AMD GPUs

What is HIP?

The Heterogeneous-computing Interface for Portability (HIP) API is a C++ runtime API and kernel language that lets developers create portable applications running in heterogeneous systems, using CPUs and AMD GPUs or NVIDIA GPUs from a single source code. HIP provides a simple marshalling language to access either the AMD ROCM back-end, or NVIDIA CUDA back-end, to build and run application kernels.

Previous Work

@vberthet forked Candle and made a rocm-kernel (HIP is a part of ROCm) in January of 2024. It seemed promising but may have been abandoned since.

The topic has been discussed in #346 but there has been little activity the last few months

What Now?

I've started implementing a Rust wrapper for the HIP C++ lib, called hip_rs. It's very bare bones now. But it's got the basic functions and can call the BLAS lib safely.

Call for collaborators

I'd like to start implementing the hip-kernel for Candle using hip_rs in parallel with expanding hip_rs, but would love some advice and guidance, and hopefully some collaborators to do so.

[smedegaard]

@EricLBuehler , do you have any advise on what I should consider when starting implementation of hip-kernel ?