Automated CUDA kernel optimization and CI/CD deployment platform for LLM inference.
KernelFlow is a production-style MLOps platform built around fused CUDA kernels for LLM inference. A researcher pushes a fused kernel to GitHub — the system automatically builds, validates numerical correctness, benchmarks against an unfused baseline, and deploys it as a PyTorch Extension. No manual steps.
The pipeline mirrors NVIDIA's internal kernel development infrastructure (cuDNN / FlashInfer style): every kernel must clear a defined speedup gate and numerical error budget before it can be promoted to main and packaged.
Kernels implemented:
| Milestone | Kernel | Speedup Gate |
|---|---|---|
| 1 | Fused RMSNorm + RoPE | ≥ 1.5× vs baseline |
| 2 | Fused SiLU × Elementwise Multiply | ≥ 1.3× vs baseline |
| 3 | Fused Attention (Flash Attention simplified) | ≥ 2.0× vs baseline |
Apache 2.0 — consistent with NVIDIA open-source projects (CUTLASS, TensorRT-LLM, cuDF).