CUDA Learning Kernels

This repository records my learning process of CUDA kernel development, from naive implementations to optimized versions, with performance analysis using Nsight Compute (ncu).

Goals

• Implement common CUDA kernels from scratch
• Track naive → optimized evolution
• Analyze performance bottlenecks with ncu
• Build reproducible benchmarks and correctness checks

Planned Kernels

• Element-wise (vector add)
• Transpose
• Reduce
• MatMul
• Softmax
• RMSNorm
• Flash Attention

Study Notes

• Experimental procedure：https://gcn4wojyc654.feishu.cn/docx/NZdKdUAd9ofYVsx4MfYcnIERnng?from=from_copylink
• PMPP Notes：https://gcn4wojyc654.feishu.cn/wiki/JKwFwjcvfi8wqbk1SiBcEUW9nfg?from=from_copylink
• ...