Fused kernels and removal of torch cuda synchronize #105

Merged

mseeger merged 8 commits into

awslabs:mainfrom

vihangp:async_offload

May 7, 2026

Commits on Apr 28, 2026

Add fused Triton RoPE kernel
VihangPatil
committed

Commits on Apr 29, 2026

Add fused Triton RMSNorm kernel
VihangPatil
committed
Add fused Triton SwiGLU kernel
VihangPatil
committed
Skip RoPE outer cat when rope covers the full head dim
VihangPatil
committed
Add NVTX ranges for gradient accumulation phases
VihangPatil
committed
Batch create_random_index randperm calls
VihangPatil
committed

Commits on Apr 30, 2026

Drop per-layer sync in layer-input checkpoint loop
VihangPatil
committed

Commits on May 7, 2026

Merge branch 'main' into async_offload
mseeger
authored