Fused kernels cut peak LLM memory by 84%
The cross-entropy layer is where LLM training tends to run out of VRAM. This post walks through fusing the linear projection and the loss into one Triton kernel, why tiling and online softmax cut peak memory by 84%, and where my first attempt went wrong.