Fixing AI/ML data transfer bottlenecks in PyTorch
GPU starvation, not GPU speed, is what slows most training runs. How I find AI/ML data transfer bottlenecks in an nsys trace, then fix them with num_workers, pinned memory and a CUDA stream prefetcher.