Nvidia’s TiDAR and the LLM inference bottleneck
GPUs sit idle during text generation because model weights move slower than the math runs. Nvidia’s TiDAR drafts tokens with diffusion and verifies them in one parallel pass, reaching up to 5.91x speedup on an 8B parameter model.