Disaggregated LLM inference and the GPU you are wasting
Prefill is compute-bound, decode is memory-bound, and running both on one GPU wastes most of it. Splitting them into separate pools, the KV-cache transfer tax that comes with it, and when it pays off.