Enterprise AI on-prem: running GPUaaS on Kubernetes
Notes from building a private GPU-as-a-Service platform on-prem: partitioning GPUs with MIG and time-slicing, keeping the cluster in sync with an idempotent reconciler, and working out the cost per million tokens before finance asks.