Inference Has Overtaken Training: What KubeCon Japan Revealed About Kubernetes Cluster Design in the AI Era
AI compute demand has flipped from training to inference, with inference compute projected to reach 1.5x training capacity by 2030. Drawing on KubeCon Japan discussions and the CNCF AI Conformance Program, this article outlines what Kubernetes/K3s clusters need to look like in the inference era.