[{"data":1,"prerenderedAt":270},["ShallowReactive",2],{"blog-en-kubernetes-ai-inference-reversal-conformance-design":3,"blog-related-en-kubernetes-ai-inference-reversal-conformance-design":220,"blog-en-kubernetes-ai-inference-reversal-conformance-design-alt":209},{"id":4,"title":5,"author":6,"body":7,"date":203,"description":204,"extension":205,"image":206,"locale":207,"meta":208,"navigation":209,"path":210,"seo":211,"stem":212,"tags":213,"__hash__":219},"blog\u002Fblog\u002Fen\u002Fkubernetes-ai-inference-reversal-conformance-design.md","Inference Has Overtaken Training: What KubeCon Japan Revealed About Kubernetes Cluster Design in the AI Era","Kubo Team",{"type":8,"value":9,"toc":194},"minimark",[10,15,19,26,37,50,59,62,66,69,75,84,93,96,100,108,114,123,138,141,145,148,154,161,174,178,181],[11,12,14],"h2",{"id":13},"ai-compute-demand-was-built-around-training-thats-quietly-reversing","AI Compute Demand Was Built Around Training. That's Quietly Reversing.",[16,17,18],"p",{},"Chances are you've noticed the phrase \"Kubernetes AI inference\" showing up more often over the past year. At the keynote of KubeCon + CloudNativeCon Japan 2026, CNCF leadership spoke about a shift in how AI computing capacity is being allocated. Until a few years ago, the vast majority of AI-related compute went toward model \"training.\" Today, the center of gravity is moving to \"inference.\"",[16,20,21],{},[22,23],"img",{"alt":24,"src":25},"section01","https:\u002F\u002Fcdn.kubo.hexabase.io\u002Fimages\u002Fblog\u002Fkubernetes-ai-inference-reversal-conformance-design\u002Fsection01.webp",[16,27,28,29,36],{},"According to ",[30,31,35],"a",{"href":32,"rel":33},"https:\u002F\u002Fbuild.inc\u002Finsights\u002Ftraining-vs-inference-data-center-design-differences",[34],"nofollow","an analysis of data center demand",", AI inference compute capacity is projected to exceed 90 gigawatts by 2030, growing at a 35% CAGR. McKinsey's own estimates are even more specific: data center demand for AI training is expected to grow from 23.1 gigawatts in 2025 to 62.2 gigawatts by 2030 (22% CAGR), while demand for inference is expected to surge from 20.9 gigawatts to 93.3 gigawatts over the same period (35% CAGR). By 2030, inference is projected to account for more than half of all AI compute, overtaking training as the single largest workload category.",[16,38,39,40,45,46],{},"Other research points in the same direction. ",[30,41,44],{"href":42,"rel":43},"https:\u002F\u002Favidsolutionsinc.com\u002F13-data-center-growth-projections-that-will-shape-2026-2030\u002F",[34],"An article on data center growth trends"," cites Deloitte's estimate that inference already accounted for half of all AI compute in 2025 and will reach two-thirds by 2026, while Brookfield projects inference will make up 75% of AI compute demand by 2030. The exact numbers vary, but the direction is consistent: ",[47,48,49],"strong",{},"infrastructure designed around training no longer matches reality.",[16,51,52,53,58],{},"Given the scale of this shift, cost efficiency and freedom from vendor lock-in are becoming critical decision criteria for any organization choosing or re-evaluating its infrastructure. It's part of why managed K3s-based Kubernetes platforms like ",[30,54,57],{"href":55,"rel":56},"https:\u002F\u002Fkubo.hexabase.io\u002F",[34],"Kubo"," are increasingly being considered.",[16,60,61],{},"What makes this shift tricky is that training and inference demand fundamentally different things from infrastructure. The next section looks at how that difference is reshaping Kubernetes operations.",[11,63,65],{"id":64},"why-inference-workloads-are-forcing-kubernetes-to-rebuild-itself","Why Inference Workloads Are Forcing Kubernetes to Rebuild Itself",[16,67,68],{},"Training jobs are essentially batch processes: they consume a large pool of GPUs for hours or days at a stretch, then release them. Inference workloads, on the other hand, run continuously, scale up and down within seconds or minutes based on request volume, and translate any latency degradation directly into a worse user experience. Scheduling and resource-management approaches optimized for training tend to waste resources when applied directly to inference.",[16,70,71],{},[22,72],{"alt":73,"src":74},"section02","https:\u002F\u002Fcdn.kubo.hexabase.io\u002Fimages\u002Fblog\u002Fkubernetes-ai-inference-reversal-conformance-design\u002Fsection02.webp",[16,76,77,78,83],{},"Kubernetes itself is adding inference-oriented capabilities to address this. A prime example is \"In-Place Pod Resize.\" According to the ",[30,79,82],{"href":80,"rel":81},"https:\u002F\u002Fkubernetes.io\u002Fblog\u002F2025\u002F05\u002F16\u002Fkubernetes-v1-33-in-place-pod-resize-beta\u002F",[34],"official Kubernetes blog",", this feature lets you change a container's CPU and memory requests\u002Flimits without restarting the Pod, and it graduated to beta with default-enabled status in v1.33. Inference serving often needs a burst of CPU right after startup and then settles into a lower steady state — being able to adjust resources without a restart has a direct impact on availability.",[16,85,86,87,92],{},"The ",[30,88,91],{"href":89,"rel":90},"https:\u002F\u002Fopensource.googleblog.com\u002F2026\u002F04\u002Fkubernetes-goes-ai-first-unpacking-the-new-ai-conformance-program.html",[34],"Google Cloud blog"," further highlights four technical building blocks for supporting both inference and training: \"Dynamic Resource Allocation\" (DRA) for fine-grained control over GPUs\u002FTPUs, \"All-or-Nothing Scheduling\" so distributed training jobs wait until all required resources are available, custom-metrics-based autoscaling driven by signals like GPU utilization, and standardized observability for accelerators. Together, these show Kubernetes expanding its role from \"container orchestrator\" to \"AI workload platform.\"",[16,94,95],{},"If every cloud and distribution implements these extensions independently, the result is a patchwork of environments where things only \"just work\" in specific places. That's exactly the fragmentation problem CNCF's standardization effort is trying to solve.",[11,97,99],{"id":98},"what-is-the-cncfs-ai-conformance-program-actually-guaranteeing","What Is the CNCF's \"AI Conformance Program\" Actually Guaranteeing?",[16,101,28,102,107],{},[30,103,106],{"href":104,"rel":105},"https:\u002F\u002Fwww.cncf.io\u002Fannouncements\u002F2025\u002F11\u002F11\u002Fcncf-launches-certified-kubernetes-ai-conformance-program-to-standardize-ai-workloads-on-kubernetes\u002F",[34],"CNCF's official announcement",", the Certified Kubernetes AI Conformance Program was first announced in beta at KubeCon Japan in June 2025 and officially launched at KubeCon North America in Atlanta on November 11 of the same year. The goal is straightforward: ensure interoperability and portability for AI workloads and reduce fragmentation across vendors. Major cloud providers — AWS, Google Cloud, Microsoft Azure, Oracle, Red Hat, and others — have already joined as certified platforms.",[16,109,110],{},[22,111],{"alt":112,"src":113},"section03","https:\u002F\u002Fcdn.kubo.hexabase.io\u002Fimages\u002Fblog\u002Fkubernetes-ai-inference-reversal-conformance-design\u002Fsection03.webp",[16,115,116,117,122],{},"A ",[30,118,121],{"href":119,"rel":120},"https:\u002F\u002Fwww.forbes.com\u002Fsites\u002Fjanakirammsv\u002F2025\u002F11\u002F18\u002Fcncf-establishes-standards-for-running-ai-workloads-on-kubernetes\u002F",[34],"Forbes explainer"," notes that the program emerged in response to research showing that 82% of organizations are building custom AI solutions, with 58% of them relying on Kubernetes — and that incompatibility between platforms was raising the risk of vendor lock-in. With certification in place, an AI application validated on one Kubernetes environment can be expected to behave the same way on any other certified environment.",[16,124,125,126,131,132,137],{},"The program's requirements are also becoming stricter over time. According to ",[30,127,130],{"href":128,"rel":129},"https:\u002F\u002Fcloudnativenow.com\u002Ffeatures\u002Fcncf-expands-efforts-to-run-ai-inference-workloads-on-kubernetes-clusters\u002F",[34],"reporting from Cloud Native Now",", the 2026 KubeCon Europe event introduced the \"Kubernetes AI Requirements (KAR) v1.35\" certification standard, adding stable support for In-Place Pod Resize and Workload-Aware Scheduling as mandatory requirements. At the same time, new platforms — including OVHcloud, SpectroCloud, JD Cloud, and China Unicom Cloud — achieved certification, bringing the total number of certified platforms to 31. CNCF's Jonathan Bryce has said that \"AI inference workloads will, in the near future, come to overwhelm every other class of workload running on Kubernetes clusters,\" and the detailed requirements are being developed in the open on ",[30,133,136],{"href":134,"rel":135},"https:\u002F\u002Fgithub.com\u002Fcncf\u002Fk8s-ai-conformance",[34],"GitHub",".",[16,139,140],{},"In short, what CNCF is aiming for is a state where \"even as inference workloads surge, you get consistent behavior regardless of environment.\" That same principle turns out to be a useful lens for choosing a managed Kubernetes platform.",[11,142,144],{"id":143},"what-should-managed-k3s-environments-do-to-meet-inference-era-infrastructure-requirements","What Should Managed K3s Environments Do to Meet Inference-Era Infrastructure Requirements?",[16,146,147],{},"Pulling together everything covered so far, the requirements for inference-era infrastructure boil down to three things. First, autoscaling based on AI-specific signals like GPU utilization and inference latency. Second, observability that surfaces accelerator metrics in a standardized way. Third, portability that isn't tied to a specific cloud or distribution — in other words, avoiding vendor lock-in.",[16,149,150],{},[22,151],{"alt":152,"src":153},"section04","https:\u002F\u002Fcdn.kubo.hexabase.io\u002Fimages\u002Fblog\u002Fkubernetes-ai-inference-reversal-conformance-design\u002Fsection04.webp",[16,155,156,157,160],{},"These three conditions are exactly where using a fully managed service like EKS or AKS on its own tends to force a trade-off between cost and lock-in. ",[30,158,57],{"href":55,"rel":159},[34],", a K3s-based managed Kubernetes platform, is built on standards-compliant Pure Kubernetes, which aligns naturally with the environment-level consistency CNCF's certification is aiming for. Its built-in Prometheus + Grafana stack also gives you a ready-made foundation for visualizing accelerator metrics.",[16,162,163,164,167,168,173],{},"Much of today's inference workload runs as part of AI agents or MLOps pipelines. For teams thinking ahead to operating these autonomous workloads, combining ",[30,165,57],{"href":55,"rel":166},[34]," with an AI agent execution platform like ",[30,169,172],{"href":170,"rel":171},"https:\u002F\u002Fwww.hexabase.com\u002Fproduct\u002Fcaptain-ai\u002F",[34],"Captain.AI"," is one option worth considering. Built-in GitOps and Helm chart support also means you don't need to rebuild your inference deployment pipeline from scratch.",[11,175,177],{"id":176},"summary","Summary",[16,179,180],{},"AI compute demand is flipping from \"training-centric\" to \"inference-centric,\" and multiple research organizations now project that inference compute capacity will substantially outpace training by 2030. Kubernetes is responding by adopting inference-specific capabilities like In-Place Pod Resize and Dynamic Resource Allocation, while CNCF's AI Conformance Program pushes toward a standard where workloads \"behave the same regardless of environment.\"",[16,182,183,184,187,188,193],{},"Given this trajectory, cluster design going forward needs to prioritize three things: autoscaling based on inference-specific metrics, observability, and portability that doesn't lock you into a single vendor. For organizations without the resources to build robust infrastructure from the ground up, ",[30,185,57],{"href":55,"rel":186},[34]," — a standards-compliant, cost-efficient, K3s-based platform — is a realistic option for supporting inference-era workloads. Start by ",[30,189,192],{"href":190,"rel":191},"https:\u002F\u002Fwww.hexabase.com\u002Fcontact-us\u002F",[34],"getting in touch"," to map out where your inference infrastructure stands today.",{"title":195,"searchDepth":196,"depth":196,"links":197},"",2,[198,199,200,201,202],{"id":13,"depth":196,"text":14},{"id":64,"depth":196,"text":65},{"id":98,"depth":196,"text":99},{"id":143,"depth":196,"text":144},{"id":176,"depth":196,"text":177},"2026-08-01","AI compute demand has flipped from training to inference, with inference compute projected to reach 1.5x training capacity by 2030. Drawing on KubeCon Japan discussions and the CNCF AI Conformance Program, this article outlines what Kubernetes\u002FK3s clusters need to look like in the inference era.","md","https:\u002F\u002Fcdn.kubo.hexabase.io\u002Fimages\u002Fblog\u002Fkubernetes-ai-inference-reversal-conformance-design\u002Feyecatch.webp","en",{},true,"\u002Fblog\u002Fen\u002Fkubernetes-ai-inference-reversal-conformance-design",{"title":5,"description":204},"blog\u002Fen\u002Fkubernetes-ai-inference-reversal-conformance-design",[214,215,216,217,218],"kubernetes","k3s","ai-inference","cncf","managed-kubernetes","tZ5CR5UTL375EEleusMI-aUDIbYt92BEtR9YU8Ec6p0",[221,229,237,245,254,262],{"path":222,"title":223,"description":224,"date":225,"tags":226},"\u002Fblog\u002Fen\u002Fcncf-graduated-project-oss-selection-criteria","Should You Trust the CNCF 'Graduated' Badge? What TOC Public Meetings Reveal About the Unglamorous Reality of the Review Process","The CNCF 'Graduated' badge is not a safety certificate. Drawing on TOC public meeting notes and official documentation, this article explains the real review process behind Sandbox, Incubating, and Graduated status, and how to judge OSS maturity before adopting it in production.","2026-07-23",[217,214,215,227,218,228],"oss","governance",{"path":230,"title":231,"description":232,"date":233,"tags":234},"\u002Fblog\u002Fen\u002Fkubevirt-calico-live-migration-networking","Moving a VM Doesn't Have to Break the Connection: Inside KubeVirt and Calico's Live Migration Magic","Why doesn't live migrating a VM (KubeVirt) between Kubernetes nodes break the connection? We break down Calico's IP persistence and BGP route convergence, and what it means for teams moving off VMware.","2026-08-07",[215,214,235,236,218],"kubevirt","networking",{"path":238,"title":239,"description":240,"date":241,"tags":242},"\u002Fblog\u002Fen\u002Fai-generated-kubernetes-manifest-resource-overprovisioning","Kubernetes Resource Design Can't Be Left to AI: Why 'Working' YAML Is Wasting 69% of Your Cloud Bill","AI-generated Kubernetes manifests pass kubectl apply and 'work' — but getting Kubernetes resource design wrong drives massive overprovisioning. Here's why AI struggles with production-grade requests\u002Flimits and what to check before you ship.","2026-08-04",[215,214,243,244,218],"resource-management","capacity-planning",{"path":246,"title":247,"description":248,"date":249,"tags":250},"\u002Fblog\u002Fen\u002Fkubernetes-microservices-chatty-calls-latency","One Order, Five Hidden Service Calls: The Real Cause of Latency in Kubernetes Microservices' \"Chatty Calls\"","A single checkout request was quietly triggering five separate service calls behind the scenes. The culprit isn't bad code — it's the \"chatty call\" architecture that Kubernetes microservices tend to fall into. This article explains the distributed N+1 problem and how to fix it.","2026-08-02",[214,215,251,252,253,218],"microservices","service-mesh","latency",{"path":255,"title":256,"description":257,"date":258,"tags":259},"\u002Fblog\u002Fen\u002Fhybrid-k3s-edge-metrics-network-overhead","2,000 IoT Devices Were Clogging the Network. The Day Push Metrics Bit Back in a Hybrid K3s Deployment","A field report from a large-scale K3s edge deployment covering 2,000+ devices: why lightweight Kubernetes gets chosen, and the hidden network cost of push-based metrics collection in a hybrid cluster architecture, backed by concrete numbers. For engineers and platform operators.","2026-07-31",[215,214,260,261,218],"edge-computing","hybrid-cluster",{"path":263,"title":264,"description":265,"date":266,"tags":267},"\u002Fblog\u002Fen\u002Fedge-k3s-observability-homelab-dashboard","A Wall-Mounted Dashboard Taught Me What 'Peace of Mind' Really Means: Observability Design Lessons for Edge K3s Clusters","The trial-and-error troubleshooting behind a home-lab wall-mounted dashboard is the same trap that hits edge Kubernetes clusters scattered across factories and stores. Drawing on official K3s, Prometheus, and Grafana docs, this piece lays out design principles for not deferring observability.","2026-07-26",[215,214,260,268,269,218],"observability","monitoring",1786701442467]