[{"data":1,"prerenderedAt":504},["ShallowReactive",2],{"blog-en-k3s-hpa-containerresource-sidecar-utilization-dilution":3,"blog-related-en-k3s-hpa-containerresource-sidecar-utilization-dilution":459,"blog-en-k3s-hpa-containerresource-sidecar-utilization-dilution-alt":447},{"id":4,"title":5,"author":6,"body":7,"date":441,"description":442,"extension":443,"image":444,"locale":445,"meta":446,"navigation":447,"path":448,"seo":449,"stem":450,"tags":451,"__hash__":458},"blog\u002Fblog\u002Fen\u002Fk3s-hpa-containerresource-sidecar-utilization-dilution.md","HPA Was Scaling on the Sidecar's Share Too. The ContainerResource Blind Spot in Kubernetes","Kubo Team",{"type":8,"value":9,"toc":434},"minimark",[10,15,23,31,47,63,67,73,82,91,100,104,110,127,139,351,363,372,376,382,391,403,410,414,417,430],[11,12,14],"h2",{"id":13},"_1-hpas-utilization-includes-the-sidecars-share-too","1. HPA's \"Utilization\" Includes the Sidecar's Share Too",[16,17,18],"p",{},[19,20],"img",{"alt":21,"src":22},"section01","https:\u002F\u002Fcdn.kubo.hexabase.io\u002Fimages\u002Fblog\u002Fen\u002Fk3s-hpa-containerresource-sidecar-utilization-dilution\u002Fsection01.webp",[16,24,25,26,30],{},"Have you ever run Kubernetes HPA (HorizontalPodAutoscaler) and noticed it won't scale even though CPU usage should be over the threshold — or that it scales out earlier than you expected? More often than not, the cause is a core part of the spec itself: HPA's standard Resource metric ",[27,28,29],"strong",{},"sums the usage of every container in the Pod",".",[16,32,33,34,38,39,46],{},"The standard Resource metric calculates utilization as ",[35,36,37],"code",{},"sum(usage of all containers) \u002F sum(requests of all containers)"," (",[40,41,45],"a",{"href":42,"rel":43},"https:\u002F\u002Fkubernetes.io\u002Fdocs\u002Ftasks\u002Frun-application\u002Fhorizontal-pod-autoscale\u002F",[44],"nofollow","Kubernetes official documentation","). That's fine when a Pod runs a single application container, but now that multi-container Pods — with a service mesh sidecar proxy or a log-shipping agent riding along — are the norm, this aggregation can skew the decision.",[16,48,49,50,53,54,38,57,62],{},"Here's a concrete example. Say an API container requests 500m CPU and uses 350m, while a log agent requests 200m and uses only 20m. The Pod-wide utilization comes out to 370m ÷ 700m ≈ ",[27,51,52],{},"52.9%",", but the application container alone is at 350m ÷ 500m = ",[27,55,56],{},"70%",[40,58,61],{"href":59,"rel":60},"https:\u002F\u002Foneuptime.com\u002Fblog\u002Fpost\u002F2026-08-01-sidecar-requests-scheduling-hpa-cost\u002Fview",[44],"oneuptime.com's analysis","). Under the exact same load, the scaling decision changes entirely depending on what you measure.",[11,64,66],{"id":65},"_2-why-its-easy-to-miss-the-invisible-roommate-called-a-service-mesh","2. Why It's Easy to Miss — The \"Invisible Roommate\" Called a Service Mesh",[16,68,69],{},[19,70],{"alt":71,"src":72},"section02","https:\u002F\u002Fcdn.kubo.hexabase.io\u002Fimages\u002Fblog\u002Fen\u002Fk3s-hpa-containerresource-sidecar-utilization-dilution\u002Fsection02.webp",[16,74,75,76,81],{},"This problem tends to surface right after adopting a service mesh like Istio. According to Istio's official performance documentation, under typical load a single sidecar proxy consumes about 0.2 vCPU and 60MB of memory (",[40,77,80],{"href":78,"rel":79},"https:\u002F\u002Fistio.io\u002Flatest\u002Fdocs\u002Fops\u002Fdeployment\u002Fperformance-and-scalability\u002F",[44],"Istio official documentation","). That looks small on its own, but once it's folded into HPA's CPU utilization calculation, it obscures the application container's real load.",[16,83,84,85,90],{},"Google Cloud's Cloud Service Mesh operations guide likewise recommends explicitly defining resource requests and limits for sidecar proxies, precisely because their resource consumption affects HPA's calculation (",[40,86,89],{"href":87,"rel":88},"https:\u002F\u002Fdocs.cloud.google.com\u002Fservice-mesh\u002Fv1.26\u002Fdocs\u002Foperate-and-maintain\u002Fscalability-best-practices",[44],"Google Cloud official documentation","). Yet many teams don't notice the dilution effect until after they've rolled out a service mesh and start wondering, \"why did HPA's behavior change?\" The more sidecar-pattern components you add — logging agents, security agents, and so on — the bigger this dilution effect gets.",[16,92,93,94,99],{},"Managed, K3s-based Kubernetes offerings like ",[40,95,98],{"href":96,"rel":97},"https:\u002F\u002Fkubo.hexabase.io\u002F",[44],"Kubo"," are frequently run with a service mesh or a logging stack on top of a built-in Prometheus + Grafana setup. Understanding the standard HPA spec correctly is unavoidable for production operations on stacks like this.",[11,101,103],{"id":102},"_3-letting-containerresource-metrics-focus-on-just-the-one-container","3. Letting ContainerResource Metrics Focus on \"Just the One Container\"",[16,105,106],{},[19,107],{"alt":108,"src":109},"section03","https:\u002F\u002Fcdn.kubo.hexabase.io\u002Fimages\u002Fblog\u002Fen\u002Fk3s-hpa-containerresource-sidecar-utilization-dilution\u002Fsection03.webp",[16,111,112,113,116,117,120,121,126],{},"The official fix for this dilution problem is the ",[35,114,115],{},"ContainerResource"," metric type. This Kubernetes feature lets HPA base its scaling decision on the resource usage of ",[27,118,119],{},"one specific container",", rather than the whole Pod (",[40,122,125],{"href":123,"rel":124},"https:\u002F\u002Fwww.kubernetes.dev\u002Fresources\u002Fkeps\u002F1610\u002F",[44],"KEP-1610: Container Resource based Autoscaling",").",[16,128,129,130,38,133,138],{},"The KEP's own motivation section spells out exactly this scenario: \"sidecar containers consume resources that don't correlate with the main application's load.\" The feature was introduced as alpha in Kubernetes 1.20, moved to beta in 1.27, and reached ",[27,131,132],{},"Stable (GA) in 1.30",[40,134,137],{"href":135,"rel":136},"https:\u002F\u002Fkubernetes.io\u002Fblog\u002F2023\u002F05\u002F02\u002Fhpa-container-resource-metric\u002F",[44],"Kubernetes official blog","). Configuring it is as simple as naming the target container explicitly:",[140,141,146],"pre",{"className":142,"code":143,"language":144,"meta":145,"style":145},"language-yaml shiki shiki-themes tokyo-night","apiVersion: autoscaling\u002Fv2\nkind: HorizontalPodAutoscaler\nmetadata:\n  name: app-hpa\nspec:\n  scaleTargetRef:\n    apiVersion: apps\u002Fv1\n    kind: Deployment\n    name: my-app\n  minReplicas: 2\n  maxReplicas: 10\n  metrics:\n  - type: ContainerResource\n    containerResource:\n      container: app-container\n      target:\n        type: Utilization\n        averageUtilization: 70\n      name: cpu\n","yaml","",[35,147,148,165,176,185,196,204,212,223,234,245,257,268,276,291,299,310,318,329,340],{"__ignoreMap":145},[149,150,153,157,161],"span",{"class":151,"line":152},"line",1,[149,154,156],{"class":155},"s0U2E","apiVersion",[149,158,160],{"class":159},"sAklC",":",[149,162,164],{"class":163},"sPY7s"," autoscaling\u002Fv2\n",[149,166,168,171,173],{"class":151,"line":167},2,[149,169,170],{"class":155},"kind",[149,172,160],{"class":159},[149,174,175],{"class":163}," HorizontalPodAutoscaler\n",[149,177,179,182],{"class":151,"line":178},3,[149,180,181],{"class":155},"metadata",[149,183,184],{"class":159},":\n",[149,186,188,191,193],{"class":151,"line":187},4,[149,189,190],{"class":155},"  name",[149,192,160],{"class":159},[149,194,195],{"class":163}," app-hpa\n",[149,197,199,202],{"class":151,"line":198},5,[149,200,201],{"class":155},"spec",[149,203,184],{"class":159},[149,205,207,210],{"class":151,"line":206},6,[149,208,209],{"class":155},"  scaleTargetRef",[149,211,184],{"class":159},[149,213,215,218,220],{"class":151,"line":214},7,[149,216,217],{"class":155},"    apiVersion",[149,219,160],{"class":159},[149,221,222],{"class":163}," apps\u002Fv1\n",[149,224,226,229,231],{"class":151,"line":225},8,[149,227,228],{"class":155},"    kind",[149,230,160],{"class":159},[149,232,233],{"class":163}," Deployment\n",[149,235,237,240,242],{"class":151,"line":236},9,[149,238,239],{"class":155},"    name",[149,241,160],{"class":159},[149,243,244],{"class":163}," my-app\n",[149,246,248,251,253],{"class":151,"line":247},10,[149,249,250],{"class":155},"  minReplicas",[149,252,160],{"class":159},[149,254,256],{"class":255},"sOJ5S"," 2\n",[149,258,260,263,265],{"class":151,"line":259},11,[149,261,262],{"class":155},"  maxReplicas",[149,264,160],{"class":159},[149,266,267],{"class":255}," 10\n",[149,269,271,274],{"class":151,"line":270},12,[149,272,273],{"class":155},"  metrics",[149,275,184],{"class":159},[149,277,279,283,286,288],{"class":151,"line":278},13,[149,280,282],{"class":281},"sgJMe","  -",[149,284,285],{"class":155}," type",[149,287,160],{"class":159},[149,289,290],{"class":163}," ContainerResource\n",[149,292,294,297],{"class":151,"line":293},14,[149,295,296],{"class":155},"    containerResource",[149,298,184],{"class":159},[149,300,302,305,307],{"class":151,"line":301},15,[149,303,304],{"class":155},"      container",[149,306,160],{"class":159},[149,308,309],{"class":163}," app-container\n",[149,311,313,316],{"class":151,"line":312},16,[149,314,315],{"class":155},"      target",[149,317,184],{"class":159},[149,319,321,324,326],{"class":151,"line":320},17,[149,322,323],{"class":155},"        type",[149,325,160],{"class":159},[149,327,328],{"class":163}," Utilization\n",[149,330,332,335,337],{"class":151,"line":331},18,[149,333,334],{"class":155},"        averageUtilization",[149,336,160],{"class":159},[149,338,339],{"class":255}," 70\n",[149,341,343,346,348],{"class":151,"line":342},19,[149,344,345],{"class":155},"      name",[149,347,160],{"class":159},[149,349,350],{"class":163}," cpu\n",[16,352,353,354,357,358,126],{},"There's one practical catch worth knowing. If the target container doesn't have a resource request set, that Pod's utilization becomes \"undefined\" and HPA simply won't act on it. Also, if you ever rename the container, you need to update the HPA definition before the rollout goes out (",[40,355,45],{"href":42,"rel":356},[44],") — forget that, and HPA quietly stops working, which makes it an easy trap to fall into. DoiT's blog walks through a similar configuration example, noting that it requires Kubernetes 1.27 or later plus Metrics Server installed (",[40,359,362],{"href":360,"rel":361},"https:\u002F\u002Fwww.doit.com\u002Fblog\u002Fkubernetes-fine-grained-horizontal-pod-autoscaling-with-container-resource-metrics",[44],"DoiT Cloud Intelligence",[16,364,365,366,371],{},"Either way, HPA pulls its resource data through the Metrics API that Metrics Server provides (",[40,367,370],{"href":368,"rel":369},"https:\u002F\u002Fkubernetes.io\u002Fdocs\u002Ftasks\u002Fdebug\u002Fdebug-cluster\u002Fresource-metrics-pipeline\u002F",[44],"Kubernetes' resource metrics pipeline documentation","). ContainerResource simply changes the granularity of that same pipeline from Pod-level to container-level — no extra agents or tooling required, as long as you already have Metrics Server running.",[11,373,375],{"id":374},"_4-the-k3skubo-perspective-its-a-standard-kubernetes-feature-so-it-just-works","4. The K3s\u002FKubo Perspective — It's a Standard Kubernetes Feature, So It Just Works",[16,377,378],{},[19,379],{"alt":380,"src":381},"section04","https:\u002F\u002Fcdn.kubo.hexabase.io\u002Fimages\u002Fblog\u002Fen\u002Fk3s-hpa-containerresource-sidecar-utilization-dilution\u002Fsection04.webp",[16,383,384,385,390],{},"Everything covered so far is about HPA, a core Kubernetes feature (kube-controller-manager) — it doesn't depend on any particular distribution. K3s bundles metrics-server as a standard component (",[40,386,389],{"href":387,"rel":388},"https:\u002F\u002Fgithub.com\u002Fk3s-io\u002Fk3s",[44],"K3s official repository","), so ContainerResource metrics work there too, with no extra setup.",[16,392,393,394,397,398,402],{},"That's one of the reasons to choose a K3s-based managed Kubernetes like ",[40,395,98],{"href":96,"rel":396},[44],": because standard Kubernetes features work as-is, you can follow the official documentation for your HPA design without worrying about vendor-specific constraints. ",[40,399,401],{"href":96,"rel":400},[44],"Kubo Cloud"," in particular ships with Prometheus + Grafana built in, so you can verify the effect of switching to ContainerResource metrics right away using your existing monitoring stack — a real practical advantage.",[16,404,405,406,409],{},"If you're currently designing a production setup that combines a service mesh with logging agents, or wrestling with HPA behavior that just doesn't match your expectations, it's worth considering ",[40,407,98],{"href":96,"rel":408},[44]," as a cost-efficient foundation for running that kind of operation.",[11,411,413],{"id":412},"_5-summary","5. Summary",[16,415,416],{},"HPA's standard Resource metric calculates utilization by summing the usage of every container in a Pod. In modern Pod designs that include a sidecar proxy or logging agent, that spec dilutes the scaling decision. Switching to the ContainerResource metric (Stable since Kubernetes 1.30) lets you focus scaling purely on the main application container.",[16,418,419,420,423,424,429],{},"This mechanism is a core Kubernetes feature, so it applies just as well on managed K8s environments like K3s and Kubo. When designing production workloads that use a service mesh or an observability stack, it's worth always asking \"a percentage of what, exactly?\" If you're also reconsidering your underlying operations platform, a managed K3s service like ",[40,421,98],{"href":96,"rel":422},[44]," — where standard Kubernetes features work at full strength — is worth a look. Starting at roughly ¥48,000\u002Fmonth, you can bring the same HPA design you'd use on EKS or AKS over as-is. ",[40,425,428],{"href":426,"rel":427},"https:\u002F\u002Fwww.hexabase.com\u002Fcontact-us\u002F",[44],"Contact us"," if you'd like to talk through a specific setup.",[431,432,433],"style",{},"html pre.shiki code .s0U2E, html code.shiki .s0U2E{--shiki-default:#F7768E}html pre.shiki code .sAklC, html code.shiki .sAklC{--shiki-default:#89DDFF}html pre.shiki code .sPY7s, html code.shiki .sPY7s{--shiki-default:#9ECE6A}html pre.shiki code .sOJ5S, html code.shiki .sOJ5S{--shiki-default:#FF9E64}html pre.shiki code .sgJMe, html code.shiki .sgJMe{--shiki-default:#9ABDF5}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}",{"title":145,"searchDepth":167,"depth":167,"links":435},[436,437,438,439,440],{"id":13,"depth":167,"text":14},{"id":65,"depth":167,"text":66},{"id":102,"depth":167,"text":103},{"id":374,"depth":167,"text":375},{"id":412,"depth":167,"text":413},"2026-10-21","Kubernetes HPA sums resource usage across every container in a Pod, so Istio sidecars and logging agents dilute the scaling decision. Here's how the ContainerResource metric fixes it, in a K3s\u002FKubo context.","md","https:\u002F\u002Fcdn.kubo.hexabase.io\u002Fimages\u002Fblog\u002Fen\u002Fk3s-hpa-containerresource-sidecar-utilization-dilution\u002Ftitle.webp","en",{},true,"\u002Fblog\u002Fen\u002Fk3s-hpa-containerresource-sidecar-utilization-dilution",{"title":5,"description":442},"blog\u002Fen\u002Fk3s-hpa-containerresource-sidecar-utilization-dilution",[452,453,454,455,456,457],"k3s","kubernetes","hpa","containerresource","autoscaling","service-mesh","aT_65sLrm6ZvMJsEoGNM6RRu9zIMRa7FfYyhk7SJGf4",[460,468,475,484,490,497],{"path":461,"title":462,"description":463,"date":464,"tags":465},"\u002Fblog\u002Fen\u002Fk3s-hpa-prometheus-adapter-discovery-interval-stale-metrics","Question Your discoveryInterval: Why K3s HPA Keeps Scaling on Stale Metrics from Prometheus Adapter","K3s HPA won't scale even though CPU usage looks normal. The culprit is a mismatch between Prometheus Adapter's metrics-relist-interval\u002Fmetrics-max-age and the HPA sync-period. Here's the three-layer delay chain and how to diagnose it.","2026-10-12",[452,453,454,466,456,467],"prometheus-adapter","observability",{"path":469,"title":470,"description":471,"date":472,"tags":473},"\u002Fblog\u002Fen\u002Fkubernetes-hpa-vpa-autoscaling-conflict-design","\"Running Both Feels Safer\" Is the Trap. Why Combining HPA and VPA in Kubernetes Can Send Autoscaling Into a Spiral","Enabling both HPA and VPA in Kubernetes can make two well-intentioned automations fight each other, sending autoscaling into an endless spiral. Here's why the conflict happens, and how to combine them safely with updateMode and metric separation.","2026-10-07",[452,453,454,474,456],"vpa",{"path":476,"title":477,"description":478,"date":479,"tags":480},"\u002Fblog\u002Fen\u002Fk3s-keda-hpa-event-driven-autoscaling-decision","CPU Usage Looks Normal, But Requests Are Piling Up. When to Add KEDA to HPA in Kubernetes","KEDA and HPA aren't an either-or choice. When event-driven workloads grow on K3s\u002FKubernetes, here's why HPA alone hits its limits and how to decide when to add KEDA alongside it.","2026-10-03",[452,453,481,454,456,482,483],"keda","event-driven-scaling","scaledobject",{"path":485,"title":486,"description":487,"date":488,"tags":489},"\u002Fblog\u002Fen\u002Fk3s-keda-vs-hpa-event-driven-autoscaling-decision","KEDA vs HPA on K3s: A Decision Framework for Event-Driven Autoscaling","When CPU\u002Fmemory-based HPA on K3s can't keep up with event-driven workloads, should you extend HPA or adopt KEDA, the CNCF-graduated project? Here's a 3-point decision framework, plus what to watch for on K3s.","2026-09-10",[452,453,454,481,456],{"path":491,"title":492,"description":493,"date":494,"tags":495},"\u002Fblog\u002Fen\u002Fk3s-hpa-not-scaling-metrics-unknown-vpa-death-spiral","Your HPA Config Is Correct. So Why Won't Pods Scale? The Real Cause of 'Unknown Metrics' in K3s and the HPA\u002FVPA Death Spiral","You set up an HPA in K3s and Pods still won't scale. The usual culprits are unset resource requests silently disabling metrics, and a self-reinforcing HPA\u002FVPA feedback loop. Here's how to diagnose and fix it.","2026-09-02",[452,453,454,474,456,496],"metrics-server",{"path":498,"title":499,"description":500,"date":501,"tags":502},"\u002Fblog\u002Fen\u002Fk3s-vpa-inplaceorrecreate-pod-resize-cgroup","VPA InPlaceOrRecreate Didn't Stop the Restart. Testing InPlacePodVerticalScaling on K3s","Does VPA's InPlaceOrRecreate mode and In-Place Pod Resize really avoid Pod restarts? We test QoS class constraints and the cgroup v1 trap on K3s with real configs and kubectl commands.","2026-10-17",[452,453,474,456,503],"cgroup",1792592635440]