[{"data":1,"prerenderedAt":1397},["ShallowReactive",2],{"blog-ja-prometheus-monitoring-kubernetes-guide":3,"blog-ja-prometheus-monitoring-kubernetes-guide-alt":521},{"id":4,"title":5,"author":6,"body":7,"date":1382,"description":1383,"extension":1384,"image":137,"locale":1385,"meta":1386,"navigation":521,"path":1387,"seo":1388,"stem":1389,"tags":1390,"__hash__":1396},"blog\u002Fblog\u002Fja\u002Fprometheus-monitoring-kubernetes-guide.md","Prometheus で Kubernetes を監視する完全ガイド","Kubo Team",{"type":8,"value":9,"toc":1359},"minimark",[10,28,33,47,59,62,96,104,113,117,122,131,295,302,355,358,362,375,472,478,482,496,500,538,542,590,594,637,641,650,653,887,891,1114,1122,1128,1131,1145,1148,1187,1190,1193,1201,1221,1224,1238,1241,1296,1299,1302,1332,1342,1355],[11,12,13,14,21,22,27],"p",{},"Kubernetes クラスタを安定運用するうえで、監視基盤の構築は避けて通れません。CNCF Graduated プロジェクトである ",[15,16,20],"a",{"href":17,"rel":18},"https:\u002F\u002Fprometheus.io\u002F",[19],"nofollow","Prometheus"," は、クラウドネイティブ環境における監視のデファクトスタンダードとして広く採用されています。",[15,23,26],{"href":24,"rel":25},"https:\u002F\u002Fkubo.hexabase.io\u002F",[19],"Kubo"," のような K3s ベースの軽量 Kubernetes プラットフォームでも、Prometheus は最小限のリソースで強力な監視機能を提供します。本記事では、Prometheus による Kubernetes 監視の導入から本番運用までを体系的に解説します。",[29,30,32],"h2",{"id":31},"prometheus-のアーキテクチャと基本概念","Prometheus のアーキテクチャと基本概念",[11,34,35,36,41,42,46],{},"Prometheus は、SoundCloud で 2012 年に開発され、2016 年に Kubernetes に次いで CNCF に加入した",[15,37,40],{"href":38,"rel":39},"https:\u002F\u002Fprometheus.io\u002Fdocs\u002Fintroduction\u002Foverview\u002F",[19],"オープンソースの監視・アラートシステム","です。その最大の特徴は ",[43,44,45],"strong",{},"プルベース"," のメトリクス収集モデルにあります。Prometheus サーバーが定期的に監視対象の HTTP エンドポイントをスクレイプし、時系列データとして保存します。",[11,48,49,50,53,54,58],{},"Prometheus のデータモデルは",[43,51,52],{},"多次元的","で、メトリクス名とキー\u002Fバリューペア（ラベル）で識別されます。例えば、",[55,56,57],"code",{},"http_requests_total{method=\"GET\", status=\"200\"}"," のように、単一のメトリクスに複数の次元でフィルタリングやアグリゲーションが可能です。",[11,60,61],{},"主要コンポーネントは以下のとおりです：",[63,64,65,72,78,84,90],"ul",{},[66,67,68,71],"li",{},[43,69,70],{},"Prometheus Server",": メトリクスのスクレイプと時系列データの保存を担当",[66,73,74,77],{},[43,75,76],{},"Alertmanager",": アラートのルーティング、重複排除、通知（Slack、PagerDuty、Email 等）を管理",[66,79,80,83],{},[43,81,82],{},"Pushgateway",": 短命なバッチジョブからのメトリクス受信用中継サーバー",[66,85,86,89],{},[43,87,88],{},"各種 Exporter",": Node Exporter（ハードウェア\u002FOS メトリクス）、kube-state-metrics（Kubernetes オブジェクト状態）など",[66,91,92,95],{},[43,93,94],{},"クライアントライブラリ",": Go、Java、Python、Ruby 等でアプリケーションを計装",[11,97,98,103],{},[15,99,102],{"href":100,"rel":101},"https:\u002F\u002Fwww.sysdig.com\u002Fblog\u002Fkubernetes-monitoring-prometheus",[19],"Sysdig の包括ガイド","によると、Prometheus の Go 製バイナリは単一ノードで自律的に動作し、分散ストレージに依存しないため、デプロイと運用が非常にシンプルです。",[11,105,106,107,112],{},"AI を活用した運用自動化にも興味がある方は、",[15,108,111],{"href":109,"rel":110},"https:\u002F\u002Fwww.hexabase.com\u002Fproduct\u002Fcaptain-ai\u002F",[19],"Captain.AI"," が Kubernetes 環境の運用効率化をどのように実現するかもぜひご覧ください。",[29,114,116],{"id":115},"kubernetes-環境での導入方法","Kubernetes 環境での導入方法",[118,119,121],"h3",{"id":120},"prometheus-operator-による宣言的管理","Prometheus Operator による宣言的管理",[11,123,124,125,130],{},"本番環境では、",[15,126,129],{"href":127,"rel":128},"https:\u002F\u002Fwww.plural.sh\u002Fblog\u002Fprometheus-operator-kubernetes-guide\u002F",[19],"Prometheus Operator"," を使った導入が推奨されます。Prometheus Operator は Kubernetes のカスタムリソース（CRD）を使って、Prometheus の設定をマニフェストとして宣言的に管理します。",[132,133,138],"pre",{"className":134,"code":135,"language":136,"meta":137,"style":137},"language-yaml shiki shiki-themes tokyo-night","apiVersion: monitoring.coreos.com\u002Fv1\nkind: Prometheus\nmetadata:\n  name: k8s-prometheus\n  namespace: monitoring\nspec:\n  replicas: 2\n  serviceAccountName: prometheus\n  serviceMonitorSelector:\n    matchLabels:\n      team: platform\n  retention: 30d\n  resources:\n    requests:\n      memory: 400Mi\n","yaml","",[55,139,140,157,168,177,188,199,207,219,230,238,246,257,268,276,284],{"__ignoreMap":137},[141,142,145,149,153],"span",{"class":143,"line":144},"line",1,[141,146,148],{"class":147},"s0U2E","apiVersion",[141,150,152],{"class":151},"sAklC",":",[141,154,156],{"class":155},"sPY7s"," monitoring.coreos.com\u002Fv1\n",[141,158,160,163,165],{"class":143,"line":159},2,[141,161,162],{"class":147},"kind",[141,164,152],{"class":151},[141,166,167],{"class":155}," Prometheus\n",[141,169,171,174],{"class":143,"line":170},3,[141,172,173],{"class":147},"metadata",[141,175,176],{"class":151},":\n",[141,178,180,183,185],{"class":143,"line":179},4,[141,181,182],{"class":147},"  name",[141,184,152],{"class":151},[141,186,187],{"class":155}," k8s-prometheus\n",[141,189,191,194,196],{"class":143,"line":190},5,[141,192,193],{"class":147},"  namespace",[141,195,152],{"class":151},[141,197,198],{"class":155}," monitoring\n",[141,200,202,205],{"class":143,"line":201},6,[141,203,204],{"class":147},"spec",[141,206,176],{"class":151},[141,208,210,213,215],{"class":143,"line":209},7,[141,211,212],{"class":147},"  replicas",[141,214,152],{"class":151},[141,216,218],{"class":217},"sOJ5S"," 2\n",[141,220,222,225,227],{"class":143,"line":221},8,[141,223,224],{"class":147},"  serviceAccountName",[141,226,152],{"class":151},[141,228,229],{"class":155}," prometheus\n",[141,231,233,236],{"class":143,"line":232},9,[141,234,235],{"class":147},"  serviceMonitorSelector",[141,237,176],{"class":151},[141,239,241,244],{"class":143,"line":240},10,[141,242,243],{"class":147},"    matchLabels",[141,245,176],{"class":151},[141,247,249,252,254],{"class":143,"line":248},11,[141,250,251],{"class":147},"      team",[141,253,152],{"class":151},[141,255,256],{"class":155}," platform\n",[141,258,260,263,265],{"class":143,"line":259},12,[141,261,262],{"class":147},"  retention",[141,264,152],{"class":151},[141,266,267],{"class":155}," 30d\n",[141,269,271,274],{"class":143,"line":270},13,[141,272,273],{"class":147},"  resources",[141,275,176],{"class":151},[141,277,279,282],{"class":143,"line":278},14,[141,280,281],{"class":147},"    requests",[141,283,176],{"class":151},[141,285,287,290,292],{"class":143,"line":286},15,[141,288,289],{"class":147},"      memory",[141,291,152],{"class":151},[141,293,294],{"class":155}," 400Mi\n",[11,296,297,298,301],{},"最も手軽な導入方法は ",[43,299,300],{},"kube-prometheus-stack"," Helm チャートを使う方法です：",[132,303,307],{"className":304,"code":305,"language":306,"meta":137,"style":137},"language-bash shiki shiki-themes tokyo-night","helm repo add prometheus-community https:\u002F\u002Fprometheus-community.github.io\u002Fhelm-charts\nhelm install kube-prometheus prometheus-community\u002Fkube-prometheus-stack \\\n  --namespace monitoring --create-namespace\n","bash",[55,308,309,327,343],{"__ignoreMap":137},[141,310,311,315,318,321,324],{"class":143,"line":144},[141,312,314],{"class":313},"sE3pS","helm",[141,316,317],{"class":155}," repo",[141,319,320],{"class":155}," add",[141,322,323],{"class":155}," prometheus-community",[141,325,326],{"class":155}," https:\u002F\u002Fprometheus-community.github.io\u002Fhelm-charts\n",[141,328,329,331,334,337,340],{"class":143,"line":159},[141,330,314],{"class":313},[141,332,333],{"class":155}," install",[141,335,336],{"class":155}," kube-prometheus",[141,338,339],{"class":155}," prometheus-community\u002Fkube-prometheus-stack",[141,341,342],{"class":151}," \\\n",[141,344,345,349,352],{"class":143,"line":170},[141,346,348],{"class":347},"sT800","  --namespace",[141,350,351],{"class":155}," monitoring",[141,353,354],{"class":347}," --create-namespace\n",[11,356,357],{},"この Helm チャートには Prometheus Server、Alertmanager、Grafana、Node Exporter、kube-state-metrics が含まれており、すぐに使える監視スタックが構築できます。",[118,359,361],{"id":360},"kubernetes-サービスディスカバリ","Kubernetes サービスディスカバリ",[11,363,364,365,370,371,374],{},"Prometheus は Kubernetes API と連携し、Pod、Service、Endpoint、Node を",[15,366,369],{"href":367,"rel":368},"https:\u002F\u002Fprometheus.io\u002Fdocs\u002Fprometheus\u002Flatest\u002Fconfiguration\u002Fconfiguration\u002F#kubernetes_sd_config",[19],"自動的に検出","します。",[55,372,373],{},"ServiceMonitor"," リソースを使えば、アノテーションベースで監視対象を柔軟に追加できます：",[132,376,378],{"className":134,"code":377,"language":136,"meta":137,"style":137},"apiVersion: monitoring.coreos.com\u002Fv1\nkind: ServiceMonitor\nmetadata:\n  name: my-app-monitor\nspec:\n  selector:\n    matchLabels:\n      app: my-app\n  endpoints:\n  - port: metrics\n    interval: 15s\n",[55,379,380,388,397,403,412,418,425,431,441,448,462],{"__ignoreMap":137},[141,381,382,384,386],{"class":143,"line":144},[141,383,148],{"class":147},[141,385,152],{"class":151},[141,387,156],{"class":155},[141,389,390,392,394],{"class":143,"line":159},[141,391,162],{"class":147},[141,393,152],{"class":151},[141,395,396],{"class":155}," ServiceMonitor\n",[141,398,399,401],{"class":143,"line":170},[141,400,173],{"class":147},[141,402,176],{"class":151},[141,404,405,407,409],{"class":143,"line":179},[141,406,182],{"class":147},[141,408,152],{"class":151},[141,410,411],{"class":155}," my-app-monitor\n",[141,413,414,416],{"class":143,"line":190},[141,415,204],{"class":147},[141,417,176],{"class":151},[141,419,420,423],{"class":143,"line":201},[141,421,422],{"class":147},"  selector",[141,424,176],{"class":151},[141,426,427,429],{"class":143,"line":209},[141,428,243],{"class":147},[141,430,176],{"class":151},[141,432,433,436,438],{"class":143,"line":221},[141,434,435],{"class":147},"      app",[141,437,152],{"class":151},[141,439,440],{"class":155}," my-app\n",[141,442,443,446],{"class":143,"line":232},[141,444,445],{"class":147},"  endpoints",[141,447,176],{"class":151},[141,449,450,454,457,459],{"class":143,"line":240},[141,451,453],{"class":452},"sgJMe","  -",[141,455,456],{"class":147}," port",[141,458,152],{"class":151},[141,460,461],{"class":155}," metrics\n",[141,463,464,467,469],{"class":143,"line":248},[141,465,466],{"class":147},"    interval",[141,468,152],{"class":151},[141,470,471],{"class":155}," 15s\n",[11,473,474,477],{},[15,475,26],{"href":24,"rel":476},[19]," は K3s ベースで CNCF エコシステムとの高い親和性を持つため、Prometheus の導入もスムーズに行えます。",[29,479,481],{"id":480},"promql-を使いこなす実践的なクエリ例","PromQL を使いこなす：実践的なクエリ例",[11,483,484,489,490,495],{},[15,485,488],{"href":486,"rel":487},"https:\u002F\u002Fprometheus.io\u002Fdocs\u002Fprometheus\u002Flatest\u002Fquerying\u002Fbasics\u002F",[19],"PromQL","（Prometheus Query Language）は、Prometheus の多次元データモデルを最大限に活用するための強力なクエリ言語です。",[15,491,494],{"href":492,"rel":493},"https:\u002F\u002Flogz.io\u002Fblog\u002Fkubernetes-monitoring-prometheus-guide\u002F",[19],"Logz.io のガイド","でも強調されているように、適切な PromQL クエリの設計がプロアクティブな監視の鍵となります。",[118,497,499],{"id":498},"cpuメモリ使用率の監視","CPU・メモリ使用率の監視",[132,501,505],{"className":502,"code":503,"language":504,"meta":137,"style":137},"language-promql shiki shiki-themes tokyo-night","# ノードの CPU 使用率（%）\n100 - (avg by(instance) (irate(node_cpu_seconds_total{mode=\"idle\"}[5m])) * 100)\n\n# Pod のメモリ使用率\ncontainer_memory_working_set_bytes{container!=\"POD\",container!=\"\"}\n  \u002F on(namespace, pod) kube_pod_container_resource_limits{resource=\"memory\"} * 100\n","promql",[55,506,507,512,517,523,528,533],{"__ignoreMap":137},[141,508,509],{"class":143,"line":144},[141,510,511],{},"# ノードの CPU 使用率（%）\n",[141,513,514],{"class":143,"line":159},[141,515,516],{},"100 - (avg by(instance) (irate(node_cpu_seconds_total{mode=\"idle\"}[5m])) * 100)\n",[141,518,519],{"class":143,"line":170},[141,520,522],{"emptyLinePlaceholder":521},true,"\n",[141,524,525],{"class":143,"line":179},[141,526,527],{},"# Pod のメモリ使用率\n",[141,529,530],{"class":143,"line":190},[141,531,532],{},"container_memory_working_set_bytes{container!=\"POD\",container!=\"\"}\n",[141,534,535],{"class":143,"line":201},[141,536,537],{},"  \u002F on(namespace, pod) kube_pod_container_resource_limits{resource=\"memory\"} * 100\n",[118,539,541],{"id":540},"リクエストレートとエラー率red-メソッド","リクエストレートとエラー率（RED メソッド）",[132,543,545],{"className":502,"code":544,"language":504,"meta":137,"style":137},"# リクエストレート（1分あたり）\nsum(rate(http_requests_total[5m])) by (service)\n\n# エラー率（%）\nsum(rate(http_requests_total{status=~\"5..\"}[5m])) by (service)\n  \u002F sum(rate(http_requests_total[5m])) by (service) * 100\n\n# レイテンシ P99\nhistogram_quantile(0.99, sum(rate(http_request_duration_seconds_bucket[5m])) by (le, service))\n",[55,546,547,552,557,561,566,571,576,580,585],{"__ignoreMap":137},[141,548,549],{"class":143,"line":144},[141,550,551],{},"# リクエストレート（1分あたり）\n",[141,553,554],{"class":143,"line":159},[141,555,556],{},"sum(rate(http_requests_total[5m])) by (service)\n",[141,558,559],{"class":143,"line":170},[141,560,522],{"emptyLinePlaceholder":521},[141,562,563],{"class":143,"line":179},[141,564,565],{},"# エラー率（%）\n",[141,567,568],{"class":143,"line":190},[141,569,570],{},"sum(rate(http_requests_total{status=~\"5..\"}[5m])) by (service)\n",[141,572,573],{"class":143,"line":201},[141,574,575],{},"  \u002F sum(rate(http_requests_total[5m])) by (service) * 100\n",[141,577,578],{"class":143,"line":209},[141,579,522],{"emptyLinePlaceholder":521},[141,581,582],{"class":143,"line":221},[141,583,584],{},"# レイテンシ P99\n",[141,586,587],{"class":143,"line":232},[141,588,589],{},"histogram_quantile(0.99, sum(rate(http_request_duration_seconds_bucket[5m])) by (le, service))\n",[118,591,593],{"id":592},"kubernetes-固有のクエリ","Kubernetes 固有のクエリ",[132,595,597],{"className":502,"code":596,"language":504,"meta":137,"style":137},"# Pending 状態の Pod 数\nkube_pod_status_phase{phase=\"Pending\"}\n\n# CrashLoopBackOff の検出\nincrease(kube_pod_container_status_restarts_total[1h]) > 5\n\n# PVC の使用率\nkubelet_volume_stats_used_bytes \u002F kubelet_volume_stats_capacity_bytes * 100\n",[55,598,599,604,609,613,618,623,627,632],{"__ignoreMap":137},[141,600,601],{"class":143,"line":144},[141,602,603],{},"# Pending 状態の Pod 数\n",[141,605,606],{"class":143,"line":159},[141,607,608],{},"kube_pod_status_phase{phase=\"Pending\"}\n",[141,610,611],{"class":143,"line":170},[141,612,522],{"emptyLinePlaceholder":521},[141,614,615],{"class":143,"line":179},[141,616,617],{},"# CrashLoopBackOff の検出\n",[141,619,620],{"class":143,"line":190},[141,621,622],{},"increase(kube_pod_container_status_restarts_total[1h]) > 5\n",[141,624,625],{"class":143,"line":201},[141,626,522],{"emptyLinePlaceholder":521},[141,628,629],{"class":143,"line":209},[141,630,631],{},"# PVC の使用率\n",[141,633,634],{"class":143,"line":221},[141,635,636],{},"kubelet_volume_stats_used_bytes \u002F kubelet_volume_stats_capacity_bytes * 100\n",[29,638,640],{"id":639},"alertmanager-によるアラート設計","Alertmanager によるアラート設計",[11,642,643,644,649],{},"安定した運用のためには、適切なアラート設計が不可欠です。",[15,645,648],{"href":646,"rel":647},"https:\u002F\u002Fprometheus.io\u002Fdocs\u002Falerting\u002Flatest\u002Falerting_rules\u002F",[19],"Prometheus のアラーティングルール","と Alertmanager を組み合わせることで、障害の早期検知と通知が実現できます。",[118,651,652],{"id":652},"アラートルールの定義",[132,654,656],{"className":134,"code":655,"language":136,"meta":137,"style":137},"apiVersion: monitoring.coreos.com\u002Fv1\nkind: PrometheusRule\nmetadata:\n  name: kubernetes-alerts\n  namespace: monitoring\nspec:\n  groups:\n  - name: kubernetes.rules\n    rules:\n    - alert: PodCrashLooping\n      expr: increase(kube_pod_container_status_restarts_total[1h]) > 5\n      for: 10m\n      labels:\n        severity: warning\n      annotations:\n        summary: \"Pod {{ $labels.namespace }}\u002F{{ $labels.pod }} が頻繁に再起動しています\"\n    - alert: HighMemoryUsage\n      expr: |\n        container_memory_working_set_bytes{container!=\"POD\",container!=\"\"}\n        \u002F on(namespace,pod) kube_pod_container_resource_limits{resource=\"memory\"} > 0.9\n      for: 5m\n      labels:\n        severity: critical\n      annotations:\n        summary: \"メモリ使用率が 90% を超えています\"\n",[55,657,658,666,675,681,690,698,704,711,723,730,743,753,763,770,780,787,804,816,827,833,839,849,856,866,873],{"__ignoreMap":137},[141,659,660,662,664],{"class":143,"line":144},[141,661,148],{"class":147},[141,663,152],{"class":151},[141,665,156],{"class":155},[141,667,668,670,672],{"class":143,"line":159},[141,669,162],{"class":147},[141,671,152],{"class":151},[141,673,674],{"class":155}," PrometheusRule\n",[141,676,677,679],{"class":143,"line":170},[141,678,173],{"class":147},[141,680,176],{"class":151},[141,682,683,685,687],{"class":143,"line":179},[141,684,182],{"class":147},[141,686,152],{"class":151},[141,688,689],{"class":155}," kubernetes-alerts\n",[141,691,692,694,696],{"class":143,"line":190},[141,693,193],{"class":147},[141,695,152],{"class":151},[141,697,198],{"class":155},[141,699,700,702],{"class":143,"line":201},[141,701,204],{"class":147},[141,703,176],{"class":151},[141,705,706,709],{"class":143,"line":209},[141,707,708],{"class":147},"  groups",[141,710,176],{"class":151},[141,712,713,715,718,720],{"class":143,"line":221},[141,714,453],{"class":452},[141,716,717],{"class":147}," name",[141,719,152],{"class":151},[141,721,722],{"class":155}," kubernetes.rules\n",[141,724,725,728],{"class":143,"line":232},[141,726,727],{"class":147},"    rules",[141,729,176],{"class":151},[141,731,732,735,738,740],{"class":143,"line":240},[141,733,734],{"class":452},"    -",[141,736,737],{"class":147}," alert",[141,739,152],{"class":151},[141,741,742],{"class":155}," PodCrashLooping\n",[141,744,745,748,750],{"class":143,"line":248},[141,746,747],{"class":147},"      expr",[141,749,152],{"class":151},[141,751,752],{"class":155}," increase(kube_pod_container_status_restarts_total[1h]) > 5\n",[141,754,755,758,760],{"class":143,"line":259},[141,756,757],{"class":147},"      for",[141,759,152],{"class":151},[141,761,762],{"class":155}," 10m\n",[141,764,765,768],{"class":143,"line":270},[141,766,767],{"class":147},"      labels",[141,769,176],{"class":151},[141,771,772,775,777],{"class":143,"line":278},[141,773,774],{"class":147},"        severity",[141,776,152],{"class":151},[141,778,779],{"class":155}," warning\n",[141,781,782,785],{"class":143,"line":286},[141,783,784],{"class":147},"      annotations",[141,786,176],{"class":151},[141,788,790,793,795,798,801],{"class":143,"line":789},16,[141,791,792],{"class":147},"        summary",[141,794,152],{"class":151},[141,796,797],{"class":151}," \"",[141,799,800],{"class":155},"Pod {{ $labels.namespace }}\u002F{{ $labels.pod }} が頻繁に再起動しています",[141,802,803],{"class":151},"\"\n",[141,805,807,809,811,813],{"class":143,"line":806},17,[141,808,734],{"class":452},[141,810,737],{"class":147},[141,812,152],{"class":151},[141,814,815],{"class":155}," HighMemoryUsage\n",[141,817,819,821,823],{"class":143,"line":818},18,[141,820,747],{"class":147},[141,822,152],{"class":151},[141,824,826],{"class":825},"sd1Qi"," |\n",[141,828,830],{"class":143,"line":829},19,[141,831,832],{"class":155},"        container_memory_working_set_bytes{container!=\"POD\",container!=\"\"}\n",[141,834,836],{"class":143,"line":835},20,[141,837,838],{"class":155},"        \u002F on(namespace,pod) kube_pod_container_resource_limits{resource=\"memory\"} > 0.9\n",[141,840,842,844,846],{"class":143,"line":841},21,[141,843,757],{"class":147},[141,845,152],{"class":151},[141,847,848],{"class":155}," 5m\n",[141,850,852,854],{"class":143,"line":851},22,[141,853,767],{"class":147},[141,855,176],{"class":151},[141,857,859,861,863],{"class":143,"line":858},23,[141,860,774],{"class":147},[141,862,152],{"class":151},[141,864,865],{"class":155}," critical\n",[141,867,869,871],{"class":143,"line":868},24,[141,870,784],{"class":147},[141,872,176],{"class":151},[141,874,876,878,880,882,885],{"class":143,"line":875},25,[141,877,792],{"class":147},[141,879,152],{"class":151},[141,881,797],{"class":151},[141,883,884],{"class":155},"メモリ使用率が 90% を超えています",[141,886,803],{"class":151},[118,888,890],{"id":889},"alertmanager-の通知設定","Alertmanager の通知設定",[132,892,894],{"className":134,"code":893,"language":136,"meta":137,"style":137},"route:\n  receiver: 'slack-notifications'\n  group_by: ['alertname', 'namespace']\n  group_wait: 30s\n  group_interval: 5m\n  repeat_interval: 4h\n  routes:\n  - match:\n      severity: critical\n    receiver: 'pagerduty-critical'\n\nreceivers:\n- name: 'slack-notifications'\n  slack_configs:\n  - channel: '#alerts'\n    send_resolved: true\n- name: 'pagerduty-critical'\n  pagerduty_configs:\n  - service_key: '\u003Cservice-key>'\n",[55,895,896,903,919,950,960,969,979,986,995,1004,1018,1022,1029,1044,1051,1067,1077,1091,1098],{"__ignoreMap":137},[141,897,898,901],{"class":143,"line":144},[141,899,900],{"class":147},"route",[141,902,176],{"class":151},[141,904,905,908,910,913,916],{"class":143,"line":159},[141,906,907],{"class":147},"  receiver",[141,909,152],{"class":151},[141,911,912],{"class":151}," '",[141,914,915],{"class":155},"slack-notifications",[141,917,918],{"class":151},"'\n",[141,920,921,924,926,929,932,935,937,940,942,945,947],{"class":143,"line":170},[141,922,923],{"class":147},"  group_by",[141,925,152],{"class":151},[141,927,928],{"class":151}," [",[141,930,931],{"class":151},"'",[141,933,934],{"class":155},"alertname",[141,936,931],{"class":151},[141,938,939],{"class":151},",",[141,941,912],{"class":151},[141,943,944],{"class":155},"namespace",[141,946,931],{"class":151},[141,948,949],{"class":151},"]\n",[141,951,952,955,957],{"class":143,"line":179},[141,953,954],{"class":147},"  group_wait",[141,956,152],{"class":151},[141,958,959],{"class":155}," 30s\n",[141,961,962,965,967],{"class":143,"line":190},[141,963,964],{"class":147},"  group_interval",[141,966,152],{"class":151},[141,968,848],{"class":155},[141,970,971,974,976],{"class":143,"line":201},[141,972,973],{"class":147},"  repeat_interval",[141,975,152],{"class":151},[141,977,978],{"class":155}," 4h\n",[141,980,981,984],{"class":143,"line":209},[141,982,983],{"class":147},"  routes",[141,985,176],{"class":151},[141,987,988,990,993],{"class":143,"line":221},[141,989,453],{"class":452},[141,991,992],{"class":147}," match",[141,994,176],{"class":151},[141,996,997,1000,1002],{"class":143,"line":232},[141,998,999],{"class":147},"      severity",[141,1001,152],{"class":151},[141,1003,865],{"class":155},[141,1005,1006,1009,1011,1013,1016],{"class":143,"line":240},[141,1007,1008],{"class":147},"    receiver",[141,1010,152],{"class":151},[141,1012,912],{"class":151},[141,1014,1015],{"class":155},"pagerduty-critical",[141,1017,918],{"class":151},[141,1019,1020],{"class":143,"line":248},[141,1021,522],{"emptyLinePlaceholder":521},[141,1023,1024,1027],{"class":143,"line":259},[141,1025,1026],{"class":147},"receivers",[141,1028,176],{"class":151},[141,1030,1031,1034,1036,1038,1040,1042],{"class":143,"line":270},[141,1032,1033],{"class":452},"-",[141,1035,717],{"class":147},[141,1037,152],{"class":151},[141,1039,912],{"class":151},[141,1041,915],{"class":155},[141,1043,918],{"class":151},[141,1045,1046,1049],{"class":143,"line":278},[141,1047,1048],{"class":147},"  slack_configs",[141,1050,176],{"class":151},[141,1052,1053,1055,1058,1060,1062,1065],{"class":143,"line":286},[141,1054,453],{"class":452},[141,1056,1057],{"class":147}," channel",[141,1059,152],{"class":151},[141,1061,912],{"class":151},[141,1063,1064],{"class":155},"#alerts",[141,1066,918],{"class":151},[141,1068,1069,1072,1074],{"class":143,"line":789},[141,1070,1071],{"class":147},"    send_resolved",[141,1073,152],{"class":151},[141,1075,1076],{"class":217}," true\n",[141,1078,1079,1081,1083,1085,1087,1089],{"class":143,"line":806},[141,1080,1033],{"class":452},[141,1082,717],{"class":147},[141,1084,152],{"class":151},[141,1086,912],{"class":151},[141,1088,1015],{"class":155},[141,1090,918],{"class":151},[141,1092,1093,1096],{"class":143,"line":818},[141,1094,1095],{"class":147},"  pagerduty_configs",[141,1097,176],{"class":151},[141,1099,1100,1102,1105,1107,1109,1112],{"class":143,"line":829},[141,1101,453],{"class":452},[141,1103,1104],{"class":147}," service_key",[141,1106,152],{"class":151},[141,1108,912],{"class":151},[141,1110,1111],{"class":155},"\u003Cservice-key>",[141,1113,918],{"class":151},[11,1115,1116,1121],{},[15,1117,1120],{"href":1118,"rel":1119},"https:\u002F\u002Fwww.apptio.com\u002Ftopics\u002Fkubernetes\u002Fmonitoring\u002Fprometheus\u002F",[19],"Apptio のガイド","が指摘するように、Alertmanager を Prometheus とは別プロセスで運用することで、Prometheus に問題が発生してもアラーティング機能が独立して動作し続けます。",[11,1123,1124,1127],{},[15,1125,111],{"href":109,"rel":1126},[19]," と組み合わせれば、アラート発生時に AI が自動的に原因を分析し、対処の提案を行うワークフローの構築も可能です。",[29,1129,1130],{"id":1130},"本番環境でのベストプラクティス",[11,1132,1133,1138,1139,1144],{},[15,1134,1137],{"href":1135,"rel":1136},"https:\u002F\u002Ftrilio.io\u002Fkubernetes-best-practices\u002Fkubernetes-monitoring-best-practices\u002F",[19],"Trilio のベストプラクティスガイド","や ",[15,1140,1143],{"href":1141,"rel":1142},"https:\u002F\u002Fwww.plural.sh\u002Fblog\u002Fprometheus-kubernetes-monitoring-guide\u002F",[19],"Plural のガイド","をもとに、本番環境で推奨される設定をまとめます。",[118,1146,1147],{"id":1147},"高可用性の確保",[132,1149,1151],{"className":134,"code":1150,"language":136,"meta":137,"style":137},"spec:\n  replicas: 2\n  shards: 1\n  replicaExternalLabelName: __replica__\n",[55,1152,1153,1159,1167,1177],{"__ignoreMap":137},[141,1154,1155,1157],{"class":143,"line":144},[141,1156,204],{"class":147},[141,1158,176],{"class":151},[141,1160,1161,1163,1165],{"class":143,"line":159},[141,1162,212],{"class":147},[141,1164,152],{"class":151},[141,1166,218],{"class":217},[141,1168,1169,1172,1174],{"class":143,"line":170},[141,1170,1171],{"class":147},"  shards",[141,1173,152],{"class":151},[141,1175,1176],{"class":217}," 1\n",[141,1178,1179,1182,1184],{"class":143,"line":179},[141,1180,1181],{"class":147},"  replicaExternalLabelName",[141,1183,152],{"class":151},[141,1185,1186],{"class":155}," __replica__\n",[11,1188,1189],{},"複数のレプリカを運用し、Thanos や Cortex で長期保存とグローバルビューを実現します。",[118,1191,1192],{"id":1192},"メトリクスの選定と最適化",[11,1194,1195,1200],{},[15,1196,1199],{"href":1197,"rel":1198},"https:\u002F\u002Fwww.tasrieit.com\u002Fblog\u002Fprometheus-monitoring-kubernetes-complete-guide-2026",[19],"Tasrie IT のガイド","が警告するように、すべてのメトリクスを無差別に収集するとストレージコストが膨れ上がります。以下の戦略を採用しましょう：",[63,1202,1203,1209,1215],{},[66,1204,1205,1208],{},[43,1206,1207],{},"ラベルカーディナリティの制御",": 高カーディナリティなラベル（ユーザー ID やリクエスト ID）は避ける",[66,1210,1211,1214],{},[43,1212,1213],{},"Recording Rules の活用",": 頻繁に使うクエリは事前に計算し、クエリ負荷を削減",[66,1216,1217,1220],{},[43,1218,1219],{},"リテンション期間の適切な設定",": ローカルは 15-30 日、長期保存はリモートストレージへ",[118,1222,1223],{"id":1223},"セキュリティ対策",[63,1225,1226,1229,1232],{},[66,1227,1228],{},"NetworkPolicy で Prometheus のアクセスを制限",[66,1230,1231],{},"RBAC で最小権限の原則を適用",[66,1233,1234,1237],{},[55,1235,1236],{},"\u002Fmetrics"," エンドポイントの認証・暗号化",[118,1239,1240],{"id":1240},"リモートストレージ連携",[132,1242,1244],{"className":134,"code":1243,"language":136,"meta":137,"style":137},"remoteWrite:\n- url: \"http:\u002F\u002Fthanos-receive:19291\u002Fapi\u002Fv1\u002Freceive\"\n  queueConfig:\n    maxSamplesPerSend: 1000\n    batchSendDeadline: 5s\n",[55,1245,1246,1253,1269,1276,1286],{"__ignoreMap":137},[141,1247,1248,1251],{"class":143,"line":144},[141,1249,1250],{"class":147},"remoteWrite",[141,1252,176],{"class":151},[141,1254,1255,1257,1260,1262,1264,1267],{"class":143,"line":159},[141,1256,1033],{"class":452},[141,1258,1259],{"class":147}," url",[141,1261,152],{"class":151},[141,1263,797],{"class":151},[141,1265,1266],{"class":155},"http:\u002F\u002Fthanos-receive:19291\u002Fapi\u002Fv1\u002Freceive",[141,1268,803],{"class":151},[141,1270,1271,1274],{"class":143,"line":170},[141,1272,1273],{"class":147},"  queueConfig",[141,1275,176],{"class":151},[141,1277,1278,1281,1283],{"class":143,"line":179},[141,1279,1280],{"class":147},"    maxSamplesPerSend",[141,1282,152],{"class":151},[141,1284,1285],{"class":217}," 1000\n",[141,1287,1288,1291,1293],{"class":143,"line":190},[141,1289,1290],{"class":147},"    batchSendDeadline",[141,1292,152],{"class":151},[141,1294,1295],{"class":155}," 5s\n",[29,1297,1298],{"id":1298},"まとめ",[11,1300,1301],{},"Prometheus は Kubernetes 監視の中核として、メトリクス収集から可視化、アラーティングまでを一貫して提供します。本記事で解説したポイントをまとめると：",[1303,1304,1305,1310,1316,1321,1326],"ol",{},[66,1306,1307,1309],{},[43,1308,129],{}," を使った宣言的な導入・管理",[66,1311,1312,1315],{},[43,1313,1314],{},"サービスディスカバリ"," による動的なターゲット検出",[66,1317,1318,1320],{},[43,1319,488],{}," による柔軟なクエリと分析",[66,1322,1323,1325],{},[43,1324,76],{}," による体系的なアラート設計",[66,1327,1328,1331],{},[43,1329,1330],{},"HA 構成とリモートストレージ"," による本番レベルの信頼性",[11,1333,1334,1337,1338,1341],{},[15,1335,26],{"href":24,"rel":1336},[19]," は K3s ベースで CNCF エコシステムとの高い親和性を持ち、Prometheus をはじめとする監視ツールをすぐに活用できる環境を提供しています。Kubernetes 環境の構築・運用でお困りの方は、ぜひ ",[15,1339,26],{"href":24,"rel":1340},[19]," をご検討ください。",[11,1343,1344,1345,1348,1349,1354],{},"また、AI を活用した Kubernetes 運用の自動化に興味がある方は、",[15,1346,111],{"href":109,"rel":1347},[19]," がインテリジェントな運用支援をどのように実現するかもご確認ください。導入のご相談は",[15,1350,1353],{"href":1351,"rel":1352},"https:\u002F\u002Fwww.hexabase.com\u002Fcontact-us\u002F",[19],"お問い合わせ","からお気軽にどうぞ。",[1356,1357,1358],"style",{},"html pre.shiki code .s0U2E, html code.shiki .s0U2E{--shiki-default:#F7768E}html pre.shiki code .sAklC, html code.shiki .sAklC{--shiki-default:#89DDFF}html pre.shiki code .sPY7s, html code.shiki .sPY7s{--shiki-default:#9ECE6A}html pre.shiki code .sOJ5S, html code.shiki .sOJ5S{--shiki-default:#FF9E64}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html pre.shiki code .sE3pS, html code.shiki .sE3pS{--shiki-default:#C0CAF5}html pre.shiki code .sT800, html code.shiki .sT800{--shiki-default:#E0AF68}html pre.shiki code .sgJMe, html code.shiki .sgJMe{--shiki-default:#9ABDF5}html pre.shiki code .sd1Qi, html code.shiki .sd1Qi{--shiki-default:#BB9AF7}",{"title":137,"searchDepth":159,"depth":159,"links":1360},[1361,1362,1366,1371,1375,1381],{"id":31,"depth":159,"text":32},{"id":115,"depth":159,"text":116,"children":1363},[1364,1365],{"id":120,"depth":170,"text":121},{"id":360,"depth":170,"text":361},{"id":480,"depth":159,"text":481,"children":1367},[1368,1369,1370],{"id":498,"depth":170,"text":499},{"id":540,"depth":170,"text":541},{"id":592,"depth":170,"text":593},{"id":639,"depth":159,"text":640,"children":1372},[1373,1374],{"id":652,"depth":170,"text":652},{"id":889,"depth":170,"text":890},{"id":1130,"depth":159,"text":1130,"children":1376},[1377,1378,1379,1380],{"id":1147,"depth":170,"text":1147},{"id":1192,"depth":170,"text":1192},{"id":1223,"depth":170,"text":1223},{"id":1240,"depth":170,"text":1240},{"id":1298,"depth":159,"text":1298},"2026-05-27","Prometheus を使った Kubernetes 監視の導入から運用まで。PromQL、Alertmanager、サービスディスカバリなど本番環境で必要な設定を体系的に解説します。","md","ja",{},"\u002Fblog\u002Fja\u002Fprometheus-monitoring-kubernetes-guide",{"title":5,"description":1383},"blog\u002Fja\u002Fprometheus-monitoring-kubernetes-guide",[20,1391,1392,1393,1394,1395,488,76],"Kubernetes","監視","モニタリング","CNCF","Observability","S0l9WOHoTry_w3O4gYMxZM_ATQI8LDazMuFIaJ6EGeM",1779964617053]