K3s and Kubernetes CronJobs can keep showing 'Succeeded' while quietly not running for weeks. Here's how concurrencyPolicy, startingDeadlineSeconds, the 100-missed-schedule limit, and exit code 0 combine to create silent failures, and how to monitor for it.
You configured a kubernetes service monitor on K3s, but nothing shows up in Prometheus Targets. 90% of the time, the cause is one of four connection points: label selectors, port names, RBAC, or the namespaceSelector. Here's a step-by-step diagnostic guide.
The trial-and-error troubleshooting behind a home-lab wall-mounted dashboard is the same trap that hits edge Kubernetes clusters scattered across factories and stores. Drawing on official K3s, Prometheus, and Grafana docs, this piece lays out design principles for not deferring observability.