Kubernetes has become the de facto standard for modern application deployment, but building and operating a full K8s cluster demands significant expertise. K3s is a lightweight Kubernetes distribution developed by Rancher Labs that delivers full Kubernetes functionality with minimal resource overhead. Combined with Proxmox VE, it enables you to build a flexible, cost-effective Kubernetes foundation.
For organizations needing enterprise-grade managed Kubernetes, Kubo On-Premise delivers a fully managed K8s environment on Proxmox, minimizing infrastructure management overhead so your team can focus on development.
Architecture Design and VM Planning
K3s cluster design balances availability requirements against resource budgets. The K3s official documentation recommends a minimum of 3 server nodes for production environments.
Recommended Configuration
| Role | Count | CPU | RAM | Storage | OS |
|---|---|---|---|---|---|
| Server (Control Plane) | 3 | 4 vCPU | 8 GB | 50 GB SSD | Ubuntu 24.04 LTS |
| Worker (Data Plane) | 3+ | 4 vCPU | 16 GB | 100 GB SSD | Ubuntu 24.04 LTS |
Network Design
Assign static IPs outside the DHCP range to cluster nodes:
| Node | Hostname | IP Address |
|---|---|---|
| Server 1 | k3s-server-1 | 192.168.1.101 |
| Server 2 | k3s-server-2 | 192.168.1.102 |
| Server 3 | k3s-server-3 | 192.168.1.103 |
| Worker 1 | k3s-worker-1 | 192.168.1.111 |
| Worker 2 | k3s-worker-2 | 192.168.1.112 |
| Worker 3 | k3s-worker-3 | 192.168.1.113 |
Leverage VLANs in Proxmox network configuration to isolate management and cluster traffic.
Creating VM Templates
Build a Cloud-Init enabled VM template for efficient node provisioning. This allows you to spin up new nodes in minutes through simple clone operations.
Downloading Ubuntu Cloud Image and Creating the Template
# Run on Proxmox host
# Download Ubuntu 24.04 Cloud Image
wget https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img
# Create VM (ID 9000 reserved for templates)
qm create 9000 --name ubuntu-2404-template --memory 4096 --cores 2 \
--net0 virtio,bridge=vmbr0 --scsihw virtio-scsi-single
# Import disk image
qm set 9000 --scsi0 local-lvm:0,import-from=/root/noble-server-cloudimg-amd64.img
# Add Cloud-Init drive
qm set 9000 --ide2 local-lvm:cloudinit
qm set 9000 --boot order=scsi0
qm set 9000 --serial0 socket --vga serial0
# Set Cloud-Init defaults
qm set 9000 --ciuser ubuntu --cipassword <password>
qm set 9000 --sshkeys ~/.ssh/id_rsa.pub
qm set 9000 --ipconfig0 ip=dhcp
# Convert to template
qm template 9000
Cloning Nodes
Create full clones from the template for each node:
# Server nodes
qm clone 9000 101 --name k3s-server-1 --full
qm clone 9000 102 --name k3s-server-2 --full
qm clone 9000 103 --name k3s-server-3 --full
# Worker nodes (with increased resources)
qm clone 9000 111 --name k3s-worker-1 --full
qm set 111 --memory 16384 --cores 4
qm clone 9000 112 --name k3s-worker-2 --full
qm set 112 --memory 16384 --cores 4
Set static IPs on each node:
qm set 101 --ipconfig0 ip=192.168.1.101/24,gw=192.168.1.1
qm set 102 --ipconfig0 ip=192.168.1.102/24,gw=192.168.1.1
# Continue for remaining nodes
With Kubo, you can bypass this complexity entirely and get a production-grade K8s environment through declarative configuration alone.
Installing and Configuring K3s
Initializing the First Server Node
Initialize the K3s cluster on the first server node (k3s-server-1). Following the K3s Quick-Start Guide:
# Run on first server node
curl -sfL https://get.k3s.io | K3S_TOKEN=<secure-token> sh -s - server \
--cluster-init \
--tls-san=192.168.1.100 \
--disable traefik \
--write-kubeconfig-mode 644
Option explanations:
--cluster-init: Initialize in HA mode using embedded etcd--tls-san: Add load balancer VIP to the certificate's Subject Alternative Names--disable traefik: Disable built-in Traefik to install a custom Ingress Controller later--write-kubeconfig-mode 644: Set readable permissions on kubeconfig
Joining Additional Server Nodes
Add remaining server nodes to the cluster:
# Run on k3s-server-2 and k3s-server-3
curl -sfL https://get.k3s.io | K3S_TOKEN=<secure-token> sh -s - server \
--server https://192.168.1.101:6443 \
--tls-san=192.168.1.100
Adding Worker Nodes
Worker nodes join by specifying the K3S_URL environment variable:
# Run on each worker node
curl -sfL https://get.k3s.io | K3S_URL=https://192.168.1.101:6443 \
K3S_TOKEN=<secure-token> sh -
The server token can be retrieved from /var/lib/rancher/k3s/server/node-token on any server node.
Configuring kubeconfig
Copy the kubeconfig to your local machine for kubectl access:
# Retrieve kubeconfig from server node
scp ubuntu@192.168.1.101:/etc/rancher/k3s/k3s.yaml ~/.kube/config
# Update server address to external IP
sed -i 's/127.0.0.1/192.168.1.101/' ~/.kube/config
# Verify node list
kubectl get nodes -o wide
Network and Storage Configuration
MetalLB Load Balancer
In bare-metal environments, MetalLB assigns external IPs to LoadBalancer-type Services:
# Install MetalLB
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.14.9/config/manifests/metallb-native.yaml
# Configure IP address pool
cat <<EOF | kubectl apply -f -
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
name: default-pool
namespace: metallb-system
spec:
addresses:
- 192.168.1.200-192.168.1.250
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
name: default
namespace: metallb-system
EOF
NGINX Ingress Controller
Route HTTP/HTTPS traffic with the NGINX Ingress Controller:
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.12.0/deploy/static/provider/baremetal/deploy.yaml
Longhorn Distributed Storage
Longhorn is a Kubernetes-native distributed block storage system with excellent K3s integration:
# Install Longhorn
kubectl apply -f https://raw.githubusercontent.com/longhorn/longhorn/v1.7.2/deploy/longhorn.yaml
# Set as default StorageClass
kubectl patch storageclass longhorn -p '{"metadata":{"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
Operations and Monitoring
Cluster Health Checks
# Node status
kubectl get nodes -o wide
# System pod verification
kubectl get pods -A
# K3s service status
systemctl status k3s # Server nodes
systemctl status k3s-agent # Worker nodes
Backup and Recovery
K3s automatically takes etcd snapshots, but manual backups are also important:
# Manual etcd snapshot
k3s etcd-snapshot save --name manual-backup-$(date +%Y%m%d)
# List snapshots
k3s etcd-snapshot ls
To automate K3s cluster operations and achieve enterprise-grade reliability, Kubo On-Premise is the ideal solution.
Conclusion
Building a K3s cluster on Proxmox VE delivers a flexible, cost-effective Kubernetes environment. This guide covered efficient provisioning with VM templates, high availability configuration, distributed storage with Longhorn, and everything needed for production operations.
For advanced management capabilities and enterprise support, consider adopting Kubo. Even on Proxmox, Kubo On-Premise operates as fully managed K8s, handling cluster lifecycle management, auto-scaling, and security patching in a single integrated solution.
For more details, contact us to discuss your requirements.
Related Links: