26.1 Access¶
ssh rancher@<node-ip>
sudo -i
export KUBECONFIG=/etc/rancher/rke2/rke2.yaml
export PATH=$PATH:/var/lib/rancher/rke2/bin
Persistent host config lives under /oem (e.g. 90_custom.yaml), and the root filesystem is read-only — changes outside /oem and persistent paths vanish on reboot.
26.2 Common problems¶
| Symptom | Likely cause | Where to look |
|---|---|---|
VM stuck Starting/Pending |
Insufficient CPU/memory (requests), network not on node, PVC not bound | kubectl describe vmi, kubectl describe pod virt-launcher-*, events |
| VM won't schedule on some nodes | VlanConfig doesn't cover node; PCI device on another node |
Node labels network.harvesterhci.io/*, PCIDeviceClaim |
| Image stuck downloading | DNS/proxy, URL redirect, TLS CA | kubectl get vmimage -o yaml, importer/backing-image-manager logs |
Volume degraded |
Node down, disk full, replica rebuild in progress | Longhorn UI (/dashboard/c/local/longhorn link from Harvester), volumes.longhorn.io |
| Live migration fails/hangs | Non-migratable volume, CPU mismatch, write-heavy VM not converging | vmim object status, virt-handler logs on source/target |
| Node stuck in maintenance | Non-migratable VM, PDB | kubectl get vmi -o wide on that node |
| Upgrade stalls | Degraded volumes, disk space, failed node job | upgrades.harvesterhci.io, harvester-system upgrade pods/jobs |
| UI slow / etcd warnings | Slow OS disk, overloaded control plane | etcd metrics, journalctl -u rke2-server |
| VM shows wrong/no IP | No guest agent; DHCP not on VLAN | Install qemu-guest-agent; check switch/DHCP |
| Guest LB IP pending | IPPool selector mismatch, DHCP absent on VLAN | loadbalancers.loadbalancer.harvesterhci.io, ippool status |
| VM OOM-killed | Memory overcommit, insufficient reserved memory | Node dmesg, virt-launcher pod events |
26.3 Useful commands¶
# VMs and where they run
kubectl get vmi -A -o wide
# Why didn't it start?
kubectl -n default describe vm web01
kubectl -n default get pods -l kubevirt.io/domain=web01
kubectl -n default logs virt-launcher-web01-xxxxx -c compute
# Migrations
kubectl get vmim -A
# Longhorn health
kubectl -n longhorn-system get volumes.longhorn.io \
-o custom-columns=NAME:.metadata.name,STATE:.status.state,ROBUSTNESS:.status.robustness,NODE:.status.currentNodeID
# Networks
kubectl get clusternetworks,vlanconfigs,vlanstatuses
ip -d link show mgmt-br
bridge vlan show
# Harvester settings
kubectl get settings.harvesterhci.io -o custom-columns=NAME:.metadata.name,VALUE:.value
# Harvester controller logs
kubectl -n harvester-system logs deploy/harvester -f
kubectl -n harvester-system logs ds/harvester-network-controller -f
26.4 Support bundle¶
Generate from UI or by creating a SupportBundle object; it collects node logs (journald), Kubernetes resources and Longhorn state. Read it yourself first — the answer is often there.
← Air-gapped deployments · Index · Migrating from VMware or OpenStack →