Backup and Restore Methods

In this lesson we cover common Kubernetes backup and restore approaches and what to include in your backups. You’ll learn which cluster artifacts to protect, practical commands for capturing them, and high-level restore notes for etcd snapshots. Earlier you deployed applications using Deployments, Pods, and Services defined in manifest files. The primary backup candidates are:

Resource configuration (object manifests)
etcd cluster (Kubernetes datastore)
Persistent Volumes (application data)

A dark-slide graphic titled "Backup Candidates" showing three items: Resource Configuration, etcd Cluster, and Persistent Volumes. Each item is represented by a blue icon (file with gear, gear-like cluster icon, and stacked-database icon).

Why back these up?

Resource manifests let you redeploy and recreate objects in a new cluster or namespace.
etcd snapshots capture the authoritative cluster state (including secrets) at a point in time.
Persistent Volumes contain application data that must be restored separately from object definitions.

Resource configurations

Kubernetes objects can be created imperatively (kubectl create) or declaratively (apply manifest files). The declarative approach—storing manifests in version control—is the recommended best practice because it enables reproducible deployments, audit history, and CI/CD automation (GitOps). Example Pod manifest (save as pod-definition.yml):

apiVersion: v1
kind: Pod
metadata:
  name: myapp-pod
  labels:
    app: myapp
    type: front-end
spec:
  containers:
  - name: nginx-container
    image: nginx

Apply the manifest:

kubectl apply -f pod-definition.yml

Store manifests in a source-code repository (GitHub, GitLab, etc.) so they are versioned and shareable. With manifests available, you can redeploy workloads even if the cluster is lost. Exporting current cluster objects

Not every team follows declarative manifests. When objects are created imperatively, you can export the current API state and persist it as YAML files.
A simple capture of common workload objects:

kubectl get all --all-namespaces -o yaml > all-deploy-services.yaml

Note: kubectl get all is a convenience resource and does not include every resource type (for example, PersistentVolumeClaims, PersistentVolumes, CustomResourceDefinitions, ClusterRoleBindings, etc.). A comprehensive backup requires enumerating the resource types you need or using a purpose-built tool. Recommended options:

Use provider or community tools (for example, Velero) to automate resource and persistent volume backups: https://velero.io
If you build scripts, enumerate resources using kubectl api-resources and capture each resource type you depend on.

etcd backups

etcd is the authoritative key-value store for Kubernetes cluster state. Backing up etcd (via filesystem copies or snapshots) gives you a full point-in-time capture of object definitions, node membership, and most cluster metadata. This can be crucial for disaster recovery. Create a snapshot (basic example):

ETCDCTL_API=3 etcdctl snapshot save snapshot.db

If you have multiple etcdctl binaries, ensure you use the v3 client (for example via ETCDCTL_API=3), and confirm compatibility with the etcd server version. Secure example with explicit endpoints and certificate files:

ETCDCTL_API=3 etcdctl \
  --endpoints=https://127.0.0.1:2379 \
  --cacert=/etc/etcd/ca.pem \
  --cert=/etc/etcd/kubernetes.pem \
  --key=/etc/etcd/kubernetes-key.pem \
  snapshot save snapshot.db

Verify the snapshot file exists:

ls -l snapshot.db
-rw-r--r-- 1 root root 12345678 Jul 10 12:34 snapshot.db

High-level restore process

Stop kube-apiserver so it does not try to communicate with etcd during restore.
Use etcdctl snapshot restore to create a new data directory from the snapshot.
Reconfigure the etcd service (or static-pod manifest) to point to the restored directory, set correct cluster bootstrap parameters, restart etcd, and then start kube-apiserver.

Important restore notes:

If control plane components are run as static pods (kubeadm managed), the kube-apiserver and etcd may be defined as static pod manifests—stop or replace those manifests appropriately.
etcdctl snapshot restore initializes a new data directory and generates fresh cluster metadata (new member IDs). Do not assume a restored data directory will automatically rejoin an existing cluster. Carefully set --initial-cluster, --name, and --initial-cluster-state to match whether you intend to form a new cluster or rejoin nodes.

Example commands (stop kube-apiserver, restore, then restart services):

sudo systemctl stop kube-apiserver

ETCDCTL_API=3 etcdctl snapshot restore snapshot.db --data-dir /var/lib/etcd-from-backup
# Output: I | mvcc: restore compact to 475629

After the restore, update your etcd service/startup options so --data-dir points to the restored directory and bootstrap flags are correct. Example systemd unit snippet (update --data-dir accordingly):

[Service]
ExecStart=/usr/local/bin/etcd \
  --name ${ETCD_NAME} \
  --cert-file=/etc/etcd/kubernetes.pem \
  --key-file=/etc/etcd/kubernetes-key.pem \
  --peer-cert-file=/etc/etcd/kubernetes.pem \
  --peer-key-file=/etc/etcd/kubernetes-key.pem \
  --trusted-ca-file=/etc/etcd/ca.pem \
  --peer-trusted-ca-file=/etc/etcd/ca.pem \
  --peer-client-cert-auth \
  --client-cert-auth \
  --initial-advertise-peer-urls https://${INTERNAL_IP}:2380 \
  --listen-peer-urls https://${INTERNAL_IP}:2380 \
  --listen-client-urls https://${INTERNAL_IP}:2379,https://127.0.0.1:2379 \
  --advertise-client-urls https://${INTERNAL_IP}:2379 \
  --initial-cluster-token etcd-cluster-0 \
  --initial-cluster controller-0=https://${CONTROLLER0_IP}:2380,controller-1=https://${CONTROLLER1_IP}:2380 \
  --initial-cluster-state new \
  --data-dir=/var/lib/etcd

Reload and restart services:

sudo systemctl daemon-reload
sudo systemctl restart etcd
sudo systemctl start kube-apiserver

When running etcdctl in a TLS-enabled cluster, always specify the --endpoints and the CA, client certificate, and key files (--cacert, --cert, --key). Without these flags, etcdctl cannot connect to a secured etcd endpoint.

Choosing a method

Both backing up resource manifests and backing up etcd snapshots have trade-offs. Use the table below to choose the right approach for your environment.

Backup Approach	What it captures	When to use
Resource manifests (YAML)	Resource definitions that you choose to export (Deployments, Services, ConfigMaps, etc.) — excludes cluster-level binary state and some status fields	When you want portability, version control, or are using a managed Kubernetes service where etcd is not accessible
etcd snapshot	Full cluster state stored in etcd, including secrets and status fields	When you have control-plane access, need a full point-in-time restore, and can manage cluster bootstrap and certificates
Persistent Volume backups	Application data stored on PVs (snapshotting volumes, copying files, or using provider snapshots)	Always for stateful apps — must be backed up separately from manifests/etcd

Additional considerations

Managed Kubernetes (EKS, GKE, AKS): You may not have etcd access. Use provider-managed backup features or export manifests and PV snapshots as supported by the cloud provider.
Secrets: Secrets stored in etcd are captured by etcd snapshots. If you only export manifests you must take care not to lose secret data (store secrets in external secret stores or securely back them up).
Consistency: For a consistent restore of application state, coordinate etcd snapshots with PV snapshots (application-consistent backups require quiescing or using storage-provider volume snapshots).

Tools and references

Velero (backup and restore for Kubernetes resources and PVs): https://velero.io
etcd snapshot guide: https://etcd.io/docs/v3.5/op-guide/snapshot/
Kubernetes documentation: https://kubernetes.io/docs/

That’s it for this lesson. Practice taking and restoring etcd snapshots and exporting manifests in a lab environment to validate your processes and scripts.

Introduction

Core Concepts

Scheduling

Logging & Monitoring

Application Lifecycle Management

Cluster Maintenance

Security

Storage

Networking

Design and Install a Kubernetes Cluster

Install "Kubernetes the kubeadm way"

Troubleshooting

Helm Basics (2025 Updates)

Kustomize Basics (2025 Updates)

Other Topics

Mock Exams

Backup and Restore Methods

Resource configurations

etcd backups

Choosing a method

Watch Video

​Resource configurations

​etcd backups

​Choosing a method

Watch Video

Resource configurations

etcd backups

Choosing a method