Backup and Restore a Stateful App using Velero
Prerequisite
You need Velero CLI setup, follow the velero-cli doc
Prepare manifests for app
You will need the following manifest to deploy the sample app.
Security Context Constraints to have proper permissions to your app
kind: SecurityContextConstraints
apiVersion: v1
allowHostPorts: true
allowedCapabilities:
- '*'
defaultAddCapabilities: []
fsGroup:
type: RunAsAny
groups:
- system:cluster-admins
- system:nodes
metadata:
name: privileged-app-test
priority: null
readOnlyRootFilesystem: false
requiredDropCapabilities: []
runAsUser:
type: RunAsAny
seLinuxContext:
type: RunAsAny
seccompProfiles:
- '*'
supplementalGroups:
type: RunAsAny
users:
- system:serviceaccount:cassandra-app:default
volumes:
- '*'
Create a namespace:
Create the Stateful app:
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: cassandra
namespace: cassandra-app
labels:
app: cassandra
spec:
serviceName: cassandra
replicas: 1
selector:
matchLabels:
app: cassandra
template:
metadata:
labels:
app: cassandra
spec:
terminationGracePeriodSeconds: 180
containers:
- name: cassandra
image: gcr.io/google-samples/cassandra:v11
ports:
- containerPort: 7000
name: intra-node
- containerPort: 7001
name: tls-intra-node
- containerPort: 7199
name: jmx
- containerPort: 9042
name: cql
resources:
limits:
cpu: "500m"
memory: 2Gi
requests:
cpu: "500m"
memory: 1Gi
securityContext:
capabilities:
add:
- IPC_LOCK
lifecycle:
preStop:
exec:
command:
- /bin/sh
- -c
- nodetool drain
env:
- name: MAX_HEAP_SIZE
value: 512M
- name: HEAP_NEWSIZE
value: 100M
# Make sure the DNS name matches the namespace
- name: CASSANDRA_SEEDS
value: "cassandra-0.cassandra.cassandra-app.svc.cluster.local"
- name: CASSANDRA_CLUSTER_NAME
value: "Demo-Cluster"
- name: CASSANDRA_DC
value: "Demo-DataCenter"
- name: CASSANDRA_RACK
value: "Demo-Rack"
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
readinessProbe:
exec:
command:
- /bin/bash
- -c
- /ready-probe.sh
initialDelaySeconds: 15
timeoutSeconds: 5
volumeMounts:
- name: cassandra-data
mountPath: /cassandra_data
volumeClaimTemplates:
- metadata:
name: cassandra-data
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: standard
resources:
requests:
storage: 1Gi
Create the service for the app:
apiVersion: v1
kind: Service
metadata:
labels:
app: cassandra
name: cassandra
namespace: cassandra-app
spec:
clusterIP: None
selector:
app: cassandra
Note: If you see some error or failed app, please try to ensure CASSANDRA_SEEDS value in your stateful app.
Deploy app, populate data and verify
Deploy all the above manifests. Once on a successful deployment, you can check the app status using:
and you should see something like this:
Datacenter: Demo-DataCenter
===========================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens Owns (effective) Host ID Rack
UN 10.130.4.200 150.08 KiB 32 100.0% 2516974e-3065-4acd-9762-f7b740867cd4 Demo-Rack
Now we will populate data, run the command to connect into pod:
Now you should be connected to cqlsh utility in the app pod, running the commands to populate data:
cqlsh> CREATE KEYSPACE demodb WITH REPLICATION = { 'class' : 'SimpleStrategy', 'replication_factor' : 3 };
cqlsh> use demodb;
cqlsh:demodb> CREATE TABLE emp(emp_id int PRIMARY KEY, emp_name text, emp_city text, emp_sal varint,emp_phone varint);
cqlsh:demodb> INSERT INTO emp (emp_id, emp_name, emp_city, emp_phone, emp_sal) VALUES (100, 'Cormac', 'Cork', 999, 1000000);
cqlsh:demodb> select * from emp;
emp_id | emp_city | emp_name | emp_phone | emp_sal
--------+----------+----------+-----------+---------
100 | Cork | Cormac | 999 | 1000000
(1 rows)
cqlsh:demodb> exit
If everything goes well, your output should resemble to the one record above.
Take backup and destroy app
To take Velero backup, use velero command:
velero backup create cassandra-backup --include-namespaces cassandra-app --namespace <VELERO_NAMESAPCE>
or you can use Backup CR:
apiVersion: velero.io/v1
kind: Backup
metadata:
name: cassandra-backup
namespace: openshift-velero
spec:
includedNamespaces:
- cassandra-app
includedResources:
- '*'
storageLocation: default
ttl: 2h0m0s
Now delete the app namespace cassandra-app and this should delete everything in the namespace including deployed app and its Volume.
Restore and verify app
To perform a Velero restore, use velero command:
Or we can use Restore CR to perform a restore:
apiVersion: velero.io/v1
kind: Restore
metadata:
name: cassandra-backup-restore
namespace: openshift-velero
spec:
backupName: cassandra-backup
excludedResources:
- nodes
- events
- events.events.k8s.io
- backups.velero.io
- restores.velero.io
- resticrepositories.velero.io
includedNamespaces:
- '*'
After a successful restore, we should be able to see a pod of our app running and should be able to link to its shell. Check the status again:
and you should see the status as before:
Datacenter: Demo-DataCenter
===========================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens Owns (effective) Host ID Rack
UN 10.130.4.200 150.08 KiB 32 100.0% 2516974e-3065-4acd-9762-f7b740867cd4 Demo-Rack
We will again exec in the app using:
Check for the populated data:
cqlsh> use demodb;
cqlsh:demodb> select * from emp;
emp_id | emp_city | emp_name | emp_phone | emp_sal
--------+----------+----------+-----------+---------
100 | Cork | Cormac | 999 | 1000000
(1 rows)
cqlsh:demodb> exit
With a correct restore, the above data should be same as before.