Skip to content

Glob pattern based resource selection #28

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open

Glob pattern based resource selection #28

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
0edc6b9
Glob pattern based resource selection
May 10, 2018
d4c3937
Clarify glob pattern places and single/plural form of resource names
May 11, 2018
2c4c423
Change job's concurrencyPolicy from Replace to Forbid in order to pre…
May 30, 2018
936ff05
Merge master
Dec 10, 2018
3ae5e01
add more fields to cleanup
Jul 26, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
169 changes: 85 additions & 84 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,10 @@ Use the deployment example ([ssh](cronjob-ssh.yaml) or [AWS CodeCommit](cronjob-
Define the following environment parameters:
* `GIT_REPO` - GIT repo url. **Required**
* `GIT_PREFIX_PATH` - Path to the subdirectory in your repository. Default: `.`
* `NAMESPACES` - List of namespaces to export. Default: all
* `GLOBALRESOURCES` - List of global resource types to export. Default: `namespace`
* `RESOURCETYPES` - List of resource types to export. Default: `ingress deployment configmap svc rc ds customresourcedefinition networkpolicy statefulset storageclass cronjob`. Notice that `Secret` objects are intentionally not exported by default (see [git-crypt section](#git-crypt) for details).
* `RESOURCES` - **Required**. List of glob patterns `<+/-><resource type>:<namespace pattern>/<object pattern>`. The namespace is optional, e.g. `clusterrole:*`.
Only namespace and object name can contain glob patterns, the resource type can't. **You can use either single or plural form for resource type but be consistent!**
For secrets consider to use [git-crypt section](#git-crypt). Note that Tiller's config maps also can contain secrets.
Example: `deployments:*/* -deployments/*test*/* +deployments/testimonial/* configmaps:*/* -configmaps:tiller/* namespaces:*` - All deployments which aren't contain 'test' except 'testimonial', all configmaps outside of namespace tiller and all namespace definitions.
* `GIT_USERNAME` - Display name of git user. Default: `kube-backup`
* `GIT_EMAIL` - Email address of git user. Default: `[email protected]`
* `GIT_BRANCH` - Use a specific git branch . Default: `master`
Expand Down Expand Up @@ -150,89 +151,89 @@ All configured resources will be exported into a directory tree structure in YAM
```
.
├── kube-system
│   ├── attachdetach-controller.serviceaccounts.yaml
│   ├── canal-config.configmap.yaml
│   ├── canal.daemonset.yaml
│   ├── canal.serviceaccounts.yaml
│   ├── certificate-controller.serviceaccounts.yaml
│   ├── cronjob-controller.serviceaccounts.yaml
│   ├── daemon-set-controller.serviceaccounts.yaml
│   ├── default.serviceaccounts.yaml
│   ├── deployment-controller.serviceaccounts.yaml
│   ├── disruption-controller.serviceaccounts.yaml
│   ├── dns-controller.deployment.yaml
│   ├── dns-controller.serviceaccounts.yaml
│   ├── endpoint-controller.serviceaccounts.yaml
│   ├── generic-garbage-collector.serviceaccounts.yaml
│   ├── horizontal-pod-autoscaler.serviceaccounts.yaml
│   ├── job-controller.serviceaccounts.yaml
│   ├── kube-backup-gpg.secret.yaml
│   ├── kube-backup.serviceaccounts.yaml
│   ├── kube-backup-ssh.secret.yaml
│   ├── kube-dns-autoscaler.configmap.yaml
│   ├── kube-dns-autoscaler.deployment.yaml
│   ├── kube-dns-autoscaler.serviceaccounts.yaml
│   ├── kube-dns.deployment.yaml
│   ├── kube-dns.serviceaccounts.yaml
│   ├── kube-dns.service.yaml
│   ├── kubelet.service.yaml
│   ├── kube-prometheus-exporter-kube-controller-manager.service.yaml
│   ├── kube-prometheus-exporter-kube-dns.service.yaml
│   ├── kube-prometheus-exporter-kube-etcd.service.yaml
│   ├── kube-prometheus-exporter-kube-scheduler.service.yaml
│   ├── kube-proxy.serviceaccounts.yaml
│   ├── kube-state-backup-new.cronjob.yaml
│   ├── kube-sysctl.daemonset.yaml
│   ├── letsencrypt-prod.secret.yaml
│   ├── namespace-controller.serviceaccounts.yaml
│   ├── node-controller.serviceaccounts.yaml
│   ├── openvpn-ccd.configmap.yaml
│   ├── openvpn-crl.configmap.yaml
│   ├── openvpn.deployment.yaml
│   ├── openvpn-ingress.service.yaml
│   ├── openvpn-pki.secret.yaml
│   ├── openvpn-portmapping.configmap.yaml
│   ├── openvpn-settings.configmap.yaml
│   ├── persistent-volume-binder.serviceaccounts.yaml
│   ├── pod-garbage-collector.serviceaccounts.yaml
│   ├── replicaset-controller.serviceaccounts.yaml
│   ├── replication-controller.serviceaccounts.yaml
│   ├── resourcequota-controller.serviceaccounts.yaml
│   ├── route53-config.secret.yaml
│   ├── service-account-controller.serviceaccounts.yaml
│   ├── service-controller.serviceaccounts.yaml
│   ├── statefulset-controller.serviceaccounts.yaml
│   ├── sysctl-options.configmap.yaml
│   ├── tiller-deploy.deployment.yaml
│   ├── tiller-deploy.service.yaml
│   ├── tiller.serviceaccounts.yaml
│   └── ttl-controller.serviceaccounts.yaml
│   ├── serviceaccounts.attachdetach-controller.yaml
│   ├── configmap.canal-config.yaml
│   ├── daemonset.canal.yaml
│   ├── serviceaccounts.canal.yaml
│   ├── serviceaccounts.certificate-controller.yaml
│   ├── serviceaccounts.cronjob-controller.yaml
│   ├── serviceaccounts.daemon-set-controller.yaml
│   ├── serviceaccounts.default.yaml
│   ├── serviceaccounts.deployment-controller.yaml
│   ├── serviceaccounts.disruption-controller.yaml
│   ├── deployment.dns-controller.yaml
│   ├── serviceaccounts.dns-controller.yaml
│   ├── serviceaccounts.endpoint-controller.yaml
│   ├── serviceaccounts.generic-garbage-collector.yaml
│   ├── serviceaccounts.horizontal-pod-autoscaler.yaml
│   ├── serviceaccounts.job-controller.yaml
│   ├── secret.kube-backup-gpg.yaml
│   ├── serviceaccounts.kube-backup.yaml
│   ├── secret.kube-backup-ssh.yaml
│   ├── configmap.kube-dns-autoscaler.yaml
│   ├── deployment.kube-dns-autoscaler.yaml
│   ├── serviceaccounts.kube-dns-autoscaler.yaml
│   ├── deployment.kube-dns.yaml
│   ├── serviceaccounts.kube-dns.yaml
│   ├── service.kube-dns.yaml
│   ├── service.kubelet.yaml
│   ├── service.kube-prometheus-exporter-kube-controller-manager.yaml
│   ├── service.kube-prometheus-exporter-kube-dns.yaml
│   ├── service.kube-prometheus-exporter-kube-etcd.yaml
│   ├── service.kube-prometheus-exporter-kube-scheduler.yaml
│   ├── serviceaccounts.kube-proxy.yaml
│   ├── cronjob.kube-state-backup-new.yaml
│   ├── daemonset.kube-sysctl.yaml
│   ├── secret.letsencrypt-prod.yaml
│   ├── serviceaccounts.namespace-controller.yaml
│   ├── serviceaccounts.node-controller.yaml
│   ├── configmap.openvpn-ccd.yaml
│   ├── configmap.openvpn-crl.yaml
│   ├── deployment.openvpn.yaml
│   ├── service.openvpn-ingress.yaml
│   ├── secret.openvpn-pki.yaml
│   ├── configmap.openvpn-portmapping.yaml
│   ├── configmap.openvpn-settings.yaml
│   ├── serviceaccounts.persistent-volume-binder.yaml
│   ├── serviceaccounts.pod-garbage-collector.yaml
│   ├── serviceaccounts.replicaset-controller.yaml
│   ├── serviceaccounts.replication-controller.yaml
│   ├── serviceaccounts.resourcequota-controller.yaml
│   ├── secret.route53-config.yaml
│   ├── serviceaccounts.service-account-controller.yaml
│   ├── serviceaccounts.service-controller.yaml
│   ├── serviceaccounts.statefulset-controller.yaml
│   ├── configmap.sysctl-options.yaml
│   ├── deployment.tiller-deploy.yaml
│   ├── service.tiller-deploy.yaml
│   ├── serviceaccounts.tiller.yaml
│   └── serviceaccounts.ttl-controller.yaml
├── prd
│   ├── initdb.configmap.yaml
│   ├── example-app.deployment.yaml
│   ├── example-app.ingress.yaml
│   ├── example-app.secret.yaml
│   ├── example-app.service.yaml
│   ├── postgres-admin.secret.yaml
│   ├── postgresql.deployment.yaml
│   ├── postgresql.service.yaml
│   ├── postgres.secret.yaml
│   ├── prd.example.com.secret.yaml
│   ├── redis.service.yaml
│   └── redis-standalone.rc.yaml
│   ├── configmap.initdb.yaml
│   ├── deployment.example-app.yaml
│   ├── ingress.example-app.yaml
│   ├── secret.example-app.yaml
│   ├── service.example-app.yaml
│   ├── secret.postgres-admin.yaml
│   ├── deployment.postgresql.yaml
│   ├── service.postgresql.yaml
│   ├── secret.postgres.yaml
│   ├── secret.prd.example.com.yaml
│   ├── service.redis.yaml
│   └── rc.redis-standalone.yaml
└── staging
   ├── initdb.configmap.yaml
   ├── example-app.deployment.yaml
   ├── example-app.ingress.yaml
   ├── example-app.secret.yaml
   ├── example-app.service.yaml
   ├── postgres-admin.secret.yaml
   ├── postgresql.deployment.yaml
   ├── postgresql.service.yaml
   ├── postgres.secret.yaml
   ├── staging.example.com.secret.yaml
   ├── redis.service.yaml
   └── redis-standalone.rc.yaml
   ├── configmap.initdb.yaml
   ├── deployment.example-app.yaml
   ├── ingress.example-app.yaml
   ├── secret.example-app.yaml
   ├── service.example-app.yaml
   ├── secret.postgres-admin.yaml
   ├── deployment.postgresql.yaml
   ├── service.postgresql.yaml
   ├── secret.postgres.yaml
   ├── secret.staging.example.com.yaml
   ├── service.redis.yaml
   └── rc.redis-standalone.yaml

3 directories, 80 files
```
Expand Down
2 changes: 1 addition & 1 deletion cronjob-codecommit.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ metadata:
app: kube-backup
spec:
schedule: "*/10 * * * *"
concurrencyPolicy: Replace
concurrencyPolicy: Forbid
successfulJobsHistoryLimit: 3
failedJobsHistoryLimit: 3
jobTemplate:
Expand Down
2 changes: 1 addition & 1 deletion cronjob-ssh.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ metadata:
app: kube-backup
spec:
schedule: "*/10 * * * *"
concurrencyPolicy: Replace
concurrencyPolicy: Forbid
successfulJobsHistoryLimit: 3
failedJobsHistoryLimit: 3
jobTemplate:
Expand Down
123 changes: 79 additions & 44 deletions entrypoint.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,68 @@
#!/bin/bash -e

if [ -z "$NAMESPACES" ]; then
NAMESPACES=$(kubectl get ns -o jsonpath={.items[*].metadata.name})
fi
allNamespaces=( $(kubectl get namespaces -o jsonpath='{.items[*].metadata.name}') )
declare -A allObjects=() # key: "namespace/resourceType" or resourceType, value: name list
declare -A selectedObjects=() # key: file name, value: kubectl command

# Find matching objects in the cluster
for resource in $RESOURCES; do
if [[ $resource =~ ^([+-]?)([a-zA-Z]+):(([^/]+)/)?(.*) ]]; then # eg: '+deployments:test/db-*'
direction="${BASH_REMATCH[1]:-+}"
declare -l resourceType="${BASH_REMATCH[2]}"
declare -l namespacePattern="${BASH_REMATCH[4]}"
declare -l objectPattern="${BASH_REMATCH[5]}"
objectsKeys=()

# Get all matching objects from cluster and namespace scopes
# TODO: Handle plural forms like ingress -> ingresses
if [[ -n $namespacePattern ]]; then # namespace scope
for namespace in ${allNamespaces[@]}; do
if [[ $namespace = $namespacePattern ]]; then
objectsKeys=( ${objectsKeys[@]} $namespace/$resourceType )

if [[ ! ${allObjects[$namespace/$resourceType]:+xyz} ]]; then
echo "Querying all $resourceType in $namespace..."
allObjects[$namespace/$resourceType]=$(kubectl get $resourceType -n $namespace -o jsonpath='{.items[*].metadata.name}')
fi
fi
done
else # cluster scope
objectsKeys=( ${objectsKeys[@]} $resourceType )

if [[ ! ${allObjects[$resourceType]:+xyz} ]]; then
echo "Querying all $resourceType in cluster-scope..."
allObjects[$resourceType]=$(kubectl get $resourceType -o jsonpath='{.items[*].metadata.name}')
fi
fi

RESOURCETYPES="${RESOURCETYPES:-"ingress deployment configmap svc rc ds networkpolicy statefulset cronjob pvc"}"
GLOBALRESOURCES="${GLOBALRESOURCES:-"namespace storageclass clusterrole clusterrolebinding customresourcedefinition"}"
# Assemble the object list
if [[ ${#objectsKeys[@]} -ne 0 ]]; then
for key in ${objectsKeys[@]}; do
[[ $key = */* ]] && namespace=${key%%/*}
for objectName in ${allObjects[$key]}; do
if [[ $objectName = $objectPattern ]]; then
fileName="$key.$objectName.yaml"
if [[ $direction = + ]]; then
selectedObjects[$fileName]="kubectl get $resourceType $objectName ${namespace:+--namespace} $namespace"
else
unset selectedObjects[$fileName]
fi
fi
done
done
else
echo "Warning: $resource doesn't match to anything!" >&2
fi
else
echo "Warning: Invalid expression: $resource" >&2
fi

unset namespace
done

# Initialize git repo
[ -z "$DRY_RUN" ] && [ -z "$GIT_REPO" ] && echo "Need to define GIT_REPO environment variable" && exit 1
[ -z "$RESOURCES" ] && echo "Need to define RESOURCES environment variable" && exit 1
GIT_REPO_PATH="${GIT_REPO_PATH:-"/backup/git"}"
GIT_PREFIX_PATH="${GIT_PREFIX_PATH:-"."}"
GIT_USERNAME="${GIT_USERNAME:-"kube-backup"}"
Expand All @@ -22,8 +76,6 @@ if [[ ! -f /backup/.ssh/id_rsa ]]; then
git config --global credential.helper '!aws codecommit credential-helper $@'
git config --global credential.UseHttpPath true
fi
[ -z "$DRY_RUN" ] && git config --global user.name "$GIT_USERNAME"
[ -z "$DRY_RUN" ] && git config --global user.email "$GIT_EMAIL"

[ -z "$DRY_RUN" ] && (test -d "$GIT_REPO_PATH" || git clone --depth 1 "$GIT_REPO" "$GIT_REPO_PATH" --branch "$GIT_BRANCH" || git clone "$GIT_REPO" "$GIT_REPO_PATH")
cd "$GIT_REPO_PATH"
Expand All @@ -47,54 +99,37 @@ fi
[ -z "$DRY_RUN" ] && git rm -r '*.yaml' --ignore-unmatch -f

# Start kubernetes state export
for resource in $GLOBALRESOURCES; do
[ -d "$GIT_REPO_PATH/$GIT_PREFIX_PATH" ] || mkdir -p "$GIT_REPO_PATH/$GIT_PREFIX_PATH"
echo "Exporting resource: ${resource}" >/dev/stderr
kubectl get -o=json "$resource" | jq --sort-keys \
'del(
.items[].metadata.annotations."kubectl.kubernetes.io/last-applied-configuration",
.items[].metadata.annotations."control-plane.alpha.kubernetes.io/leader",
.items[].metadata.uid,
.items[].metadata.selfLink,
.items[].metadata.resourceVersion,
.items[].metadata.creationTimestamp,
.items[].metadata.generation
)' | python -c 'import sys, yaml, json; yaml.safe_dump(json.load(sys.stdin), sys.stdout, default_flow_style=False)' >"$GIT_REPO_PATH/$GIT_PREFIX_PATH/${resource}.yaml"
done

for namespace in $NAMESPACES; do
[ -d "$GIT_REPO_PATH/$GIT_PREFIX_PATH/${namespace}" ] || mkdir -p "$GIT_REPO_PATH/$GIT_PREFIX_PATH/${namespace}"

for type in $RESOURCETYPES; do
echo "[${namespace}] Exporting resources: ${type}" >/dev/stderr
echo "Exporting ${#selectedObjects[@]} object(s):"
for objectKey in ${!selectedObjects[@]}; do
fileName="$GIT_REPO_PATH/$GIT_PREFIX_PATH/$objectKey"
kubectlCommand="${selectedObjects[$objectKey]}"

label_selector=""
if [[ "$type" == 'configmap' && -z "${INCLUDE_TILLER_CONFIGMAPS:-}" ]]; then
label_selector="-l OWNER!=TILLER"
fi

kubectl --namespace="${namespace}" get "$type" $label_selector -o custom-columns=SPACE:.metadata.namespace,KIND:..kind,NAME:.metadata.name --no-headers | while read -r a b name; do
[ -z $name ] && continue
mkdir -p "$(dirname "$fileName")"

# Service account tokens cannot be exported
if [[ "$type" == 'secret' && $(kubectl get -n "${namespace}" -o jsonpath="{.type}" secret "$name") == "kubernetes.io/service-account-token" ]]; then
continue
fi
echo "Exporting to $fileName" >&2

kubectl --namespace="${namespace}" get -o=json "$type" "$name" | jq --sort-keys \
# echo Debug: $kubectlCommand
$kubectlCommand -o=json | jq --sort-keys \
'del(
.metadata.annotations."control-plane.alpha.kubernetes.io/leader",
.metadata.annotations."kubectl.kubernetes.io/last-applied-configuration",
.metadata.annotations."pv.kubernetes.io/bind-completed",
.metadata.annotations."pv.kubernetes.io/bound-by-controller",
.metadata.annotations."volume.beta.kubernetes.io/storage-provisioner",
.metadata.finalizers,
.metadata.creationTimestamp,
.metadata.generation,
.metadata.resourceVersion,
.metadata.selfLink,
.metadata.uid,
.spec.clusterIP,
.status
)' | python -c 'import sys, yaml, json; yaml.safe_dump(json.load(sys.stdin), sys.stdout, default_flow_style=False)' >"$GIT_REPO_PATH/$GIT_PREFIX_PATH/${namespace}/${name}.${type}.yaml"
done
done
.spec.storageClassName,
.spec.volumeMode,
.spec.volumeName,
.secrets,
.status
)' | python -c 'import sys, yaml, json; yaml.safe_dump(json.load(sys.stdin), sys.stdout, default_flow_style=False)' > "$fileName"
done

[ -z "$DRY_RUN" ] || exit
Expand All @@ -103,8 +138,8 @@ cd "${GIT_REPO_PATH}"
git add .

if ! git diff-index --quiet HEAD --; then
git commit -m "Automatic backup at $(date)"
git -c user.name="$GIT_USERNAME" -c user.email="$GIT_EMAIL" commit -m "Automatic backup at $(date)"
git push origin "${GIT_BRANCH}"
else
echo "No change"
echo "No changes"
fi