Skip to content

Continuous delivery pipelines to deploy to Kubernetes

Continuous delivery (CD) is the process of deploying your application constantly (for example on every commit) and automatically, usually by utilizing automated pipelines. Kubernetes makes utilizing CD rather simple by exposing an API that can easily be used from inside CD pipelines.

CI/CD

There are several ways and technologies how to do CD to Kubernetes, and there's also steps you'd probably want to take before the deployment process. All of these are documented in the GitLab documentation , and there's also example GitLab CI/CD pipeline in this public repository.

These examples have been built for UTHPC necessities, your use case may vary.

Kubernetes permissions for CI

Please do not use your own credentials for CI/CD access to Kubernetes. Instead, delivery should be automated via service accounts, with as low permissions as possible.

Here is an example of ServiceAccount, Role and RoleBinding objects, that follow a sensible least-privilege option:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: cicd-serviceaccount
  namespace: <namespace>
---
apiVersion: v1
kind: Secret
metadata:
  name: cicd-serviceaccount-token
  namespace: <namespace>
  annotations:
    kubernetes.io/service-account.name: cicd-serviceaccount
type: kubernetes.io/service-account-token
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: cicd-serviceaccount-role
  namespace: <namespace>
rules:
- apiGroups: [""]
  resources: ["pods", "pods/log", "configmaps", "secrets"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: ["apps", "extensions"]
  resources: ["deployments", "replicasets", "statefulsets"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: cicd-serviceaccount-rolebinding
  namespace: <namespace>
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: cicd-role
subjects:
- kind: ServiceAccount
  name: cicd-serviceaccount
  namespace: <namespace>

Now that a service account has been made, there's one more step - crafting the KUBECONFIG file for the service account, which should be inserted into the CI/CD pipeline as credentials. Sadly, this process is slightly convoluted with just kubectl.

# Define variables for namespace and service account name
NAMESPACE=<namespace> # (1)!
SERVICE_ACCOUNT_NAME=cicd-serviceaccount # (2)!

# Fetch the service account's secret name
SECRET_NAME=$(kubectl get serviceaccount $SERVICE_ACCOUNT_NAME -n $NAMESPACE -o jsonpath='{.secrets[0].name}')

# Extract the token, CA data, and API server from the secret
TOKEN=$(kubectl get secret $SECRET_NAME -n $NAMESPACE -o jsonpath='{.data.token}' | base64 --decode)
CA_DATA=$(kubectl get secret $SECRET_NAME -n $NAMESPACE -o jsonpath='{.data.ca\.crt}')
API_SERVER=$(kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}')

# Generate the KUBECONFIG file
cat <<EOF > ${SERVICE_ACCOUNT_NAME}-kubeconfig
apiVersion: v1
kind: Config
clusters:
- name: user@kubernetes
  cluster:
    certificate-authority-data: $CA_DATA
    server: $API_SERVER
contexts:
- name: user@kubernetes-${SERVICE_ACCOUNT_NAME}
  context:
    cluster: user@kubernetes
    namespace: $NAMESPACE
    user: $SERVICE_ACCOUNT_NAME
current-context: user@kubernetes-${SERVICE_ACCOUNT_NAME}
users:
- name: $SERVICE_ACCOUNT_NAME
  user:
    token: $TOKEN
EOF

echo "KUBECONFIG file created: ${SERVICE_ACCOUNT_NAME}-kubeconfig" # (3)!
  1. A variable for the namespace where the service account was made.
  2. A variable for the service account name.
  3. The KUBECONFIG file gets written to local folder, file named ${SERVICE_ACCOUNT_NAME}-kubeconfig. Make sure to delete it after moving it to CI/CD variables, to limit potential security concerns.

This process can be simplified with other tools, like OpenLens or kubectl-view-serviceaccount-kubeconfig-plugin, which make the process of getting the KUBECONFIG file much more straightforward.