Continuous delivery pipelines to deploy to Kubernetes¶

Continuous delivery (CD) is the process of deploying your application constantly (for example on every commit) and automatically, usually by utilizing automated pipelines. Kubernetes makes utilizing CD rather simple by exposing an API that can easily be used from inside CD pipelines.

There are several ways and technologies how to do CD to Kubernetes, and there's also steps you'd probably want to take before the deployment process. All of these are documented in the GitLab documentation , and there's also example GitLab CI/CD pipeline in this public repository.

• .gitlab-ci-kaniko-build.yml shows how to build images.
• .gitlab-ci-kubectl-deploy.yml shows how to use kubectl to deploy via a pipeline.
• .gitlab-ci-kustomize-deploy.yml shows how to also use kustomize templating on top of kubectl.

These examples have been built for UTHPC necessities, your use case may vary.

Kubernetes permissions for CI¶

Please do not use your own credentials for CI/CD access to Kubernetes. Instead, delivery should be automated via service accounts, with as low permissions as possible.

Here is an example of ServiceAccount, Role and RoleBinding objects, that follow a sensible least-privilege option:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: cicd-serviceaccount
  namespace: <namespace>
---
apiVersion: v1
kind: Secret
metadata:
  name: cicd-serviceaccount-token
  namespace: <namespace>
  annotations:
    kubernetes.io/service-account.name: cicd-serviceaccount
type: kubernetes.io/service-account-token
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: cicd-serviceaccount-role
  namespace: <namespace>
rules:
- apiGroups: [""]
  resources: ["pods", "pods/log", "configmaps", "secrets"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: ["apps", "extensions"]
  resources: ["deployments", "replicasets", "statefulsets"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: cicd-serviceaccount-rolebinding
  namespace: <namespace>
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: cicd-role
subjects:
- kind: ServiceAccount
  name: cicd-serviceaccount
  namespace: <namespace>

Now that a service account has been made, there's one more step - crafting the KUBECONFIG file for the service account, which should be inserted into the CI/CD pipeline as credentials. Sadly, this process is slightly convoluted with just kubectl.

# Define variables for namespace and service account name
NAMESPACE=<namespace> # (1)!
SERVICE_ACCOUNT_NAME=cicd-serviceaccount # (2)!

# Fetch the service account's secret name
SECRET_NAME=$(kubectl get serviceaccount $SERVICE_ACCOUNT_NAME -n $NAMESPACE -o jsonpath='{.secrets[0].name}')

# Extract the token, CA data, and API server from the secret
TOKEN=$(kubectl get secret $SECRET_NAME -n $NAMESPACE -o jsonpath='{.data.token}' | base64 --decode)
CA_DATA=$(kubectl get secret $SECRET_NAME -n $NAMESPACE -o jsonpath='{.data.ca\.crt}')
API_SERVER=$(kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}')

# Generate the KUBECONFIG file
cat <<EOF > ${SERVICE_ACCOUNT_NAME}-kubeconfig
apiVersion: v1
kind: Config
clusters:
- name: default-cluster
  cluster:
    certificate-authority-data: $CA_DATA
    server: $API_SERVER
contexts:
- name: default-context
  context:
    cluster: default-cluster
    namespace: $NAMESPACE
    user: $SERVICE_ACCOUNT_NAME
current-context: default-context
users:
- name: $SERVICE_ACCOUNT_NAME
  user:
    token: $TOKEN
EOF

echo "KUBECONFIG file created: ${SERVICE_ACCOUNT_NAME}-kubeconfig" # (3)!

1. A variable for the namespace where the service account was made.
2. A variable for the service account name.
3. The KUBECONFIG file gets written to local folder, file named ${SERVICE_ACCOUNT_NAME}-kubeconfig. Make sure to delete it after moving it to CI/CD variables, to limit potential security concerns.

This process can be simplified with other tools, like OpenLens or kubectl-view-serviceaccount-kubeconfig-plugin, which make the process of getting the KUBECONFIG file much more straightforward.