Operators¶

Operators are software extensions to Kubernetes that make use of custom resources to manage applications and their components. Operators follow Kubernetes principles. Operators are clients of the Kubernetes API that act as controllers for a Custom Resource.

Redis operator¶

In HPC Kubernetes cluster the Redis Operator is deployed via Helm. It offers these custom resources for setting up Redis:

RedisReplication to provide Replication solution.
Redis to provide Standalone solution.
RedisSentinel to provide sentinel enabled mode.
RedisCluster to have Redis deployed as a cluster.

Here is an example how to set up Redis in sentinel mode.

It is important that the replicated Redis service is deployed first and then sentinel. The Redis replicated solution must be ready and the pods must have roles defined as master/slave.

apiVersion: redis.redis.opstreelabs.in/v1beta2
kind: RedisReplication
metadata:
  name:  redis-replicated
  namespace: <your-namespace> # (1)!
spec:
  clusterSize: 3
  podSecurityContext:
    runAsUser: 1000
    fsGroup: 1000
  kubernetesConfig:
    image: quay.io/opstree/redis:latest # (2)!
    imagePullPolicy: IfNotPresent
    resources:
      requests:
        cpu: 100m
        memory: 128Mi
      limits:
        cpu: 500m
        memory: 256Mi
  storage:
    volumeClaimTemplate:
      spec:
        accessModes: 
          - "ReadWriteOnce"
        resources:
          requests:
            storage: 1Gi # (3)!

define your namespace.
It is recommended to set a specific image version rather than using latest.
Set the needed volume size.

After the Redis replicated setup is in Ready state, deploy Sentinel:

apiVersion: redis.redis.opstreelabs.in/v1beta2
kind: RedisSentinel
metadata:
  name: redis-sentinel
  namespace: <your-namespace>
spec:
  clusterSize: 3
  podSecurityContext:
    runAsUser: 1000
    fsGroup: 1000
  redisSentinelConfig:
    redisReplicationName: redis-replicated # (1)!
    masterGroupName: myMaster # (2)!
  kubernetesConfig:
    image: quay.io/opstree/redis-sentinel:latest
    imagePullPolicy: IfNotPresent
    resources:
      requests:
        cpu: 100m
        memory: 128Mi
      limits:
        cpu: 500m
        memory: 256Mi

The name of the previously created RedisReplication resource name.
The default value - change it if you need a specific name.

It is good practice to set resource limits for any deployment.

Vault Secrets Operator¶

For secure management of sensitive information like API keys, database passwords, or certificates, UTHPC provides a managed HashiCorp Vault service (vault.hpc.ut.ee). This is integrated into Kubernetes via the Vault Secrets Operator (VSO).

VSO synchronizes secrets from Vault directly into native Kubernetes Secret objects. This allows your applications to consume them as environment variables or mounted files without needing to interact with the Vault API directly.

Because vault.hpc.ut.ee is a managed service, setting up a specific integration requires the help of UTHPC administrators.

Step 1: Create a ServiceAccount

First, deploy a Kubernetes ServiceAccount in your namespace. This account will be used to authenticate with Vault.

Step 2: Contact UTHPC Administrators

Provide UTHPC support with your namespace, the name of the created ServiceAccount, and the specific Vault paths that need to be read. Supplying the exact paths is required to ensure the principle of least privilege. Using this information, the administrators will create:

A Vault Policy that grants read access strictly to your requested secret paths.
A Vault Role that binds your ServiceAccount and namespace to the created policy.

Step 3: Deploy Vault Resources

Once UTHPC confirms the setup and provides you with your Vault Role name, you can deploy the VaultAuth and VaultStaticSecret resources to pull your secrets.

Configuration Examples¶

First, deploy your ServiceAccount:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: <your-service-account>
  namespace: <your-namespace>

After UTHPC administrators have configured Vault using your ServiceAccount name and namespace, deploy the VaultAuth resource:

apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultAuth
metadata:
  name: <your-vault-auth-name>
  namespace: <your-namespace>
spec:
  method: kubernetes
  mount: kubernetes
  kubernetes:
    role: <your-vault-role> # (1)!
    serviceAccount: <your-service-account> # (2)!

The Vault role created by the UTHPC admins for your namespace.
The name of the ServiceAccount created in the first step.

Once the VaultAuth setup is in place, you can deploy the VaultStaticSecret to synchronize your actual secret. This is an example of a TLS secret taken from Vault and transformed to the kubernetes.io/tls type:

apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
  name: <your-static-secret-name>
  namespace: <your-namespace>
spec:
  vaultAuthRef: <your-vault-auth-name> # (1)!
  mount: secrets
  type: kv-v1
  path: <your-vault-secret-path> # (2)!
  destination:
    type: kubernetes.io/tls
    name: <your-k8s-secret-name> # (3)!
    create: true
    overwrite: true
    transformation:
      excludeRaw: true
      excludes:
        - .*
      templates: # (4)!
        tls.crt:
          text: "{{ .Secrets.certificate }}"
        tls.key:
          text: "{{ .Secrets.private_key }}"

The name of the VaultAuth resource you just created.
The exact path in the Vault server where your secret is stored.
The name of the native Kubernetes Secret that the operator will create and keep synced.
Maps the specific keys from your Vault secret to the required keys in the Kubernetes tls secret.

Elastic Cloud on Kubernetes operator (ECK)¶

In HPC Kubernetes cluster the Elastic Cloud on Kubernetes operator is deployed via Helm.

The ECK operator can deploy various Elastic services for collecting, processing, and searching large volumes of data. Users currently have access to these custom resources:

Elasticsearch to create and manage Elasticsearch clusters.
Kibana to create and manage Kibana instances for Elasticsearch web interface access.
Logstash to create and manage Logstash instances for collecting logs and sending them to Elasticsearch.

Here is an example how to set up an Elastic cluster with Kibana and Logstash.

First create the Elasticsearch cluster:

apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: elasticsearch-example
spec:
  nodeSets:
  - name: default
    count: 3 # (1)!
    volumeClaimTemplates:
    - metadata:
        name: elasticsearch-data # (2)!
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 5Gi # (3)!
        storageClassName: standard

Elasticsearch node count for high availability.
This name shouldn't be changed unless you set up a volume mount for the data path.
Set the needed volume size.

After that, deploy the Kibana instance:

apiVersion: kibana.k8s.elastic.co/v1
kind: Kibana
metadata:
  name: kibana-example
spec:
  version: 9.5.0
  count: 1
  elasticsearchRef:
    name: elasticsearch-example # (1)!

The Elasticsearch cluster deployment name.

And finally, deploy the Logstash instance and configure a pipeline (below is just an example):

apiVersion: logstash.k8s.elastic.co/v1alpha1
kind: Logstash
metadata:
  name: logstash-example
spec:
  count: 1
  version: 9.4.4
  elasticsearchRefs:
    - clusterName: example # (1)!
      name: elasticsearch-example # (2)!
  pipelines:
    - pipeline.id: main
      config.string: |
        input {
          http {
            port => 8080
            id => "my_http_input"
          }
        }
        output {
          elasticsearch {
            hosts => [ "${EXAMPLE_ES_HOSTS}" ] # (3)!
            user => "${EXAMPLE_ES_USER}"
            password => "${EXAMPLE_ES_PASSWORD}"
            ssl_certificate_authorities => "${EXAMPLE_ES_SSL_CERTIFICATE_AUTHORITY}"
            data_stream_type => "logs"
            data_stream_dataset => "http"
            data_stream_namespace => "default"
          }
        }
  services:
    - name: http
      service:
        spec:
          type: ClusterIP
          ports:
            - port: 8080
              name: http
              protocol: TCP
              targetPort: 8080

Will be the prefix to the Elastic variables in the pipeline.
The Elasticsearch cluster deployment name.
the EXAMPLE_ prefix can be seen here.