Vault

Vault is a tool for securely accessing secrets. A secret is anything that you want to tightly control access to, such as API keys, passwords, or certificates. Vault provides a unified interface to any secret, while providing tight access control and recording a detailed audit log.

When you create a secret in Kubernetes it is stored in etcd as plain text, also the secret is accessible to anyone that has access to your cluster. Vault solves this issue by providing a central secret management store that provides an additional layer of security using it's authentication methods. Secrets are only accessible when you provide a corresponding token

There are 2 kinds of secrets in the Vault:

Secrets for managed applications provided by Stakater (ex: Nexus repository credential, etc.) Users only have read permission. The path is managed-addons/*.
Tenant specific secrets. A KV v2 secret engine is enabled on TENANT_NAME/kv path in default. Even though users delete that path, it is created automatically so please don't remove that. Users can enable/disable secret engines on TENANT_NAME/* paths and create/delete/update/read secrets in them.

Users can manage secrets via Vault UI or Vault CLI.

Manage Vault secrets via UI

Users included in any tenants can access to the Vault UI using OIDC authentication.

Once login, users can do all actions on the path TENANT_NAME/*.

Enable/disable any kinds of secret engines
Create/update/get/list/delete secrets

Authentication

Vault-oidc-login

Access https://stakater-vault-openshift-stakater-vault.CLUSTER_DOMAIN
Select OIDC method on Sing in to Vault page.
Keep Role as default.
Click Sign in with OIDC Provider
Then login on popup authentication dialog. The browser should allow the popup dialog.

Vault-login_popup

Enable secret engines

select-secret-engine

configure-secret-engine

Create secrets

create-secret

Manage Vault secrets via CLI

To use Vault CLI, the token is required. Users can get/renew/revoke the token on the UI. (Click the user account Avatar.)

Vault-token

Once token is fetched, users can use the CLI provided by UI. So there is no need to install Vault CLI.

Vault-cli

Vault login token=${TOKEN}

Consume Vault secrets

SAAP supports 3 different ways to consume secrets from Vault:

Option # 1 - Consume Vault secret via a Volume
Option # 2 - Consume Vault secret via Environment Variable
Option # 3 - Consume Vault secret via ExternalSecret (Recommended)

Below you can find step by step guide to consume via different options.

Option # 1 - Consume Vault secret via a Volume

To mount Vault secret in a volume do following:

TODO Is this step required by all three options?

Step 1: Add label in serviceaccount so it can be granted Vault read access to secret path

 serviceAccount:
   enabled: true
   additionalLabels: 
     stakater.com/vault-access: "true"

Step 2: Enable SecretProviderClass object in Helm values and define key and value path of Vault. For example

secretProviderClass:
 enabled: true
 name: postgres-secret
 roleName: '{{.Release.Namespace}}'
 objects: 
   - objectName: postgresql-password
     secretPath: gabbar/data/postgres
     secretKey: postgresql-password

Step 3: Define volume in Helm values that use above created SecretProviderClass

deployment:
  volumes: 
    - name: postgres-secret
      csi:
        driver: secrets-store.csi.k8s.io
        readOnly: true
        volumeAttributes:
          secretProviderClass: postgres-secret

Step 4: Now mount this volume in container

volumeMounts:
- name: postgres-secret
  readOnly: true
  mountPath: /data/db-creds

Option # 2 - Consume Vault secret via Environment Variable

To mount Vault secret in an environment variable do following:

Step 1: Enable SecretProviderClass object in Helm values and define key/value path and secret objects in Vault. For example

secretProviderClass:
 enabled: true
 name: postgres-secret
 roleName: '{{.Release.Namespace}}'
 objects: 
   - objectName: postgresql-password
     secretPath: gabbar/data/postgres
     secretKey: postgresql-password
 secretObjects:
   - data:
     - key: postgres-password
       objectName: postgresql-password
     secretName: postgres-secret
     type: Opaque

The value of secretName will be the name of Kubernetes secret

Step 2: Define volume in Helm values that use above created SecretProviderClass

deployment:
  volumes: 
    - name: postgres-secret
      csi:
        driver: secrets-store.csi.k8s.io
        readOnly: true
        volumeAttributes:
          secretProviderClass: postgres-secret

Step 3: Now mount this volume in container.

volumeMounts:
- name: postgres-secret
  readOnly: true
  mountPath: /data/db-creds

Volume mount is required in order to create a Kubernetes secret.

Step 4: This secret can be used as environment variable

env:
   - name: POSTGRES_PASSWORD
     valueFrom:
        secretKeyRef:
            name: postgres-secret
            key: postgres-password

Here is a working example.

Your secret should be available at the path defined above in Vault; a change in secret value in Vault will automatically restart the application by Stakater Reloader

Option # 3 - Consume Vault secret via ExternalSecret

Kubernetes secret do not support storing or retrieving secret data from external secret management systems, e.g. HashiCorp Vault

External Secrets solves this problem by providing access to secrets stored externally. It does this by adding an ExternalSecret object to Kubernetes using a CustomResourceDefinition.

SAAP comes with fully managed External Secrets Operator to integrate with Vault and makes it extremely easy to consume secrets from Vault.

Step 1: Add tenant-vault-access template to the tenant

apiVersion: tenantoperator.stakater.com/v1alpha1
kind: Tenant
metadata:
  name: gabbar
spec:
  users:
    owner:
    - user1
    - user2
  quota: medium
  namespacetemplate:
    templateInstances:
    - spec:
        template: tenant-vault-access
        sync: true

TODO What will this template do? Who owns and manages this template? Is it owned by SAAP?

Step 2: Enable externalSecret in your deploy/values.yaml and provide details of the secret path in Vault.

externalSecret:
  enabled: true
  secretStore:
    name: tenant-vault-secret-store
  files:   
  inventory-postgres: #Name of Kubernetes Secret
    data:
      postgresql-password: #Name of Kubernetes Secret Key
        remoteRef:
          key: inventory-postgres #Name of Vault Secret
          property: postgresql-password #Name of Vault Secret Key

TODO So, when I update a secret in Vault; then is the application restarted automatically by Reloader?