Skip to content

How to Use Reloader with AWS Secrets Manager and CSI Driver#

This guide shows how to mount secrets from AWS Secrets Manager into Kubernetes pods using the AWS Secrets and Configuration Provider (ASCP) and the Secrets Store CSI Driver, then use Reloader to automatically restart pods when those secrets change.

The ASCP syncs secrets into a Kubernetes Secret via secretObjects. Reloader watches that Kubernetes Secret and triggers a rolling restart whenever it is updated.

See also: ESO Pattern — if you are already using External Secrets Operator, that pattern requires no CSI Driver.


How It Works#

sequenceDiagram
    actor Ops as Operator / Rotation Job
    participant SM as AWS Secrets Manager
    participant ASCP as ASCP + CSI Driver
    participant K8s as Kubernetes Secret
    participant RL as Reloader
    participant Pod as Application Pod

    Ops->>SM: Rotate secret
    loop Every rotationPollInterval
        ASCP->>SM: GetSecretValue
        SM-->>ASCP: Updated secret value
        ASCP->>Pod: Refresh mounted files
        ASCP->>K8s: Sync via secretObjects
    end
    K8s-->>RL: Watch event (Secret changed)
    RL->>Pod: Trigger rolling restart
    Note over Pod: New pod starts with updated secret

Prerequisites#

  • Amazon EKS cluster (Kubernetes 1.17+)
  • AWS CLI configured locally
  • Helm v3+
  • An existing secret in AWS Secrets Manager
  • Stakater Reloader installed
  • OIDC provider associated with the cluster (for IRSA authentication)

Step 1 — Install the Secrets Store CSI Driver#

The CSI Driver must have syncSecret.enabled=true so that the ASCP can sync secrets into Kubernetes Secret objects.

helm repo add secrets-store-csi-driver https://kubernetes-sigs.github.io/secrets-store-csi-driver/charts
helm repo update
helm install csi-secrets-store secrets-store-csi-driver/secrets-store-csi-driver \
  --namespace kube-system \
  --set syncSecret.enabled=true \
  --set enableSecretRotation=true \
  --set rotationPollInterval=3600s

rotationPollInterval=3600s is a recommended starting point. AWS charges per API call to Secrets Manager; shorter intervals increase cost.


Step 2 — Install the AWS Secrets and Configuration Provider (ASCP)#

helm repo add aws-secrets-manager https://aws.github.io/secrets-store-csi-driver-provider-aws
helm repo update
helm install -n kube-system secrets-provider-aws \
  aws-secrets-manager/secrets-store-csi-driver-provider-aws

Step 3 — Install Reloader#

helm repo add stakater https://stakater.github.io/stakater-charts
helm repo update
helm install reloader stakater/reloader \
  --namespace reloader \
  --create-namespace

Step 4 — Create an IAM policy#

Create an IAM policy that allows the pod to read secrets from Secrets Manager.

aws iam create-policy \
  --policy-name ReloaderASCPPolicy \
  --policy-document '{
    "Version": "2012-10-17",
    "Statement": [{
      "Effect": "Allow",
      "Action": [
        "secretsmanager:GetSecretValue",
        "secretsmanager:DescribeSecret"
      ],
      "Resource": "arn:aws:secretsmanager:<region>:<account-id>:secret:<secret-name>-*"
    }]
  }'

Replace <region>, <account-id>, and <secret-name> with your values. The trailing -* accounts for the random suffix AWS appends to secret ARNs.


Step 5 — Set up IRSA (IAM Roles for Service Accounts)#

IRSA is the recommended authentication method for EKS. It binds an IAM role to a Kubernetes ServiceAccount using OIDC federation, so no static credentials are stored in the cluster.

Associate the OIDC provider with your cluster#

eksctl utils associate-iam-oidc-provider \
  --cluster <cluster-name> \
  --approve

Create an IAM role for the service account#

eksctl create iamserviceaccount \
  --name ascp-sa \
  --namespace default \
  --cluster <cluster-name> \
  --attach-policy-arn arn:aws:iam::<account-id>:policy/ReloaderASCPPolicy \
  --approve \
  --override-existing-serviceaccounts

This creates the ascp-sa ServiceAccount in the default namespace, annotated with the IAM role ARN:

metadata:
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::<account-id>:role/eksctl-<cluster>-addon-iamserviceaccount-Role

Step 6 — Create the SecretProviderClass#

The SecretProviderClass defines which secrets to fetch from Secrets Manager and how to sync them into a Kubernetes Secret via secretObjects.

Option A — JSON secret (use jmesPath to extract individual fields)#

If your secret is stored as a JSON string (for example, {"username":"admin","password":"secret123"}):

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: aws-app-secrets
  namespace: default
spec:
  provider: aws
  parameters:
    objects: |
      - objectName: "myapp/database"
        objectType: "secretsmanager"
        jmesPath:
          - path: username
            objectAlias: db-username
          - path: password
            objectAlias: db-password
  secretObjects:
    - secretName: app-secrets
      type: Opaque
      annotations:
        reloader.stakater.com/match: "true"
      data:
        - key: username
          objectName: db-username
        - key: password
          objectName: db-password

Apply:

kubectl apply -f secret-provider-class.yaml

Option B — Simple string secrets (one value per secret)#

If each secret contains a single string value:

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: aws-app-secrets
  namespace: default
spec:
  provider: aws
  parameters:
    objects: |
      - objectName: "myapp/db-username"
        objectType: "secretsmanager"
        objectAlias: db-username
      - objectName: "myapp/db-password"
        objectType: "secretsmanager"
        objectAlias: db-password
  secretObjects:
    - secretName: app-secrets
      type: Opaque
      annotations:
        reloader.stakater.com/match: "true"
      data:
        - key: username
          objectName: db-username
        - key: password
          objectName: db-password

Key configuration points:

Field Description
objectName Name or full ARN of the secret in Secrets Manager
objectType secretsmanager for Secrets Manager; ssmparameter for Parameter Store
objectAlias Filename on the CSI volume mount, and reference key in secretObjects
jmesPath[].path Key to extract from a JSON-formatted secret value
jmesPath[].objectAlias Filename and objectName reference for the extracted field
secretObjects[].secretName Name of the Kubernetes Secret to create
secretObjects[].data[].objectName Must match objectAlias from the objects list

Step 7 — Deploy the application#

The CSI volume mount is required even when your application reads secrets from environment variables. Without the mount, the ASCP does not sync the secretObjects.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp
  namespace: default
  annotations:
    reloader.stakater.com/search: "true"
spec:
  replicas: 2
  selector:
    matchLabels:
      app: myapp
  template:
    metadata:
      labels:
        app: myapp
    spec:
      serviceAccountName: ascp-sa
      containers:
        - name: app
          image: busybox:latest
          command: ["sh", "-c", "while true; do echo \"user=$DB_USERNAME\"; sleep 30; done"]
          env:
            - name: DB_USERNAME
              valueFrom:
                secretKeyRef:
                  name: app-secrets
                  key: username
            - name: DB_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: app-secrets
                  key: password
          volumeMounts:
            - name: aws-secrets
              mountPath: /mnt/secrets
              readOnly: true
      volumes:
        - name: aws-secrets
          csi:
            driver: secrets-store.csi.k8s.io
            readOnly: true
            volumeAttributes:
              secretProviderClass: aws-app-secrets

Apply:

kubectl apply -f deployment.yaml

Step 8 — Verify the setup#

# Check the pod is running
kubectl get pods -n default -l app=myapp

# Confirm the Kubernetes Secret was created and contains data
kubectl get secret app-secrets -n default
kubectl get secret app-secrets -n default -o jsonpath='{.data.username}' | base64 -d

# Confirm the CSI volume is mounted
kubectl get secretproviderclasspodstatuses -n default

Step 9 — Test secret rotation#

# Rotate the secret in Secrets Manager
aws secretsmanager put-secret-value \
  --secret-id myapp/database \
  --secret-string '{"username":"admin","password":"new-rotated-password"}'

Wait for the next rotationPollInterval cycle. The CSI driver polls Secrets Manager, updates the mounted files, and syncs the new value into the Kubernetes Secret. Reloader detects the Secret change and triggers a rolling restart.

# Confirm the K8s Secret was updated
kubectl get secret app-secrets -n default -o jsonpath='{.data.password}' | base64 -d

# Confirm pods were restarted (check pod age)
kubectl get pods -n default -l app=myapp

Reloader annotations#

Resource Annotation Effect
Deployment reloader.stakater.com/search: "true" Restart when any referenced Secret with match: "true" changes
Kubernetes Secret (via secretObjects) reloader.stakater.com/match: "true" Mark this Secret as eligible to trigger a restart

File-based pattern (no Kubernetes Secret)#

If you want to mount secrets as files only, without creating a Kubernetes Secret, omit the secretObjects block and enable Reloader's CSI integration:

reloader:
  enableCSIIntegration: true

Then annotate the Deployment with the SecretProviderClass name:

metadata:
  annotations:
    secretproviderclass.reloader.stakater.com/reload: "aws-app-secrets"

In this mode, Reloader watches SecretProviderClassPodStatus resources for version hash changes instead of watching a Kubernetes Secret. This approach avoids creating intermediate Secret objects but requires enableCSIIntegration: true in Reloader.