How to automate secrets management in CI/CD pipelines with HashiCorp Vault

Step‑by‑step guide for DevOps engineers to securely automate secrets management in CI/CD using HashiCorp Vault, covering setup, rotation and compliance.

How to automate secrets management in CI/CD pipelines with HashiCorp Vault - Cybersecurity
Short answer: Automate secrets management in CI/CD by deploying HashiCorp Vault, configuring a secrets engine, granting Jenkins or GitHub Actions a short‑lived token via the AppRole auth method, and adding pipeline steps that fetch, use, and rotate secrets without ever storing them in code or environment files.

CI/CD pipelines need to access passwords, API keys, and certificates, but hard‑coding those values or checking them into source control creates a massive security risk. By integrating HashiCorp Vault directly into your build system, you can request secrets on demand, enforce lease expirations, and let Vault rotate credentials automatically.

After reading this guide you will be able to:

  • Install and initialise a Vault server suitable for a CI/CD environment.
  • Configure a KV secrets engine and an AppRole for Jenkins or GitHub Actions.
  • Add pipeline steps that retrieve secrets securely and trigger automated rotation.
  • Apply best‑practice controls such as minimal token TTL, audit logging, and secret revocation.

What You’ll Learn

  • Vault deployment options for DevOps teams.
  • AppRole authentication flow for CI agents.
  • Practical Jenkinsfile and GitHub Actions examples.
  • Automated secret rotation using Vault’s database and ssh engines.
  • Common pitfalls and how to avoid them.

Why automate secrets management?

Manual secret distribution leads to:

  • Stale credentials that linger after a developer leaves.
  • Accidental exposure in logs or artifact archives.
  • Compliance gaps for standards such as SOC 2 or PCI‑DSS.

Vault solves these problems by issuing short‑lived, revocable tokens and by rotating underlying credentials automatically. When the CI/CD runner authenticates, it receives a token with a configurable TTL (e.g., 10 minutes); the token is discarded after the job finishes, leaving no persistent secret on the build host.

1. Install & initialise Vault

Choose a deployment model

OptionUse caseLearning curvePricing modelLimits
Vault Open Source (binary) Small teams, self‑hosted VMs or Kubernetes Medium – requires storage backend config Free (self‑hosted) No built‑in HA; need external storage (Consul, Raft)
Vault Enterprise Large orgs, need namespaces, Sentinel policies High – additional features to learn Subscription (contact sales; at the time of writing – check pricing page) Built‑in HA, performance replication
HashiCorp Cloud Platform (HCP) Vault Teams that prefer managed service Low – UI wizard for cluster creation Pay‑as‑you‑go (per HCP instance) Limits defined by plan tier (e.g., 10 GiB storage on free tier)

Quick install on a Linux VM (Open Source)

curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo apt-key add -
sudo apt-add-repository "deb [arch=amd64] https://apt.releases.hashicorp.com $(lsb_release -cs) main"
sudo apt-get update && sudo apt-get install vault

Initialise and unseal

# Initialise with a single key share and a 7‑day root token TTL
vault operator init -key-shares=1 -key-threshold=1 -root-token-ttl=168h > init-output.txt

# Extract the unseal key and root token
UNSEAL_KEY=$(grep 'Unseal Key 1:' init-output.txt | awk '{print $NF}')
ROOT_TOKEN=$(grep 'Initial Root Token:' init-output.txt | awk '{print $NF}')

# Unseal
vault operator unseal $UNSEAL_KEY

# Export token for admin commands
export VAULT_TOKEN=$ROOT_TOKEN
Pro Tip: Store the unseal key and root token in a separate, encrypted password manager. Never commit them to source control.

2. Configure KV and rotation‑ready engines

Enable the KV v2 secrets engine

vault secrets enable -path=ci-kv kv-v2

Write a sample secret

vault kv put ci-kv/github token=ghp_<PLACEHOLDER>

Enable a database engine for automated rotation (example: PostgreSQL)

Assuming a PostgreSQL instance reachable at postgres://db-admin:password@db.example.com:5432/postgres:

vault secrets enable database
vault write database/config/postgresql \
    plugin_name=postgresql-database-plugin \
    connection_url="postgresql://{{username}}:{{password}}@db.example.com:5432/postgres?sslmode=disable" \
    allowed_roles="readonly"

vault write database/roles/readonly \
    db_name=postgresql \
    creation_statements="CREATE ROLE \"{{name}}\" WITH LOGIN PASSWORD '{{password}}' VALID UNTIL '{{expiration}}' IN ROLE readonly;" \
    default_ttl="1h" max_ttl="24h"

3. Set up AppRole for CI agents

Create a policy that limits access to needed paths

cat > ci-policy.hcl <<EOF
path "ci-kv/data/*" {
  capabilities = ["read"]
}
path "database/creds/readonly" {
  capabilities = ["read"]
}
EOF

vault policy write ci-policy ci-policy.hcl

Create the AppRole and retrieve its role_id and secret_id

vault auth enable approle

vault write auth/approle/role/ci-role \
    token_policies="ci-policy" \
    token_ttl="10m" token_max_ttl="30m" \
    secret_id_ttl="24h" secret_id_num_uses=5

ROLE_ID=$(vault read -field=role_id auth/approle/role/ci-role/role-id)
SECRET_ID=$(vault write -f -field=secret_id auth/approle/role/ci-role/secret-id)

echo "ROLE_ID=$ROLE_ID"
echo "SECRET_ID=$SECRET_ID"

Store ROLE_ID and SECRET_ID as protected credentials in your CI system (Jenkins Credentials, GitHub Actions secrets).

4. Jenkins pipeline integration

Install the HashiCorp Vault Plugin

Navigate to **Manage Jenkins → Manage Plugins → Available**, search for “HashiCorp Vault Plugin”, and install.

Define Vault credentials in Jenkins

  1. Go to **Credentials → System → Global credentials**.
  2. Add a “Secret text” credential for ROLE_ID and another for SECRET_ID.

Sample Jenkinsfile

pipeline {
    agent any
    environment {
        VAULT_ADDR = 'https://vault.example.com:8200'
        VAULT_ROLE_ID = credentials('vault-role-id')
        VAULT_SECRET_ID = credentials('vault-secret-id')
    }
    stages {
        stage('Fetch Secrets') {
            steps {
                // Authenticate and get a short‑lived token
                script {
                    def auth = sh(
                        script: """curl -s --request POST \\
                            --data '{"role_id":"${env.VAULT_ROLE_ID}","secret_id":"${env.VAULT_SECRET_ID}"}' \\
                            ${env.VAULT_ADDR}/v1/auth/approle/login""",
                        returnStdout: true
                    ).trim()
                    env.VAULT_TOKEN = readJSON text: auth).auth.client_token
                }

                // Pull a KV secret
                script {
                    def secret = sh(
                        script: """curl -s --header "X-Vault-Token: ${env.VAULT_TOKEN}" \\
                            ${env.VAULT_ADDR}/v1/ci-kv/data/github""",
                        returnStdout: true
                    ).trim()
                    env.GITHUB_TOKEN = readJSON(text: secret).data.data.token
                }

                // Pull a dynamic DB credential
                script {
                    def dbCred = sh(
                        script: """curl -s --header "X-Vault-Token: ${env.VAULT_TOKEN}" \\
                            ${env.VAULT_ADDR}/v1/database/creds/readonly""",
                        returnStdout: true
                    ).trim()
                    def creds = readJSON(text: dbCred).data
                    env.PGUSER = creds.username
                    env.PGPASSWORD = creds.password
                }
            }
        }
        stage('Run Tests') {
            steps {
                sh '''
                echo "Using GitHub token: $GITHUB_TOKEN"
                psql -h db.example.com -U $PGUSER -d mydb -c "SELECT 1"
                '''
            }
        }
    }
    post {
        always {
            // Revoke the token to avoid lingering leases
            sh "curl -s --request POST --header \"X-Vault-Token: ${env.VAULT_TOKEN}\" ${env.VAULT_ADDR}/v1/auth/token/revoke-self"
        }
    }
}

5. GitHub Actions integration

Store AppRole credentials as repository secrets

  • VAULT_ROLE_ID
  • VAULT_SECRET_ID
  • VAULT_ADDR (e.g., https://vault.example.com:8200)

Workflow example

name: CI with Vault

on: [push, pull_request]

jobs:
  build:
    runs-on: ubuntu-latest
    env:
      VAULT_ADDR: ${{ secrets.VAULT_ADDR }}
    steps:
      - uses: actions/checkout@v4

      - name: Authenticate to Vault
        id: vault-auth
        run: |
          AUTH=$(curl -s --request POST \
            --data '{"role_id":"${{ secrets.VAULT_ROLE_ID }}","secret_id":"${{ secrets.VAULT_SECRET_ID }}"}' \
            $VAULT_ADDR/v1/auth/approle/login)
          echo "VAULT_TOKEN=$(echo $AUTH | jq -r .auth.client_token)" >> $GITHUB_ENV

      - name: Retrieve GitHub token from KV
        id: get-secret
        run: |
          SECRET=$(curl -s --header "X-Vault-Token: $VAULT_TOKEN" \
            $VAULT_ADDR/v1/ci-kv/data/github)
          echo "GITHUB_TOKEN=$(echo $SECRET | jq -r .data.data.token)" >> $GITHUB_ENV

      - name: Run tests
        env:
          GITHUB_TOKEN: ${{ env.GITHUB_TOKEN }}
        run: |
          echo "Using token $GITHUB_TOKEN"
          # Example: npm test, go test, etc.

      - name: Revoke Vault token
        if: always()
        run: |
          curl -s --request POST --header "X-Vault-Token: $VAULT_TOKEN" \
            $VAULT_ADDR/v1/auth/token/revoke-self

6. Implement automated secret rotation

Dynamic credentials (already covered)

Database and SSH engines issue time‑bound credentials automatically; no extra script is required.

Rotating static KV secrets

Use a scheduled CI job or Vault’s periodic rotation feature (available in Enterprise). For Open Source, a simple cron job can re‑write the secret.

# Example cron on a bastion host (runs daily at 02:00 UTC)
0 2 * * * /usr/local/bin/vault kv put ci-kv/github token=$(curl -s -X POST https://api.github.com/... -H "Authorization: token $OLD_TOKEN" | jq -r .new_token)

Triggering rotation from pipeline

If a secret fails a health check, you can request a new version and invalidate the old one:

# In Jenkins or GitHub Actions
curl -s --header "X-Vault-Token: $VAULT_TOKEN" \
  -X POST $VAULT_ADDR/v1/ci-kv/delete/metadata/github
# Follow with a fresh write as shown above

7. DevOps secret handling best practice checklist

  • Never store raw secrets in source code, Dockerfiles, or CI environment variables.
  • Use short‑lived AppRole tokens (TTL ≤ 15 min) for build agents.
  • Enable audit logging in Vault (vault audit enable file path=/var/log/vault_audit.log).
  • Restrict policies to the minimum required paths.
  • Rotate static secrets regularly (at least every 30 days) and automate the process.
  • Run Vault in HA mode for production pipelines; consider HCP Vault for managed HA.
  • Revoke tokens explicitly in the pipeline post or finally step.

Frequently Asked Questions

Can I use Vault with Docker Compose?

Yes. Define a Vault container and a dependent service, then expose VAULT_ADDR via an environment variable. The official Docker image is documented here.

Do I need a separate Vault cluster for each environment (dev, staging, prod)?

Separate namespaces (Enterprise) or distinct clusters (Open Source) are recommended to avoid accidental cross‑environment secret leakage. Using HCP Vault you can create multiple clusters under the same organization.

What if my CI runner cannot reach Vault because of network policies?

Allow outbound HTTPS from the runner to the Vault address and open the Vault listener port (default 8200). If you use a private VPC, configure a VPC endpoint or VPN so the runner can reach the internal Vault address.

How does Vault handle secret versioning?

KV v2 stores each write as a new version. You can read a specific version via /data/<path>?version=<n> and delete old versions with /metadata/<path> APIs. This enables safe rollback if a rotation introduces a breaking change.

Is it safe to store the AppRole secret_id in GitHub Actions secrets?

GitHub Actions secrets are encrypted at rest and only exposed to the runner during the job. This meets most compliance requirements, but for highly regulated environments you may prefer a short‑lived secret_id generated by a separate automation (e.g., a Lambda that creates a secret_id per workflow run).

Can Vault rotate AWS IAM keys?

Yes. Enable the aws secrets engine and configure a role that creates IAM access keys with a limited TTL. See the official guide here.

What happens if a pipeline crashes before revoking its token?

Vault automatically expires tokens based on their TTL. Additionally, you can configure a maximum TTL (e.g., 30 minutes) so that orphaned tokens cannot linger indefinitely.

Official documentation