Skip to content

Vault Secrets in CI/CD

CI/CD pipelines use Vault JWT authentication to securely fetch secrets at runtime.

How It Works

  1. GitLab generates a JWT token for each pipeline job
  2. The job sends this token to Vault's JWT auth endpoint
  3. Vault verifies the token and returns a Vault token scoped to the job's policies
  4. The job uses the Vault token to read secrets

This means no long-lived credentials are stored in GitLab CI/CD variables.

Pipeline Configuration

Vault Authentication Job

.vault-auth:
  before_script:
    # Authenticate to Vault using GitLab JWT
    - export VAULT_TOKEN=$(curl -s --request POST
        --data "{\"role\":\"$VAULT_ROLE\",\"jwt\":\"$CI_JOB_JWT_V2\"}"
        $VAULT_ADDR/v1/auth/jwt/login | python3 -c "import sys,json; print(json.load(sys.stdin)['auth']['client_token'])")

Using Secrets in a Job

deploy:
  extends: .vault-auth
  script:
    # Read a secret from Vault
    - export DB_PASSWORD=$(vault kv get -field=password secret/orchestrator/server/database)
    # Use the secret
    - deploy-app --db-password=$DB_PASSWORD

Vault Roles

Each project has a Vault JWT role that determines which secrets it can access:

Role Project Secret Paths
orchestrator-server server secret/orchestrator/server/*
orchestrator-ui ui secret/orchestrator/ui/*
orchestrator-ci all repos secret/ci/*, Nexus and SonarQube creds

Environment Variables

These GitLab CI/CD variables are set at the group level:

Variable Value Purpose
VAULT_ADDR https://vault.shared.cwiq.io Vault server URL
VAULT_ROLE Per-project role name JWT auth role

Adding a New Secret

  1. Store the secret in Vault:

    # Replace YOUR_REAL_API_KEY with the actual secret value
    vault kv put secret/orchestrator/server/new-service api_key=YOUR_REAL_API_KEY
    
  2. Ensure the project's Vault policy allows reading the path

  3. Reference in .gitlab-ci.yml:

    script:
      - export API_KEY=$(vault kv get -field=api_key secret/orchestrator/server/new-service)
    

Troubleshooting

"permission denied" in Vault auth

  • Verify VAULT_ADDR is set at the GitLab group level
  • Check the JWT role exists in Vault: vault read auth/jwt/role/$VAULT_ROLE
  • Ensure the role's bound_claims match the project path

"token expired" during job

  • Vault tokens have a TTL. For long-running jobs, re-authenticate mid-job
  • Check the role's token_ttl setting