Skip to content

SSL: ACM Import

Two services — Authentik (sso.shared.cwiq.io) and GitLab Shared (gitlab.shared.cwiq.io) — sit behind AWS Application Load Balancers. Their SSL certificates are imported to AWS Certificate Manager (ACM) for termination at the ALB level rather than being deployed directly to EC2 containers.

Why ACM for These Services

Service Why ALB + ACM
Authentik SSO HA setup: 2 EC2 instances behind an internal ALB. External access via ALB requires ACM.
GitLab Shared Production GitLab with public internet access. ALB provides DDoS protection and SSL offloading.

For all other CWIQ services (Tailscale-only, direct EC2), certificates are deployed directly to the server. ACM is only needed when an ALB terminates TLS on behalf of the service.

See SSL: Architecture for the direct EC2 vs ALB+ACM comparison.

ACM Import Playbook

The acm-import.yml playbook reads the certificate from /etc/letsencrypt/live/<domain>/ on the cert-server and imports it to AWS ACM in us-west-2.

# Run from the cert-server
ssh ansible@ansible-shared-cwiq-io
cd /data/ansible/cwiq-ansible-playbooks/cert-server

# Import Authentik cert
ansible-playbook -i inventory.yml acm-import.yml -e "cert_domain=sso.shared.cwiq.io"

# Import GitLab cert
ansible-playbook -i inventory.yml acm-import.yml -e "cert_domain=gitlab.shared.cwiq.io"

What the Playbook Does

  1. Reads fullchain.pem, privkey.pem, and chain.pem from /etc/letsencrypt/live/<cert_domain>/ on the cert-server
  2. Calls aws acm list-certificates to check for an existing certificate with the same domain name
  3. If a matching cert exists, re-imports (updates) it using the existing ARN
  4. If no matching cert exists, creates a new ACM certificate entry
  5. Tags the certificate with ManagedBy=cert-server

The ALB listener is pre-configured to reference the ACM certificate ARN. After a re-import, the ALB automatically serves the updated certificate on the next TLS handshake — no ALB reconfiguration is needed.

Force Re-Import

If the standard import does not pick up a renewed certificate:

ansible-playbook -i inventory.yml acm-import.yml \
  -e "cert_domain=sso.shared.cwiq.io" \
  -e "force_reimport=true"

Automated Import on Renewal

The ssl-renew-deploy.yml playbook automatically imports to ACM as part of the renewal pipeline. After certbot renew and ssl-deploy-all.yml, it imports both ALB-backed domains:

# ssl-renew-deploy.yml (excerpt)
- name: Import Authentik certificate to AWS ACM
  import_playbook: acm-import.yml
  vars:
    cert_domain: sso.shared.cwiq.io
    acm_region: us-west-2

- name: Import GitLab certificate to AWS ACM
  import_playbook: acm-import.yml
  vars:
    cert_domain: gitlab.shared.cwiq.io
    acm_region: us-west-2

This runs twice daily via the systemd timer. No manual intervention is needed under normal operation.

Authentik: Dual Deployment

Authentik gets both EC2 and ACM

Authentik is the only service that receives the certificate in both places:

  1. EC2 deployment — ssl-deploy-all.yml copies the cert to /data/ssl/sso.shared.cwiq.io/ on both authentik-shared-cwiq-io-1 and authentik-shared-cwiq-io-2. The cert is mounted into the Authentik server container.
  2. ACM import — acm-import.yml imports the same cert to ACM for the Authentik ALB.

The ALB uses ACM for external TLS termination. The EC2 cert is present for any internal operations that reference the certificate directly (such as SAML signing).

Authentik uses RSA — not ECDSA

sso.shared.cwiq.io is issued as an RSA certificate. AWS Identity Center (which federates through Authentik SAML) requires RSA for SAML signing keys. The cert_domains entry in group_vars/all.yml specifies key_type: rsa for this domain only.

GitLab Shared: ACM Only

Unlike Authentik, GitLab Shared receives the certificate in ACM only. The ALB terminates TLS and forwards plain HTTP to GitLab on port 80. GitLab is configured to trust the ALB's X-Forwarded-* headers for correct protocol and IP reporting.

Aspect GitLab Dev GitLab Shared
SSL termination GitLab nginx (direct) AWS ALB
Certificate storage /data/ssl/gitlab.dev.cwiq.io/ on EC2 ACM (us-west-2)
GitLab port 443 (HTTPS) 80 (HTTP from ALB)
Public internet access No (Tailscale only) Yes (via ALB)
ACM import needed No Yes

IAM Requirements

The cert-server EC2 instance role must have these ACM permissions:

{
  "Effect": "Allow",
  "Action": [
    "acm:ImportCertificate",
    "acm:ListCertificates",
    "acm:DescribeCertificate",
    "acm:AddTagsToCertificate"
  ],
  "Resource": "*"
}

The cert-server also needs Route53 permissions for DNS-01 challenge validation. Both permission sets are attached to the cert-server's IAM instance role in Terraform (terraform-plan/organization/environments/shared-services/).

Manual ACM Operations

# List all ACM certificates in us-west-2
aws acm list-certificates --region us-west-2 --profile shared-services

# Check certificate status and expiry
aws acm describe-certificate \
  --certificate-arn <arn> \
  --region us-west-2 \
  --profile shared-services \
  --query "Certificate.{Domain:DomainName,Status:Status,NotAfter:NotAfter}"

# Verify the ALB is using the expected certificate
aws elbv2 describe-listeners \
  --load-balancer-arn <alb_arn> \
  --region us-west-2 \
  --profile shared-services \
  --query "Listeners[*].Certificates"

When to Run ACM Import Manually

The automated renewal pipeline handles ACM imports. Manually run acm-import.yml only in these situations:

Situation Command
First-time setup of a new ALB-backed service ansible-playbook -i inventory.yml acm-import.yml -e "cert_domain=<domain>"
ALB is showing an expired certificate after renewal ansible-playbook -i inventory.yml acm-import.yml -e "cert_domain=<domain>"
ACM certificate was accidentally deleted ansible-playbook -i inventory.yml acm-import.yml -e "cert_domain=<domain>"
Certificate was force-renewed with --force-renewal Run acm-import.yml after certbot completes

Adding a New ALB-Backed Service

If a new service is deployed behind an ALB:

  1. Issue the certificate: ansible-playbook ssl-issue-all.yml (after adding the domain to cert_domains)
  2. Import to ACM: ansible-playbook -i inventory.yml acm-import.yml -e "cert_domain=<domain>"
  3. Configure the ALB listener to use the ACM certificate ARN (via Terraform)
  4. Add the domain to ssl-renew-deploy.yml as a new acm-import.yml step
  5. Update SSL: Inventory with the new service