Skip to content

CI/CD Pipeline Overview

Pipeline stages, deploy behavior, runner configuration, and GitLab API commands for managing pipelines across all CWIQ services.

Pipeline Stages

All platform services follow this stage order:

validate → test → build → release → deploy-dev → verify
Stage Purpose
validate Linting, formatting checks, schema validation
test Unit and integration tests
build Docker image build via Kaniko, push to Nexus
release Create GitLab release + tag (version tags only)
deploy-dev Deploy to DEV environment via SSH from K8s runner pod
verify Health checks, smoke tests, E2E tests

Deploy-dev Behavior

Trigger Behavior
Push to main Auto-deploys to DEV on every push — no manual trigger
Push to feature branch No auto-deploy. Use deploy-dev-standalone job (manual)

SSH_USER is a group-level variable

SSH_USER is defined at GitLab group level (group 9). Never define it in per-project .gitlab-ci.yml variables: sections — it will be silently overridden.

Deploy-dev jobs must use VPC private IPs

Runner pods execute on EKS (VPC CNI, subnets 10.1.34.0/24, 10.1.35.0/24). Tailscale is not available inside pods. All DEV_SERVER_IP / DEV_SERVER_HOST CI variables must point to VPC private IPs, not Tailscale IPs or hostnames.

Release Jobs

Release jobs create a GitLab release with release notes via release-cli.

  • Trigger condition: $CI_COMMIT_TAG =~ /^v\d+\.\d+\.\d+/
  • Stage: release (between build and deploy-dev)
  • Tags releases as {version} and stable on Docker images

GitLab Runner Architecture

See GitLab Runner Architecture for the full runner setup.

Active runners (since 2026-02-26):

Runner ID Tag Use
k8s-small 19 small Default jobs
k8s-medium 20 medium UI Kaniko builds (8GB RAM)
k8s-large 21 large Executor Nuitka + rpmbuild

Legacy Fleeting runners paused

Dev-small (16), dev-medium (17), dev-large (18) have been paused since 2026-02-27. Runner Manager EC2 (i-0af3f2d4bf8a4f1d2) is stopped.

GitLab CI Workflow Commands

Load the token once, then use it for all API calls:

TOKEN=$(grep GITLAB_PERSONAL_ACCESS_TOKEN .claude-env | cut -d= -f2 | tr -d '"' | tr -d "'" | tr -d ' ')
GL="https://gitlab.shared.cwiq.io/api/v4"

Check Latest Pipelines

# List last 3 pipelines for a project
curl -s "$GL/projects/{ID}/pipelines?per_page=3&private_token=$TOKEN" | \
  python3 -c "import sys,json; [print(p['id'], p['status'], p['ref'], p['created_at']) for p in json.load(sys.stdin)]"

Check Jobs in a Pipeline

curl -s "$GL/projects/{ID}/pipelines/{PIPELINE_ID}/jobs?private_token=$TOKEN" | \
  python3 -c "import sys,json; [print(j['name'], j['status']) for j in json.load(sys.stdin)]"

Get Job Log (Last 100 Lines)

curl -s "$GL/projects/{ID}/jobs/{JOB_ID}/trace?private_token=$TOKEN" | tail -100

Trigger a Full Pipeline on main

curl -s -X POST "$GL/projects/{ID}/pipeline?private_token=$TOKEN" -F "ref=main"

Trigger a Deploy-Only Pipeline (Skip Build)

Skips build stage, runs deploy-dev-standalone only:

curl -s -X POST "$GL/projects/{ID}/pipeline?private_token=$TOKEN" \
  -F "ref=main" -F "variables[DEPLOY_ONLY]=true"

Retry a Failed Pipeline

curl -s -X POST "$GL/projects/{ID}/pipelines/{PIPELINE_ID}/retry?private_token=$TOKEN"

Play a Manual Job

curl -s -X POST "$GL/projects/{ID}/jobs/{JOB_ID}/play?private_token=$TOKEN"

Common Project IDs

Service Project ID
server 5
ui 4
agent 6
mcp (agent-runner) 8
cli 7
executor 11
platform (parent) 3
runner-api 22
audit-consumer 23
audit-api 24
ai-catalogue-api 25
monitoring-api 26
notification-api 27
iam-api 28
cwiq-common 29
notification-worker 31
monitoring-worker 32
runner-worker 33

YAML Pitfalls

Colon-space in script lines causes YAML parse error

- echo "foo (version: ${VAR})" is parsed as a dict key. Avoid ": " in unquoted script lines. Wrap in - | block or reword.

GitLab CE does not support secrets: keyword

Use Vault JWT auth (id_tokens) instead.

when: manual is ignored when rules: is present

Put when: manual inside each rule block, not at the job level.

Script block isolation

Each - | block runs in a separate shell. Env vars set in one block are not available in the next. Merge into a single - | block if variables must persist.