CI/CD Pipelines, GitOps & Progressive Delivery
Version: 2.0.0
Purpose: Canonical standalone hands-on lab structure.
Required Inputs: Associated lesson, lab objective, environment details.
Outputs: Reproducible, independently testable hands-on lab markdown.
Lab Metadata
- Lab ID:
LAB-CICD-01 - Associated Lesson: Module 11 (
MOD-CICD: Continuous Integration vs. Continuous Delivery Paradigms, GitHub Actions, Container Pipelines, Progressive Delivery) - Objective: Author a declarative GitHub Actions workflow with OIDC federation, create a Multi-Stage Dockerfile with BuildKit cache mounts, simulate Trivy vulnerability quality gates, and author an Argo Rollouts Canary manifest with an AnalysisTemplate.
- Estimated Time: 45 minutes
- Difficulty: Intermediate to Advanced
Prerequisites
- Completion of Module 05 (
MOD-GIT: Version Control & GitOps) and Module 10 (MOD-K8S: Kubernetes Engineering). - Foundational understanding of YAML syntax, Linux bash execution, container images, and Kubernetes Services.
- Access to a local bash terminal environment (with standard tools like
mkdir,cat, andgrep).
Environment Setup
Prepare your local terminal sandbox environment by setting up the required directory structure for your CI/CD and progressive delivery manifests.
# Create the parent directory for the CI/CD and GitOps lab manifests
mkdir -p ~/enterprise-cicd-lab/.github/workflows
mkdir -p ~/enterprise-cicd-lab/k8s-manifests
cd ~/enterprise-cicd-lab
# Verify the directory structure was created successfully
pwdStep-by-Step Instructions
Step 1: Authoring a Declarative GitHub Actions Workflow with OIDC Federation
In this step, you will author a declarative GitHub Actions workflow file (deploy.yml) that implements absolute zero-trust cloud authentication using OpenID Connect (OIDC) identity federation (sts:AssumeRoleWithWebIdentity) and defines strict path filtering (paths:).
cat << 'EOF' > .github/workflows/deploy.yml
name: Production Microservice CI/CD
on:
push:
branches: [ main ]
paths:
- 'src/**'
- 'Dockerfile'
workflow_dispatch:
inputs:
environment:
description: 'Target deployment environment'
required: true
default: 'staging'
permissions:
id-token: write # Mandatory permission required for OIDC JWT token generation!
contents: read
jobs:
build-and-push:
runs-on: ubuntu-latest
steps:
- name: Checkout Source Code
uses: actions/checkout@v4
- name: Configure AWS Credentials via OIDC Federation
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::123456789012:role/GitHubActionsECRPushRole
aws-region: us-east-1
audience: sts.amazonaws.com
- name: Log in to Amazon ECR
id: login-ecr
uses: aws-actions/amazon-ecr-login@v2
- name: Build, Tag, and Push Immutable Container Artifact
env:
ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
ECR_REPOSITORY: production-payment-api
IMAGE_TAG: git-${{ github.sha }}
run: |
echo "Building immutable container image $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG..."
# docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG .
# docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
echo "SUCCESS: Immutable artifact successfully built and pushed to ECR!"
EOFStep 2: Creating a Multi-Stage Dockerfile with BuildKit Cache Mounts
In this step, you will author a highly optimized Multi-Stage Dockerfile manifest (Dockerfile) that leverages BuildKit cache mounts (--mount=type=cache) to accelerate compilation speeds and utilizes a pristine, distroless runtime stage to eliminate heavy build tools (gcc, curl).
cat << 'EOF' > Dockerfile
# ==============================================================================
# STAGE 1: BUILDER ENVIRONMENT (Heavy compilation tools & BuildKit Cache)
# ==============================================================================
FROM golang:1.22-alpine AS builder
RUN apk add --no-cache git build-base
WORKDIR /build
COPY go.mod go.sum ./
RUN --mount=type=cache,target=/go/pkg/mod \
go mod download
COPY . .
RUN --mount=type=cache,target=/go/pkg/mod \
--mount=type=cache,target=/root/.cache/go-build \
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -a -ldflags '-extldflags "-static"' -o /build/payment-api .
# ==============================================================================
# STAGE 2: PRODUCTION RUNTIME ENVIRONMENT (Pristine & Distroless)
# ==============================================================================
FROM gcr.io/distroless/static:nonroot
WORKDIR /app
# Copy EXCLUSIVELY the compiled static application binary from Stage 1!
COPY --from=builder --chown=nonroot:nonroot /build/payment-api /app/payment-api
USER nonroot:nonroot
EXPOSE 8080
ENTRYPOINT ["/app/payment-api"]
EOFStep 3: Simulating a Trivy Vulnerability Quality Gate and Cosign Image Signing
In this step, you will create and execute a simulation script (simulate-pipeline.sh) that verifies how Trivy container vulnerability scanning quality gates (--exit-code 1) and Cosign keyless cryptographic image signing (cosign sign) operate in an enterprise CI pipeline.
cat << 'EOF' > simulate-pipeline.sh
#!/bin/bash
set -e
echo "================================================================================"
echo "[PIPELINE STEP 1]: Executing Multi-Stage Docker Build..."
echo "================================================================================"
echo "Successfully built mycompany/payment-api:git-7b89c12 (Size: 18MB)"
echo "================================================================================"
echo "[PIPELINE STEP 2]: Executing Trivy container vulnerability quality gate..."
echo "================================================================================"
echo "trivy image --exit-code 1 --severity CRITICAL,HIGH mycompany/payment-api:git-7b89c12"
echo "2026-06-28T12:30:00.000Z INFO Vulnerability scanning..."
echo "mycompany/payment-api:git-7b89c12 (distroless static)"
echo "====================================================="
echo "Total: 0 (CRITICAL: 0, HIGH: 0)"
echo "SUCCESS: Zero vulnerabilities detected. Quality gate passed perfectly!"
echo "================================================================================"
echo "[PIPELINE STEP 3]: Executing Cosign keyless cryptographic image signing..."
echo "================================================================================"
echo "cosign sign --yes mycompany/payment-api:git-7b89c12"
echo "Generating ephemeral cryptographic key pair via Sigstore OIDC federation..."
echo "Retrieving ephemeral signing certificate from Fulcio Root CA..."
echo "Recording immutable signing transparency log entry in Rekor (Index: 987654321)..."
echo "Pushing cryptographic signature attachment to OCI registry..."
echo "SUCCESS: Container image successfully signed with keyless provenance!"
EOF
# Grant execution permissions and run the simulation script
chmod +x simulate-pipeline.sh
./simulate-pipeline.shStep 4: Authoring an Argo Rollouts Canary Manifest and AnalysisTemplate
In this step, you will author a declarative Argo Rollouts Canary manifest (k8s-manifests/canary-rollout.yaml) that defines fine-grained percentage-based traffic shaping (setWeight: 10) and pairs it with a declarative AnalysisTemplate (k8s-manifests/metric-analysis.yaml) to evaluate Prometheus HTTP success rates during the rollout.
cat << 'EOF' > k8s-manifests/canary-rollout.yaml
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
name: production-payment-api
namespace: production
spec:
replicas: 10
strategy:
canary:
stableService: payment-api-stable
canaryService: payment-api-canary
trafficRouting:
nginx:
stableIngress: payment-api-ingress
steps:
- setWeight: 10
- pause: { duration: 5m }
- analysis:
templates:
- templateName: verify-http-success-rate
- setWeight: 50
- pause: { duration: 5m }
selector:
matchLabels:
app: payment-api
template:
metadata:
labels:
app: payment-api
spec:
containers:
- name: api
image: mycompany/payment-api:git-7b89c12
ports:
- containerPort: 8080
EOF
cat << 'EOF' > k8s-manifests/metric-analysis.yaml
apiVersion: argoproj.io/v1alpha1
kind: AnalysisTemplate
metadata:
name: verify-http-success-rate
namespace: production
spec:
metrics:
- name: success-rate
interval: 30s
count: 10
successCondition: result[0] >= 0.99
failureLimit: 2
provider:
prometheus:
address: http://prometheus-k8s.monitoring.svc.cluster.local:9090
query: >+
sum(rate(http_requests_total{job="payment-api-canary",status=~"2.*|3.*"}[1m]))
/
sum(rate(http_requests_total{job="payment-api-canary"}[1m]))
EOFVerification
To verify that your enterprise CI/CD and progressive delivery lab was completed successfully, execute the following verification commands to inspect your manifest contents and verify OIDC permissions, Multi-Stage copying, Trivy exit codes, and Canary traffic shaping.
# 1. Verify OIDC permissions in the GitHub Actions workflow manifest
cat .github/workflows/deploy.yml | grep -E "id-token.*write"
# 2. Verify Multi-Stage copying in the Dockerfile manifest
cat Dockerfile | grep -E "COPY --from=builder"
# 3. Verify Trivy vulnerability exit codes in the simulation script
cat simulate-pipeline.sh | grep -E "trivy.*exit-code 1"
# 4. Verify Canary traffic weights in the Argo Rollouts manifest
cat k8s-manifests/canary-rollout.yaml | grep -E "setWeight: 10"
# 5. Verify Prometheus metric success conditions in the AnalysisTemplate manifest
cat k8s-manifests/metric-analysis.yaml | grep -E "successCondition: result.*0.99"Expected Output:
id-token: write # Mandatory permission required for OIDC JWT token generation!
COPY --from=builder --chown=nonroot:nonroot /build/payment-api /app/payment-api
echo "trivy image --exit-code 1 --severity CRITICAL,HIGH mycompany/payment-api:git-7b89c12"
- setWeight: 10
successCondition: result[0] >= 0.99Troubleshooting
-
Symptom:
cat .github/workflows/deploy.yml | grep -E "id-token.*write"returns no output.- Cause: You completely omitted the
permissions:block at the top of your workflow YAML file. - Solution: Add
permissions: id-token: writeto your workflow manifest to ensure GitHub generates an OIDC JWT token.
- Cause: You completely omitted the
-
Symptom:
cat Dockerfile | grep -E "COPY --from=builder"returns no output.- Cause: You authored a single-stage Dockerfile or forgot to name your first stage
AS builder. - Solution: Ensure your Dockerfile contains
FROM golang:1.22-alpine AS builderandCOPY --from=builder ....
- Cause: You authored a single-stage Dockerfile or forgot to name your first stage
-
Symptom: Executing
./simulate-pipeline.shfails withbash: ./simulate-pipeline.sh: Permission denied.- Cause: You completely forgot to make the shell script executable.
- Solution: Execute
chmod +x simulate-pipeline.shto grant execution permissions.
Cleanup
Safely remove the enterprise CI/CD lab directory and temporary manifest files from your terminal environment.
# Safely remove the enterprise CI/CD lab directory
rm -rf ~/enterprise-cicd-lab
# Verify the directory was removed successfully
ls -la ~/enterprise-cicd-lab 2>/dev/null || echo "Cleanup complete. Directory successfully removed."