api7-demo/web/docs/cicd-pipeline.md

CI/CD Pipeline

Automated build and deployment pipeline using Git-based CI/CD systems.

Overview

The CI/CD pipeline automatically builds Docker images and pushes them to the container registry whenever code is pushed to any branch.

Repository: Git repository with application code

Registry: Container registry for Docker images

Pipeline Architecture

Git Push (any branch)
   ↓
CI/CD System Trigger
   ↓
┌─────────────────────────────────────┐
│  Job: build-web (parallel)          │
│  Job: build-api (parallel)          │
└─────────────────────────────────────┘
   ↓                    ↓
Build Web Image     Build API Image
   ↓                    ↓
Tag with branch     Tag with branch
   ↓                    ↓
Push to Registry    Push to Registry
   ↓                    ↓
<registry>/web:<branch>
<registry>/api:<branch>

Workflow Configuration

Location: CI/CD workflow files (.github/workflows/, .gitlab-ci.yml, etc.)

name: Build and Push Docker Images

on:
  push:
  workflow_dispatch:

jobs:
  build-web:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Login to Container Registry
        uses: docker/login-action@v3
        with:
          registry: <registry-url>
          username: ${{ secrets.REGISTRY_USER }}
          password: ${{ secrets.REGISTRY_TOKEN }}

      - name: Extract metadata for web image
        id: meta-web
        uses: docker/metadata-action@v5
        with:
          images: <registry>/web
          tags: |
            type=ref,event=branch

      - name: Build and push web image
        uses: docker/build-push-action@v5
        with:
          context: ./web
          file: ./web/Dockerfile
          push: true
          tags: ${{ steps.meta-web.outputs.tags }}
          labels: ${{ steps.meta-web.outputs.labels }}
          cache-from: type=registry,ref=<registry>/web:buildcache
          cache-to: type=registry,ref=<registry>/web:buildcache,mode=max

  build-api:
    runs-on: ubuntu-latest
    steps:
      # Similar steps for API application
      ...

Trigger Events

Automatic Triggers

Push to any branch:

git push origin <branch-name>

Images tagged with branch name:

• <registry>/web:<branch-name>
• <registry>/api:<branch-name>

Manual Trigger

Via CI/CD Interface:

1. Go to repository → CI/CD or Actions
2. Select workflow: Build and Push Docker Images
3. Click: Run workflow
4. Select branch
5. Click: Run

Via Git:

# Trigger workflow with empty commit
git commit --allow-empty -m "Trigger CI/CD pipeline"
git push

Setup Instructions

1. Create Registry Token

Generate Token:

1. Go to: User Settings or Access Tokens
2. Click: Generate New Token
3. Name: container-registry
4. Select appropriate permissions:
   • ✅ Write/push access (required)
   • ✅ Read/pull access (required)
5. Click: Generate Token
6. Copy token (shown once only)

2. Add Repository Secret

In Repository Settings:

1. Go to: Repository → Settings → Secrets/Variables
2. Click: Add Secret
3. Add required secrets:
   • REGISTRY_TOKEN: Token from step 1
   • REGISTRY_USER: Username
4. Click: Save

3. Verify Workflow

Check workflow file exists:

ls -la .github/workflows/  # or appropriate CI/CD directory

Push to trigger:

git add .
git commit -m "Test CI/CD pipeline"
git push origin main

Monitor execution:

1. Go to repository → CI/CD or Actions
2. Click on the running workflow
3. View logs for each job

Image Naming Convention

Format: <registry>/<namespace>/<app>:<tag>

Examples:

• <registry>/web:main
• <registry>/web:develop
• <registry>/web:feature-xyz
• <registry>/api:main

Using Built Images in Kubernetes

1. Pull Images from Container Registry

Create registry secret:

kubectl create secret docker-registry registry-secret \
  --docker-server=<registry-url> \
  --docker-username=<USERNAME> \
  --docker-password=<TOKEN> \
  -n <namespace>

2. Update Deployment

deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
  namespace: <namespace>
spec:
  template:
    spec:
      imagePullSecrets:
      - name: registry-secret
      containers:
      - name: web
        image: <registry>/web:main
        imagePullPolicy: Always

3. Deploy to Kubernetes

kubectl apply -f deployment.yaml

4. Rollout New Version

After pipeline builds new image:

# Force rollout restart (pulls latest image with same tag)
kubectl rollout restart deployment/web -n <namespace>
kubectl rollout restart deployment/api -n <namespace>

# Check rollout status
kubectl rollout status deployment/web -n <namespace>

Or update image explicitly:

kubectl set image deployment/web \
  web=<registry>/web:main \
  -n <namespace>

kubectl set image deployment/api \
  api=<registry>/api:main \
  -n <namespace>

Docker Layer Caching

The pipeline uses Docker layer caching to speed up builds:

Cache Configuration:

cache-from: type=registry,ref=<registry>/web:buildcache
cache-to: type=registry,ref=<registry>/web:buildcache,mode=max

Benefits:

• Faster builds (reuses unchanged layers)
• Reduced build time from ~3min to ~30sec
• Lower resource usage

Cache Location: Stored in registry as special tag :buildcache

Parallel Builds

Both applications build in parallel:

Start
  ├─→ build-web job   (3-5 min)
  └─→ build-api job   (3-5 min)
         ↓
    Complete (~5 min total)

Without parallelization: ~10 minutes With parallelization: ~5 minutes

Viewing Build Logs

Via CI/CD Interface

1. Repository → CI/CD or Actions
2. Click on workflow run
3. Select job (build-web or build-api)
4. View step-by-step logs

Via CLI (if available)

# List workflow runs
<cli-tool> list runs

# View specific run logs
<cli-tool> view run <run-id>

Troubleshooting

Build Fails: Authentication Error

Error: unauthorized: authentication required

Solution:

1. Verify registry secrets exist in repository settings
2. Check token has appropriate permissions
3. Regenerate token if expired

# Test token manually
docker login <registry-url> -u <USERNAME> -p <TOKEN>

Build Fails: Dockerfile Not Found

Error: unable to prepare context: unable to evaluate symlinks in Dockerfile path

Solution:

1. Check Dockerfile exists in correct location:
   • web/Dockerfile
   • api/Dockerfile
2. Verify workflow context path matches:

   context: ./web
   file: ./web/Dockerfile
   

Image Pull Fails in Kubernetes

Error: ImagePullBackOff

Solution:

1. Verify registry secret exists:

   kubectl get secret registry-secret -n <namespace>
   

2. Check secret is referenced in deployment:

   spec:
     template:
       spec:
         imagePullSecrets:
         - name: registry-secret
   

3. Test manual pull:

   docker pull <registry>/web:main
   

Workflow Doesn't Trigger

Check:

1. ✅ Workflow file in appropriate CI/CD directory
2. ✅ File is named correctly (.yml or .yaml)
3. ✅ File is committed and pushed to repository
4. ✅ CI/CD is enabled in repository settings

Force trigger:

git commit --allow-empty -m "Trigger workflow"
git push

Best Practices

Version Tagging

For production deployments:

# Tag release
git tag -a v1.0.0 -m "Release v1.0.0"
git push origin v1.0.0

Update workflow for semantic versioning:

tags: |
  type=ref,event=branch
  type=semver,pattern={{version}}
  type=semver,pattern={{major}}.{{minor}}

Branch Strategy

Recommended:

• main → production images
• develop → staging images
• feature/* → development images

Kubernetes deployments:

# Production
image: <registry>/web:main

# Staging
image: <registry>/web:develop

# Development
image: <registry>/web:feature-xyz

Security

Protect main branch:

1. Repository Settings → Branches or Protected Branches
2. Add branch protection for main
3. Require pull/merge request reviews
4. Enable status checks

Rotate tokens regularly:

• Generate new registry token every 90 days
• Update secrets in repository settings

Advanced Configuration

Multi-Stage Builds

Optimize Dockerfile for smaller images:

# Build stage
FROM python:3.11-slim AS builder
WORKDIR /app
COPY requirements.txt .
RUN pip install --user --no-cache-dir -r requirements.txt

# Runtime stage
FROM python:3.11-slim
WORKDIR /app
COPY --from=builder /root/.local /root/.local
COPY . .
ENV PATH=/root/.local/bin:$PATH
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

Build Arguments

Pass build-time variables:

- name: Build and push
  uses: docker/build-push-action@v5
  with:
    build-args: |
      VERSION=${{ github.sha }}
      BUILD_DATE=${{ github.event.head_commit.timestamp }}

Matrix Builds

Build for multiple platforms:

strategy:
  matrix:
    platform: [linux/amd64, linux/arm64]

- name: Build and push
  uses: docker/build-push-action@v5
  with:
    platforms: ${{ matrix.platform }}

Monitoring Pipeline

Metrics to Track

• ✅ Build success rate
• ✅ Average build time
• ✅ Cache hit rate
• ✅ Image size trends

Notifications

Add Discord/Slack notifications (example):

- name: Notify on failure
  if: failure()
  run: |
    curl -X POST ${{ secrets.WEBHOOK_URL }} \
      -H 'Content-Type: application/json' \
      -d '{"content":"Build failed for ${{ github.ref }}"}'

---

Automated CI/CD pipeline for building and deploying containerized applications.