api7-demo/web/docs/cicd-pipeline.md

# CI/CD Pipeline

Automated build and deployment pipeline using Git-based CI/CD systems.

## Overview

The CI/CD pipeline automatically builds Docker images and pushes them to the container registry whenever code is pushed to any branch.

**Repository**: Git repository with application code

**Registry**: Container registry for Docker images

## Pipeline Architecture

```
Git Push (any branch)
   ↓
CI/CD System Trigger
   ↓
┌─────────────────────────────────────┐
│  Job: build-web (parallel)          │
│  Job: build-api (parallel)          │
└─────────────────────────────────────┘
   ↓                    ↓
Build Web Image     Build API Image
   ↓                    ↓
Tag with branch     Tag with branch
   ↓                    ↓
Push to Registry    Push to Registry
   ↓                    ↓
<registry>/web:<branch>
<registry>/api:<branch>
```

## Workflow Configuration

**Location**: CI/CD workflow files (`.github/workflows/`, `.gitlab-ci.yml`, etc.)

```yaml
name: Build and Push Docker Images

on:
  push:
  workflow_dispatch:

jobs:
  build-web:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Login to Container Registry
        uses: docker/login-action@v3
        with:
          registry: <registry-url>
          username: ${{ secrets.REGISTRY_USER }}
          password: ${{ secrets.REGISTRY_TOKEN }}

      - name: Extract metadata for web image
        id: meta-web
        uses: docker/metadata-action@v5
        with:
          images: <registry>/web
          tags: |
            type=ref,event=branch

      - name: Build and push web image
        uses: docker/build-push-action@v5
        with:
          context: ./web
          file: ./web/Dockerfile
          push: true
          tags: ${{ steps.meta-web.outputs.tags }}
          labels: ${{ steps.meta-web.outputs.labels }}
          cache-from: type=registry,ref=<registry>/web:buildcache
          cache-to: type=registry,ref=<registry>/web:buildcache,mode=max

  build-api:
    runs-on: ubuntu-latest
    steps:
      # Similar steps for API application
      ...
```

## Trigger Events

### Automatic Triggers

**Push to any branch**:
```bash
git push origin <branch-name>
```

Images tagged with branch name:
- `<registry>/web:<branch-name>`
- `<registry>/api:<branch-name>`

### Manual Trigger

**Via CI/CD Interface**:
1. Go to repository → **CI/CD** or **Actions**
2. Select workflow: **Build and Push Docker Images**
3. Click: **Run workflow**
4. Select branch
5. Click: **Run**

**Via Git**:
```bash
# Trigger workflow with empty commit
git commit --allow-empty -m "Trigger CI/CD pipeline"
git push
```

## Setup Instructions

### 1. Create Registry Token

**Generate Token**:
1. Go to: **User Settings** or **Access Tokens**
2. Click: **Generate New Token**
3. Name: `container-registry`
4. Select appropriate permissions:
   - ✅ Write/push access (required)
   - ✅ Read/pull access (required)
5. Click: **Generate Token**
6. **Copy token** (shown once only)

### 2. Add Repository Secret

**In Repository Settings**:
1. Go to: **Repository → Settings → Secrets/Variables**
2. Click: **Add Secret**
3. Add required secrets:
   - `REGISTRY_TOKEN`: Token from step 1
   - `REGISTRY_USER`: Username
4. Click: **Save**

### 3. Verify Workflow

**Check workflow file exists**:
```bash
ls -la .github/workflows/  # or appropriate CI/CD directory
```

**Push to trigger**:
```bash
git add .
git commit -m "Test CI/CD pipeline"
git push origin main
```

**Monitor execution**:
1. Go to repository → **CI/CD** or **Actions**
2. Click on the running workflow
3. View logs for each job

## Image Naming Convention

**Format**: `<registry>/<namespace>/<app>:<tag>`

**Examples**:
- `<registry>/web:main`
- `<registry>/web:develop`
- `<registry>/web:feature-xyz`
- `<registry>/api:main`

## Using Built Images in Kubernetes

### 1. Pull Images from Container Registry

**Create registry secret**:
```bash
kubectl create secret docker-registry registry-secret \
  --docker-server=<registry-url> \
  --docker-username=<USERNAME> \
  --docker-password=<TOKEN> \
  -n <namespace>
```

### 2. Update Deployment

**deployment.yaml**:
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
  namespace: <namespace>
spec:
  template:
    spec:
      imagePullSecrets:
      - name: registry-secret
      containers:
      - name: web
        image: <registry>/web:main
        imagePullPolicy: Always
```

### 3. Deploy to Kubernetes

```bash
kubectl apply -f deployment.yaml
```

### 4. Rollout New Version

**After pipeline builds new image**:

```bash
# Force rollout restart (pulls latest image with same tag)
kubectl rollout restart deployment/web -n <namespace>
kubectl rollout restart deployment/api -n <namespace>

# Check rollout status
kubectl rollout status deployment/web -n <namespace>
```

**Or update image explicitly**:
```bash
kubectl set image deployment/web \
  web=<registry>/web:main \
  -n <namespace>

kubectl set image deployment/api \
  api=<registry>/api:main \
  -n <namespace>
```

## Docker Layer Caching

The pipeline uses Docker layer caching to speed up builds:

**Cache Configuration**:
```yaml
cache-from: type=registry,ref=<registry>/web:buildcache
cache-to: type=registry,ref=<registry>/web:buildcache,mode=max
```

**Benefits**:
- Faster builds (reuses unchanged layers)
- Reduced build time from ~3min to ~30sec
- Lower resource usage

**Cache Location**: Stored in registry as special tag `:buildcache`

## Parallel Builds

Both applications build in parallel:

```
Start
  ├─→ build-web job   (3-5 min)
  └─→ build-api job   (3-5 min)
         ↓
    Complete (~5 min total)
```

Without parallelization: ~10 minutes
With parallelization: ~5 minutes

## Viewing Build Logs

### Via CI/CD Interface

1. Repository → **CI/CD** or **Actions**
2. Click on workflow run
3. Select job (`build-web` or `build-api`)
4. View step-by-step logs

### Via CLI (if available)

```bash
# List workflow runs
<cli-tool> list runs

# View specific run logs
<cli-tool> view run <run-id>
```

## Troubleshooting

### Build Fails: Authentication Error

**Error**: `unauthorized: authentication required`

**Solution**:
1. Verify registry secrets exist in repository settings
2. Check token has appropriate permissions
3. Regenerate token if expired

```bash
# Test token manually
docker login <registry-url> -u <USERNAME> -p <TOKEN>
```

### Build Fails: Dockerfile Not Found

**Error**: `unable to prepare context: unable to evaluate symlinks in Dockerfile path`

**Solution**:
1. Check Dockerfile exists in correct location:
   - `web/Dockerfile`
   - `api/Dockerfile`
2. Verify workflow context path matches:
   ```yaml
   context: ./web
   file: ./web/Dockerfile
   ```

### Image Pull Fails in Kubernetes

**Error**: `ImagePullBackOff`

**Solution**:
1. Verify registry secret exists:
   ```bash
   kubectl get secret registry-secret -n <namespace>
   ```

2. Check secret is referenced in deployment:
   ```yaml
   spec:
     template:
       spec:
         imagePullSecrets:
         - name: registry-secret
   ```

3. Test manual pull:
   ```bash
   docker pull <registry>/web:main
   ```

### Workflow Doesn't Trigger

**Check**:
1. ✅ Workflow file in appropriate CI/CD directory
2. ✅ File is named correctly (`.yml` or `.yaml`)
3. ✅ File is committed and pushed to repository
4. ✅ CI/CD is enabled in repository settings

**Force trigger**:
```bash
git commit --allow-empty -m "Trigger workflow"
git push
```

## Best Practices

### Version Tagging

**For production deployments**:
```bash
# Tag release
git tag -a v1.0.0 -m "Release v1.0.0"
git push origin v1.0.0
```

**Update workflow for semantic versioning**:
```yaml
tags: |
  type=ref,event=branch
  type=semver,pattern={{version}}
  type=semver,pattern={{major}}.{{minor}}
```

### Branch Strategy

**Recommended**:
- `main` → production images
- `develop` → staging images
- `feature/*` → development images

**Kubernetes deployments**:
```yaml
# Production
image: <registry>/web:main

# Staging
image: <registry>/web:develop

# Development
image: <registry>/web:feature-xyz
```

### Security

**Protect main branch**:
1. Repository Settings → **Branches** or **Protected Branches**
2. Add branch protection for `main`
3. Require pull/merge request reviews
4. Enable status checks

**Rotate tokens regularly**:
- Generate new registry token every 90 days
- Update secrets in repository settings

## Advanced Configuration

### Multi-Stage Builds

Optimize Dockerfile for smaller images:

```dockerfile
# Build stage
FROM python:3.11-slim AS builder
WORKDIR /app
COPY requirements.txt .
RUN pip install --user --no-cache-dir -r requirements.txt

# Runtime stage
FROM python:3.11-slim
WORKDIR /app
COPY --from=builder /root/.local /root/.local
COPY . .
ENV PATH=/root/.local/bin:$PATH
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]
```

### Build Arguments

Pass build-time variables:

```yaml
- name: Build and push
  uses: docker/build-push-action@v5
  with:
    build-args: |
      VERSION=${{ github.sha }}
      BUILD_DATE=${{ github.event.head_commit.timestamp }}
```

### Matrix Builds

Build for multiple platforms:

```yaml
strategy:
  matrix:
    platform: [linux/amd64, linux/arm64]

- name: Build and push
  uses: docker/build-push-action@v5
  with:
    platforms: ${{ matrix.platform }}
```

## Monitoring Pipeline

### Metrics to Track

- ✅ Build success rate
- ✅ Average build time
- ✅ Cache hit rate
- ✅ Image size trends

### Notifications

**Add Discord/Slack notifications** (example):

```yaml
- name: Notify on failure
  if: failure()
  run: |
    curl -X POST ${{ secrets.WEBHOOK_URL }} \
      -H 'Content-Type: application/json' \
      -d '{"content":"Build failed for ${{ github.ref }}"}'
```

---

*Automated CI/CD pipeline for building and deploying containerized applications.*