first commit
This commit is contained in:
d.viti
477
web/docs/cicd-pipeline.md
Normal file
477
web/docs/cicd-pipeline.md
Normal file
@@ -0,0 +1,477 @@
|
||||
# CI/CD Pipeline
|
||||
|
||||
Automated build and deployment pipeline using Git-based CI/CD systems.
|
||||
|
||||
## Overview
|
||||
|
||||
The CI/CD pipeline automatically builds Docker images and pushes them to the container registry whenever code is pushed to any branch.
|
||||
|
||||
**Repository**: Git repository with application code
|
||||
|
||||
**Registry**: Container registry for Docker images
|
||||
|
||||
## Pipeline Architecture
|
||||
|
||||
```
|
||||
Git Push (any branch)
|
||||
↓
|
||||
CI/CD System Trigger
|
||||
↓
|
||||
┌─────────────────────────────────────┐
|
||||
│ Job: build-web (parallel) │
|
||||
│ Job: build-api (parallel) │
|
||||
└─────────────────────────────────────┘
|
||||
↓ ↓
|
||||
Build Web Image Build API Image
|
||||
↓ ↓
|
||||
Tag with branch Tag with branch
|
||||
↓ ↓
|
||||
Push to Registry Push to Registry
|
||||
↓ ↓
|
||||
<registry>/web:<branch>
|
||||
<registry>/api:<branch>
|
||||
```
|
||||
|
||||
## Workflow Configuration
|
||||
|
||||
**Location**: CI/CD workflow files (`.github/workflows/`, `.gitlab-ci.yml`, etc.)
|
||||
|
||||
```yaml
|
||||
name: Build and Push Docker Images
|
||||
|
||||
on:
|
||||
push:
|
||||
workflow_dispatch:
|
||||
|
||||
jobs:
|
||||
build-web:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@v3
|
||||
|
||||
- name: Login to Container Registry
|
||||
uses: docker/login-action@v3
|
||||
with:
|
||||
registry: <registry-url>
|
||||
username: ${{ secrets.REGISTRY_USER }}
|
||||
password: ${{ secrets.REGISTRY_TOKEN }}
|
||||
|
||||
- name: Extract metadata for web image
|
||||
id: meta-web
|
||||
uses: docker/metadata-action@v5
|
||||
with:
|
||||
images: <registry>/web
|
||||
tags: |
|
||||
type=ref,event=branch
|
||||
|
||||
- name: Build and push web image
|
||||
uses: docker/build-push-action@v5
|
||||
with:
|
||||
context: ./web
|
||||
file: ./web/Dockerfile
|
||||
push: true
|
||||
tags: ${{ steps.meta-web.outputs.tags }}
|
||||
labels: ${{ steps.meta-web.outputs.labels }}
|
||||
cache-from: type=registry,ref=<registry>/web:buildcache
|
||||
cache-to: type=registry,ref=<registry>/web:buildcache,mode=max
|
||||
|
||||
build-api:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
# Similar steps for API application
|
||||
...
|
||||
```
|
||||
|
||||
## Trigger Events
|
||||
|
||||
### Automatic Triggers
|
||||
|
||||
**Push to any branch**:
|
||||
```bash
|
||||
git push origin <branch-name>
|
||||
```
|
||||
|
||||
Images tagged with branch name:
|
||||
- `<registry>/web:<branch-name>`
|
||||
- `<registry>/api:<branch-name>`
|
||||
|
||||
### Manual Trigger
|
||||
|
||||
**Via CI/CD Interface**:
|
||||
1. Go to repository → **CI/CD** or **Actions**
|
||||
2. Select workflow: **Build and Push Docker Images**
|
||||
3. Click: **Run workflow**
|
||||
4. Select branch
|
||||
5. Click: **Run**
|
||||
|
||||
**Via Git**:
|
||||
```bash
|
||||
# Trigger workflow with empty commit
|
||||
git commit --allow-empty -m "Trigger CI/CD pipeline"
|
||||
git push
|
||||
```
|
||||
|
||||
## Setup Instructions
|
||||
|
||||
### 1. Create Registry Token
|
||||
|
||||
**Generate Token**:
|
||||
1. Go to: **User Settings** or **Access Tokens**
|
||||
2. Click: **Generate New Token**
|
||||
3. Name: `container-registry`
|
||||
4. Select appropriate permissions:
|
||||
- ✅ Write/push access (required)
|
||||
- ✅ Read/pull access (required)
|
||||
5. Click: **Generate Token**
|
||||
6. **Copy token** (shown once only)
|
||||
|
||||
### 2. Add Repository Secret
|
||||
|
||||
**In Repository Settings**:
|
||||
1. Go to: **Repository → Settings → Secrets/Variables**
|
||||
2. Click: **Add Secret**
|
||||
3. Add required secrets:
|
||||
- `REGISTRY_TOKEN`: Token from step 1
|
||||
- `REGISTRY_USER`: Username
|
||||
4. Click: **Save**
|
||||
|
||||
### 3. Verify Workflow
|
||||
|
||||
**Check workflow file exists**:
|
||||
```bash
|
||||
ls -la .github/workflows/ # or appropriate CI/CD directory
|
||||
```
|
||||
|
||||
**Push to trigger**:
|
||||
```bash
|
||||
git add .
|
||||
git commit -m "Test CI/CD pipeline"
|
||||
git push origin main
|
||||
```
|
||||
|
||||
**Monitor execution**:
|
||||
1. Go to repository → **CI/CD** or **Actions**
|
||||
2. Click on the running workflow
|
||||
3. View logs for each job
|
||||
|
||||
## Image Naming Convention
|
||||
|
||||
**Format**: `<registry>/<namespace>/<app>:<tag>`
|
||||
|
||||
**Examples**:
|
||||
- `<registry>/web:main`
|
||||
- `<registry>/web:develop`
|
||||
- `<registry>/web:feature-xyz`
|
||||
- `<registry>/api:main`
|
||||
|
||||
## Using Built Images in Kubernetes
|
||||
|
||||
### 1. Pull Images from Container Registry
|
||||
|
||||
**Create registry secret**:
|
||||
```bash
|
||||
kubectl create secret docker-registry registry-secret \
|
||||
--docker-server=<registry-url> \
|
||||
--docker-username=<USERNAME> \
|
||||
--docker-password=<TOKEN> \
|
||||
-n <namespace>
|
||||
```
|
||||
|
||||
### 2. Update Deployment
|
||||
|
||||
**deployment.yaml**:
|
||||
```yaml
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: web
|
||||
namespace: <namespace>
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
imagePullSecrets:
|
||||
- name: registry-secret
|
||||
containers:
|
||||
- name: web
|
||||
image: <registry>/web:main
|
||||
imagePullPolicy: Always
|
||||
```
|
||||
|
||||
### 3. Deploy to Kubernetes
|
||||
|
||||
```bash
|
||||
kubectl apply -f deployment.yaml
|
||||
```
|
||||
|
||||
### 4. Rollout New Version
|
||||
|
||||
**After pipeline builds new image**:
|
||||
|
||||
```bash
|
||||
# Force rollout restart (pulls latest image with same tag)
|
||||
kubectl rollout restart deployment/web -n <namespace>
|
||||
kubectl rollout restart deployment/api -n <namespace>
|
||||
|
||||
# Check rollout status
|
||||
kubectl rollout status deployment/web -n <namespace>
|
||||
```
|
||||
|
||||
**Or update image explicitly**:
|
||||
```bash
|
||||
kubectl set image deployment/web \
|
||||
web=<registry>/web:main \
|
||||
-n <namespace>
|
||||
|
||||
kubectl set image deployment/api \
|
||||
api=<registry>/api:main \
|
||||
-n <namespace>
|
||||
```
|
||||
|
||||
## Docker Layer Caching
|
||||
|
||||
The pipeline uses Docker layer caching to speed up builds:
|
||||
|
||||
**Cache Configuration**:
|
||||
```yaml
|
||||
cache-from: type=registry,ref=<registry>/web:buildcache
|
||||
cache-to: type=registry,ref=<registry>/web:buildcache,mode=max
|
||||
```
|
||||
|
||||
**Benefits**:
|
||||
- Faster builds (reuses unchanged layers)
|
||||
- Reduced build time from ~3min to ~30sec
|
||||
- Lower resource usage
|
||||
|
||||
**Cache Location**: Stored in registry as special tag `:buildcache`
|
||||
|
||||
## Parallel Builds
|
||||
|
||||
Both applications build in parallel:
|
||||
|
||||
```
|
||||
Start
|
||||
├─→ build-web job (3-5 min)
|
||||
└─→ build-api job (3-5 min)
|
||||
↓
|
||||
Complete (~5 min total)
|
||||
```
|
||||
|
||||
Without parallelization: ~10 minutes
|
||||
With parallelization: ~5 minutes
|
||||
|
||||
## Viewing Build Logs
|
||||
|
||||
### Via CI/CD Interface
|
||||
|
||||
1. Repository → **CI/CD** or **Actions**
|
||||
2. Click on workflow run
|
||||
3. Select job (`build-web` or `build-api`)
|
||||
4. View step-by-step logs
|
||||
|
||||
### Via CLI (if available)
|
||||
|
||||
```bash
|
||||
# List workflow runs
|
||||
<cli-tool> list runs
|
||||
|
||||
# View specific run logs
|
||||
<cli-tool> view run <run-id>
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Build Fails: Authentication Error
|
||||
|
||||
**Error**: `unauthorized: authentication required`
|
||||
|
||||
**Solution**:
|
||||
1. Verify registry secrets exist in repository settings
|
||||
2. Check token has appropriate permissions
|
||||
3. Regenerate token if expired
|
||||
|
||||
```bash
|
||||
# Test token manually
|
||||
docker login <registry-url> -u <USERNAME> -p <TOKEN>
|
||||
```
|
||||
|
||||
### Build Fails: Dockerfile Not Found
|
||||
|
||||
**Error**: `unable to prepare context: unable to evaluate symlinks in Dockerfile path`
|
||||
|
||||
**Solution**:
|
||||
1. Check Dockerfile exists in correct location:
|
||||
- `web/Dockerfile`
|
||||
- `api/Dockerfile`
|
||||
2. Verify workflow context path matches:
|
||||
```yaml
|
||||
context: ./web
|
||||
file: ./web/Dockerfile
|
||||
```
|
||||
|
||||
### Image Pull Fails in Kubernetes
|
||||
|
||||
**Error**: `ImagePullBackOff`
|
||||
|
||||
**Solution**:
|
||||
1. Verify registry secret exists:
|
||||
```bash
|
||||
kubectl get secret registry-secret -n <namespace>
|
||||
```
|
||||
|
||||
2. Check secret is referenced in deployment:
|
||||
```yaml
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
imagePullSecrets:
|
||||
- name: registry-secret
|
||||
```
|
||||
|
||||
3. Test manual pull:
|
||||
```bash
|
||||
docker pull <registry>/web:main
|
||||
```
|
||||
|
||||
### Workflow Doesn't Trigger
|
||||
|
||||
**Check**:
|
||||
1. ✅ Workflow file in appropriate CI/CD directory
|
||||
2. ✅ File is named correctly (`.yml` or `.yaml`)
|
||||
3. ✅ File is committed and pushed to repository
|
||||
4. ✅ CI/CD is enabled in repository settings
|
||||
|
||||
**Force trigger**:
|
||||
```bash
|
||||
git commit --allow-empty -m "Trigger workflow"
|
||||
git push
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
### Version Tagging
|
||||
|
||||
**For production deployments**:
|
||||
```bash
|
||||
# Tag release
|
||||
git tag -a v1.0.0 -m "Release v1.0.0"
|
||||
git push origin v1.0.0
|
||||
```
|
||||
|
||||
**Update workflow for semantic versioning**:
|
||||
```yaml
|
||||
tags: |
|
||||
type=ref,event=branch
|
||||
type=semver,pattern={{version}}
|
||||
type=semver,pattern={{major}}.{{minor}}
|
||||
```
|
||||
|
||||
### Branch Strategy
|
||||
|
||||
**Recommended**:
|
||||
- `main` → production images
|
||||
- `develop` → staging images
|
||||
- `feature/*` → development images
|
||||
|
||||
**Kubernetes deployments**:
|
||||
```yaml
|
||||
# Production
|
||||
image: <registry>/web:main
|
||||
|
||||
# Staging
|
||||
image: <registry>/web:develop
|
||||
|
||||
# Development
|
||||
image: <registry>/web:feature-xyz
|
||||
```
|
||||
|
||||
### Security
|
||||
|
||||
**Protect main branch**:
|
||||
1. Repository Settings → **Branches** or **Protected Branches**
|
||||
2. Add branch protection for `main`
|
||||
3. Require pull/merge request reviews
|
||||
4. Enable status checks
|
||||
|
||||
**Rotate tokens regularly**:
|
||||
- Generate new registry token every 90 days
|
||||
- Update secrets in repository settings
|
||||
|
||||
## Advanced Configuration
|
||||
|
||||
### Multi-Stage Builds
|
||||
|
||||
Optimize Dockerfile for smaller images:
|
||||
|
||||
```dockerfile
|
||||
# Build stage
|
||||
FROM python:3.11-slim AS builder
|
||||
WORKDIR /app
|
||||
COPY requirements.txt .
|
||||
RUN pip install --user --no-cache-dir -r requirements.txt
|
||||
|
||||
# Runtime stage
|
||||
FROM python:3.11-slim
|
||||
WORKDIR /app
|
||||
COPY --from=builder /root/.local /root/.local
|
||||
COPY . .
|
||||
ENV PATH=/root/.local/bin:$PATH
|
||||
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]
|
||||
```
|
||||
|
||||
### Build Arguments
|
||||
|
||||
Pass build-time variables:
|
||||
|
||||
```yaml
|
||||
- name: Build and push
|
||||
uses: docker/build-push-action@v5
|
||||
with:
|
||||
build-args: |
|
||||
VERSION=${{ github.sha }}
|
||||
BUILD_DATE=${{ github.event.head_commit.timestamp }}
|
||||
```
|
||||
|
||||
### Matrix Builds
|
||||
|
||||
Build for multiple platforms:
|
||||
|
||||
```yaml
|
||||
strategy:
|
||||
matrix:
|
||||
platform: [linux/amd64, linux/arm64]
|
||||
|
||||
- name: Build and push
|
||||
uses: docker/build-push-action@v5
|
||||
with:
|
||||
platforms: ${{ matrix.platform }}
|
||||
```
|
||||
|
||||
## Monitoring Pipeline
|
||||
|
||||
### Metrics to Track
|
||||
|
||||
- ✅ Build success rate
|
||||
- ✅ Average build time
|
||||
- ✅ Cache hit rate
|
||||
- ✅ Image size trends
|
||||
|
||||
### Notifications
|
||||
|
||||
**Add Discord/Slack notifications** (example):
|
||||
|
||||
```yaml
|
||||
- name: Notify on failure
|
||||
if: failure()
|
||||
run: |
|
||||
curl -X POST ${{ secrets.WEBHOOK_URL }} \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"content":"Build failed for ${{ github.ref }}"}'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
*Automated CI/CD pipeline for building and deploying containerized applications.*
|
||||
Reference in New Issue
Block a user