refactor: simplify CI/CD to single build pipeline
Some checks failed
Build / Code Quality Checks (push) Successful in 7m43s
Build / Build & Push Docker Images (chat) (push) Failing after 3m6s
Build / Build & Push Docker Images (api) (push) Failing after 3m38s
Build / Build & Push Docker Images (worker) (push) Failing after 3m36s
Build / Build & Push Docker Images (frontend) (push) Successful in 4m12s
Some checks failed
Build / Code Quality Checks (push) Successful in 7m43s
Build / Build & Push Docker Images (chat) (push) Failing after 3m6s
Build / Build & Push Docker Images (api) (push) Failing after 3m38s
Build / Build & Push Docker Images (worker) (push) Failing after 3m36s
Build / Build & Push Docker Images (frontend) (push) Successful in 4m12s
Remove all unnecessary pipelines and consolidate into a single simple build pipeline that runs on every push to main. ## Pipeline Flow 1. **Lint Job**: Code quality checks - Black (formatting) - Ruff (linting) - MyPy (type checking) 2. **Build & Push Job**: Docker images (needs: lint) - Matrix build: [api, chat, worker, frontend] - Push to container registry - BuildKit caching ## Removed Pipelines - ❌ test.yml (no tests yet) - ❌ security.yml (not needed for now) - ❌ deploy-staging.yml (manual deployment) - ❌ deploy-production.yml (manual deployment) - ❌ docs-generation.yml (not needed) - ❌ lint.yml (merged into build) - ❌ ci.yml.old (old backup) - ❌ README.md (unnecessary complexity) ## Benefits - ✅ Simple and focused - ✅ Fast feedback on main pushes - ✅ Quality checks before build - ✅ Easy to understand and maintain 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -1,174 +0,0 @@
|
|||||||
# CI/CD Workflows
|
|
||||||
|
|
||||||
This directory contains separated CI/CD pipeline workflows for the Datacenter Documentation System.
|
|
||||||
|
|
||||||
## Workflow Organization
|
|
||||||
|
|
||||||
Each workflow is designed for a specific purpose and can run independently:
|
|
||||||
|
|
||||||
### 1. Code Quality (`lint.yml`)
|
|
||||||
**Triggers**: Push/PR to main/develop
|
|
||||||
|
|
||||||
Runs code quality checks:
|
|
||||||
- **Black**: Code formatting verification
|
|
||||||
- **Ruff**: Linting and code quality
|
|
||||||
- **MyPy**: Static type checking
|
|
||||||
|
|
||||||
**Purpose**: Ensure code quality standards before merge.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### 2. Testing (`test.yml`)
|
|
||||||
**Triggers**: Push/PR to main/develop
|
|
||||||
|
|
||||||
Runs test suite with coverage:
|
|
||||||
- Unit tests with pytest
|
|
||||||
- Integration tests (when available)
|
|
||||||
- Coverage reporting to CodeCov
|
|
||||||
- MongoDB service container for database tests
|
|
||||||
|
|
||||||
**Purpose**: Verify functionality and prevent regressions.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### 3. Security (`security.yml`)
|
|
||||||
**Triggers**: Push/PR to main/develop, Weekly schedule (Monday)
|
|
||||||
|
|
||||||
Security scanning:
|
|
||||||
- **Bandit**: Python code security analysis
|
|
||||||
- **Safety**: Dependency vulnerability scanning (future)
|
|
||||||
|
|
||||||
**Purpose**: Identify security vulnerabilities early.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### 4. Build (`build.yml`)
|
|
||||||
**Triggers**: Push to main, Version tags
|
|
||||||
|
|
||||||
Builds and pushes Docker images:
|
|
||||||
- **Components**: API, Chat, Worker, Frontend
|
|
||||||
- **Registry**: Gitea Container Registry
|
|
||||||
- **Caching**: BuildKit cache for faster builds
|
|
||||||
- **Tagging**: Branch names, SHAs, semantic versions
|
|
||||||
|
|
||||||
**Purpose**: Create deployable container images.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### 5. Deploy - Staging (`deploy-staging.yml`)
|
|
||||||
**Triggers**: Push to main (after successful build)
|
|
||||||
|
|
||||||
Deploys to staging environment:
|
|
||||||
- **Platform**: Kubernetes
|
|
||||||
- **Namespace**: datacenter-docs
|
|
||||||
- **Components**: API, Chat, Worker
|
|
||||||
- **Validation**: Rollout status checks
|
|
||||||
|
|
||||||
**Purpose**: Test changes in staging before production.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### 6. Deploy - Production (`deploy-production.yml`)
|
|
||||||
**Triggers**: Version tags (v*)
|
|
||||||
|
|
||||||
Deploys to production environment:
|
|
||||||
- **Platform**: Kubernetes
|
|
||||||
- **Namespace**: datacenter-docs
|
|
||||||
- **Components**: API, Chat, Worker
|
|
||||||
- **Validation**: Rollout status + smoke tests
|
|
||||||
|
|
||||||
**Purpose**: Deploy stable versions to production.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### 7. Documentation Generation (`docs-generation.yml`)
|
|
||||||
**Triggers**: Schedule (every 6 hours), Manual
|
|
||||||
|
|
||||||
Automated documentation generation:
|
|
||||||
- Collects infrastructure data
|
|
||||||
- Generates documentation via LLM
|
|
||||||
- Uploads artifacts
|
|
||||||
- Commits changes (if any)
|
|
||||||
|
|
||||||
**Purpose**: Keep documentation up-to-date automatically.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Workflow Dependencies
|
|
||||||
|
|
||||||
```
|
|
||||||
┌─────────────┐
|
|
||||||
│ lint.yml │ ─┐
|
|
||||||
└─────────────┘ │
|
|
||||||
│
|
|
||||||
┌─────────────┐ │
|
|
||||||
│ test.yml │ ─┤
|
|
||||||
└─────────────┘ │
|
|
||||||
├──> ┌──────────────┐ ┌────────────────────┐
|
|
||||||
┌─────────────┐ │ │ build.yml │ ──> │ deploy-staging.yml │
|
|
||||||
│security.yml │ ─┘ └──────────────┘ └────────────────────┘
|
|
||||||
└─────────────┘ │
|
|
||||||
│ (on tags)
|
|
||||||
▼
|
|
||||||
┌──────────────────────┐
|
|
||||||
│deploy-production.yml │
|
|
||||||
└──────────────────────┘
|
|
||||||
|
|
||||||
┌──────────────────────┐
|
|
||||||
│docs-generation.yml │ (independent, scheduled)
|
|
||||||
└──────────────────────┘
|
|
||||||
```
|
|
||||||
|
|
||||||
## Environment Variables
|
|
||||||
|
|
||||||
### Required Secrets
|
|
||||||
- `USERNAME`: Registry username
|
|
||||||
- `TOKEN`: Registry access token
|
|
||||||
- `LLM_API_KEY`: LLM provider API key
|
|
||||||
- `LLM_BASE_URL`: LLM provider base URL
|
|
||||||
- `MCP_SERVER_URL`: Infrastructure API endpoint
|
|
||||||
- `KUBE_CONFIG_STAGING`: Staging Kubernetes config
|
|
||||||
- `KUBE_CONFIG_PRODUCTION`: Production Kubernetes config
|
|
||||||
|
|
||||||
### Required Variables
|
|
||||||
- `PACKAGES_REGISTRY`: Container registry URL (e.g., `git.commandware.com`)
|
|
||||||
|
|
||||||
## Running Workflows
|
|
||||||
|
|
||||||
### Automatic Triggers
|
|
||||||
All workflows run automatically based on their configured triggers.
|
|
||||||
|
|
||||||
### Manual Triggers
|
|
||||||
Some workflows support manual triggering via Gitea Actions UI:
|
|
||||||
- `docs-generation.yml` (workflow_dispatch)
|
|
||||||
|
|
||||||
## Best Practices
|
|
||||||
|
|
||||||
1. **Always pass lint and test** before merging to main
|
|
||||||
2. **Review security reports** weekly
|
|
||||||
3. **Test in staging** before creating version tags
|
|
||||||
4. **Use semantic versioning** for production releases (v1.2.3)
|
|
||||||
5. **Monitor deployment rollouts** in Kubernetes
|
|
||||||
|
|
||||||
## Troubleshooting
|
|
||||||
|
|
||||||
### Build Failures
|
|
||||||
- Check Docker build logs
|
|
||||||
- Verify poetry.lock is committed
|
|
||||||
- Ensure all dependencies are available
|
|
||||||
|
|
||||||
### Test Failures
|
|
||||||
- Review test output in job logs
|
|
||||||
- Check MongoDB service health
|
|
||||||
- Verify environment variables
|
|
||||||
|
|
||||||
### Deployment Failures
|
|
||||||
- Check Kubernetes cluster status
|
|
||||||
- Verify secrets are configured
|
|
||||||
- Review deployment rollout logs
|
|
||||||
|
|
||||||
## Maintenance
|
|
||||||
|
|
||||||
- Review and update workflow configurations quarterly
|
|
||||||
- Update action versions when new releases are available
|
|
||||||
- Monitor workflow execution times and optimize as needed
|
|
||||||
@@ -1,22 +1,58 @@
|
|||||||
# Build and Push Docker Images
|
# Build and Push Docker Images
|
||||||
|
# Runs on every push to main branch
|
||||||
|
# 1. Lint: Black, Ruff, MyPy
|
||||||
|
# 2. Build & Push: Docker images to registry
|
||||||
|
|
||||||
name: Build
|
name: Build
|
||||||
|
|
||||||
on:
|
on:
|
||||||
push:
|
push:
|
||||||
branches: [ main ]
|
branches: [ main ]
|
||||||
tags:
|
|
||||||
- 'v*'
|
|
||||||
|
|
||||||
env:
|
env:
|
||||||
|
POETRY_VERSION: 1.8.0
|
||||||
|
PYTHON_VERSION: "3.12"
|
||||||
REGISTRY: ${{ vars.PACKAGES_REGISTRY }}
|
REGISTRY: ${{ vars.PACKAGES_REGISTRY }}
|
||||||
IMAGE_NAME: ${{ gitea.repository }}
|
IMAGE_NAME: ${{ gitea.repository }}
|
||||||
|
|
||||||
jobs:
|
jobs:
|
||||||
build-and-push:
|
lint:
|
||||||
name: Build and Push Docker Images
|
name: Code Quality Checks
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
if: github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/')
|
|
||||||
|
steps:
|
||||||
|
- name: Checkout code
|
||||||
|
uses: actions/checkout@v4
|
||||||
|
|
||||||
|
- name: Set up Python
|
||||||
|
uses: actions/setup-python@v5
|
||||||
|
with:
|
||||||
|
python-version: ${{ env.PYTHON_VERSION }}
|
||||||
|
cache: 'pip'
|
||||||
|
|
||||||
|
- name: Install Poetry
|
||||||
|
run: |
|
||||||
|
curl -sSL https://install.python-poetry.org | python3 -
|
||||||
|
echo "$HOME/.local/bin" >> $GITHUB_PATH
|
||||||
|
|
||||||
|
- name: Install dependencies
|
||||||
|
run: |
|
||||||
|
poetry config virtualenvs.in-project true
|
||||||
|
poetry install --no-root
|
||||||
|
|
||||||
|
- name: Run Black
|
||||||
|
run: poetry run black --check src/ tests/
|
||||||
|
|
||||||
|
- name: Run Ruff
|
||||||
|
run: poetry run ruff check src/ tests/
|
||||||
|
|
||||||
|
- name: Run MyPy
|
||||||
|
run: poetry run mypy src/
|
||||||
|
|
||||||
|
build-and-push:
|
||||||
|
name: Build & Push Docker Images
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
needs: lint
|
||||||
|
|
||||||
strategy:
|
strategy:
|
||||||
matrix:
|
matrix:
|
||||||
@@ -43,11 +79,8 @@ jobs:
|
|||||||
images: ${{ vars.PACKAGES_REGISTRY }}/${{ env.IMAGE_NAME }}/${{ matrix.component }}
|
images: ${{ vars.PACKAGES_REGISTRY }}/${{ env.IMAGE_NAME }}/${{ matrix.component }}
|
||||||
tags: |
|
tags: |
|
||||||
type=ref,event=branch
|
type=ref,event=branch
|
||||||
type=ref,event=pr
|
|
||||||
type=semver,pattern={{version}}
|
|
||||||
type=semver,pattern={{major}}.{{minor}}
|
|
||||||
type=sha,prefix={{branch}}-
|
type=sha,prefix={{branch}}-
|
||||||
type=raw,value=latest,enable={{is_default_branch}}
|
type=raw,value=latest
|
||||||
|
|
||||||
- name: Build and push Docker image
|
- name: Build and push Docker image
|
||||||
uses: docker/build-push-action@v5
|
uses: docker/build-push-action@v5
|
||||||
|
|||||||
@@ -1,318 +0,0 @@
|
|||||||
# Gitea Actions CI/CD Pipeline for Datacenter Documentation System
|
|
||||||
|
|
||||||
name: CI/CD Pipeline
|
|
||||||
|
|
||||||
on:
|
|
||||||
push:
|
|
||||||
branches: [ main, develop ]
|
|
||||||
pull_request:
|
|
||||||
branches: [ main, develop ]
|
|
||||||
schedule:
|
|
||||||
- cron: '0 */6 * * *' # Every 6 hours for docs generation
|
|
||||||
|
|
||||||
env:
|
|
||||||
POETRY_VERSION: 1.8.0
|
|
||||||
PYTHON_VERSION: "3.12"
|
|
||||||
REGISTRY: ${{ vars.PACKAGES_REGISTRY }}
|
|
||||||
IMAGE_NAME: ${{ gitea.repository }}
|
|
||||||
|
|
||||||
jobs:
|
|
||||||
lint:
|
|
||||||
name: Lint Code
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
|
|
||||||
steps:
|
|
||||||
- name: Checkout code
|
|
||||||
uses: actions/checkout@v4
|
|
||||||
|
|
||||||
- name: Set up Python
|
|
||||||
uses: actions/setup-python@v5
|
|
||||||
with:
|
|
||||||
python-version: ${{ env.PYTHON_VERSION }}
|
|
||||||
cache: 'pip'
|
|
||||||
|
|
||||||
- name: Install Poetry
|
|
||||||
run: |
|
|
||||||
curl -sSL https://install.python-poetry.org | python3 -
|
|
||||||
echo "$HOME/.local/bin" >> $GITHUB_PATH
|
|
||||||
|
|
||||||
- name: Install dependencies
|
|
||||||
run: |
|
|
||||||
poetry config virtualenvs.in-project true
|
|
||||||
poetry install --no-root
|
|
||||||
|
|
||||||
- name: Run Black
|
|
||||||
run: poetry run black --check src/ tests/
|
|
||||||
continue-on-error: true
|
|
||||||
|
|
||||||
- name: Run Ruff
|
|
||||||
run: poetry run ruff check src/ tests/
|
|
||||||
continue-on-error: true
|
|
||||||
|
|
||||||
- name: Run MyPy
|
|
||||||
run: poetry run mypy src/
|
|
||||||
continue-on-error: true
|
|
||||||
|
|
||||||
test:
|
|
||||||
name: Run Tests
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
needs: lint
|
|
||||||
|
|
||||||
services:
|
|
||||||
mongodb:
|
|
||||||
image: mongo:7-jammy
|
|
||||||
ports:
|
|
||||||
- 27017:27017
|
|
||||||
options: >-
|
|
||||||
--health-cmd "mongosh --eval 'db.adminCommand(\"ping\")'"
|
|
||||||
--health-interval 10s
|
|
||||||
--health-timeout 5s
|
|
||||||
--health-retries 5
|
|
||||||
|
|
||||||
steps:
|
|
||||||
- name: Checkout code
|
|
||||||
uses: actions/checkout@v4
|
|
||||||
|
|
||||||
- name: Set up Python
|
|
||||||
uses: actions/setup-python@v5
|
|
||||||
with:
|
|
||||||
python-version: ${{ env.PYTHON_VERSION }}
|
|
||||||
cache: 'pip'
|
|
||||||
|
|
||||||
- name: Install Poetry
|
|
||||||
run: |
|
|
||||||
curl -sSL https://install.python-poetry.org | python3 -
|
|
||||||
echo "$HOME/.local/bin" >> $GITHUB_PATH
|
|
||||||
|
|
||||||
- name: Install dependencies
|
|
||||||
run: |
|
|
||||||
poetry config virtualenvs.in-project true
|
|
||||||
poetry install
|
|
||||||
|
|
||||||
- name: Run unit tests
|
|
||||||
env:
|
|
||||||
MONGODB_URL: mongodb://localhost:27017
|
|
||||||
MONGODB_DATABASE: testdb
|
|
||||||
run: |
|
|
||||||
poetry run pytest tests/unit -v --cov --cov-report=xml --cov-report=html
|
|
||||||
continue-on-error: true
|
|
||||||
|
|
||||||
- name: Upload coverage
|
|
||||||
uses: codecov/codecov-action@v3
|
|
||||||
with:
|
|
||||||
files: ./coverage.xml
|
|
||||||
flags: unittests
|
|
||||||
name: codecov-umbrella
|
|
||||||
continue-on-error: true
|
|
||||||
|
|
||||||
- name: Archive coverage results
|
|
||||||
uses: actions/upload-artifact@v4
|
|
||||||
with:
|
|
||||||
name: coverage-report
|
|
||||||
path: htmlcov/
|
|
||||||
continue-on-error: true
|
|
||||||
|
|
||||||
security:
|
|
||||||
name: Security Scanning
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
needs: lint
|
|
||||||
|
|
||||||
steps:
|
|
||||||
- name: Checkout code
|
|
||||||
uses: actions/checkout@v4
|
|
||||||
|
|
||||||
- name: Set up Python
|
|
||||||
uses: actions/setup-python@v5
|
|
||||||
with:
|
|
||||||
python-version: ${{ env.PYTHON_VERSION }}
|
|
||||||
|
|
||||||
- name: Install Poetry
|
|
||||||
run: |
|
|
||||||
curl -sSL https://install.python-poetry.org | python3 -
|
|
||||||
echo "$HOME/.local/bin" >> $GITHUB_PATH
|
|
||||||
|
|
||||||
- name: Install dependencies
|
|
||||||
run: |
|
|
||||||
poetry config virtualenvs.in-project true
|
|
||||||
poetry install
|
|
||||||
|
|
||||||
- name: Run Bandit
|
|
||||||
run: |
|
|
||||||
poetry add --group dev bandit
|
|
||||||
poetry run bandit -r src/ -f json -o bandit-report.json
|
|
||||||
continue-on-error: true
|
|
||||||
|
|
||||||
- name: Run Safety
|
|
||||||
run: |
|
|
||||||
poetry export -f requirements.txt --output requirements.txt --without-hashes
|
|
||||||
poetry add --group dev safety
|
|
||||||
poetry run safety check --file requirements.txt
|
|
||||||
continue-on-error: true
|
|
||||||
|
|
||||||
build-and-push:
|
|
||||||
name: Build and Push Docker Images
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
needs: [test, security]
|
|
||||||
if: github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/')
|
|
||||||
|
|
||||||
strategy:
|
|
||||||
matrix:
|
|
||||||
component: [api, chat, worker, frontend]
|
|
||||||
|
|
||||||
steps:
|
|
||||||
- name: Checkout code
|
|
||||||
uses: actions/checkout@v4
|
|
||||||
|
|
||||||
- name: Set up Docker Buildx
|
|
||||||
uses: docker/setup-buildx-action@v3
|
|
||||||
|
|
||||||
- name: Log in to Container Registry
|
|
||||||
uses: docker/login-action@v3
|
|
||||||
with:
|
|
||||||
registry: ${{ vars.PACKAGES_REGISTRY }}
|
|
||||||
username: ${{ secrets.USERNAME }}
|
|
||||||
password: ${{ secrets.TOKEN }}
|
|
||||||
|
|
||||||
- name: Extract metadata
|
|
||||||
id: meta
|
|
||||||
uses: docker/metadata-action@v5
|
|
||||||
with:
|
|
||||||
images: ${{ vars.PACKAGES_REGISTRY }}/${{ env.IMAGE_NAME }}/${{ matrix.component }}
|
|
||||||
tags: |
|
|
||||||
type=ref,event=branch
|
|
||||||
type=ref,event=pr
|
|
||||||
type=semver,pattern={{version}}
|
|
||||||
type=semver,pattern={{major}}.{{minor}}
|
|
||||||
type=sha,prefix={{branch}}-
|
|
||||||
type=raw,value=latest,enable={{is_default_branch}}
|
|
||||||
|
|
||||||
- name: Build and push Docker image
|
|
||||||
uses: docker/build-push-action@v5
|
|
||||||
with:
|
|
||||||
context: .
|
|
||||||
file: deploy/docker/Dockerfile.${{ matrix.component }}
|
|
||||||
push: true
|
|
||||||
tags: ${{ steps.meta.outputs.tags }}
|
|
||||||
labels: ${{ steps.meta.outputs.labels }}
|
|
||||||
cache-from: type=registry,ref=${{ vars.PACKAGES_REGISTRY }}/${{ env.IMAGE_NAME }}/${{ matrix.component }}:buildcache
|
|
||||||
cache-to: type=registry,ref=${{ vars.PACKAGES_REGISTRY }}/${{ env.IMAGE_NAME }}/${{ matrix.component }}:buildcache,mode=max
|
|
||||||
|
|
||||||
deploy-staging:
|
|
||||||
name: Deploy to Staging
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
needs: build-and-push
|
|
||||||
if: github.ref == 'refs/heads/main'
|
|
||||||
environment:
|
|
||||||
name: staging
|
|
||||||
url: https://staging-docs.company.local
|
|
||||||
|
|
||||||
steps:
|
|
||||||
- name: Checkout code
|
|
||||||
uses: actions/checkout@v4
|
|
||||||
|
|
||||||
- name: Set up kubectl
|
|
||||||
uses: azure/setup-kubectl@v3
|
|
||||||
with:
|
|
||||||
version: 'latest'
|
|
||||||
|
|
||||||
- name: Configure kubectl
|
|
||||||
run: |
|
|
||||||
echo "${{ secrets.KUBE_CONFIG_STAGING }}" > kubeconfig
|
|
||||||
export KUBECONFIG=kubeconfig
|
|
||||||
|
|
||||||
- name: Deploy to Kubernetes
|
|
||||||
run: |
|
|
||||||
export KUBECONFIG=kubeconfig
|
|
||||||
kubectl set image deployment/api api=${{ vars.PACKAGES_REGISTRY }}/${{ env.IMAGE_NAME }}/api:${{ github.sha }} -n datacenter-docs
|
|
||||||
kubectl set image deployment/chat chat=${{ vars.PACKAGES_REGISTRY }}/${{ env.IMAGE_NAME }}/chat:${{ github.sha }} -n datacenter-docs
|
|
||||||
kubectl set image deployment/worker worker=${{ vars.PACKAGES_REGISTRY }}/${{ env.IMAGE_NAME }}/worker:${{ github.sha }} -n datacenter-docs
|
|
||||||
kubectl rollout status deployment/api -n datacenter-docs --timeout=5m
|
|
||||||
kubectl rollout status deployment/chat -n datacenter-docs --timeout=5m
|
|
||||||
kubectl rollout status deployment/worker -n datacenter-docs --timeout=5m
|
|
||||||
continue-on-error: true
|
|
||||||
|
|
||||||
deploy-production:
|
|
||||||
name: Deploy to Production
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
needs: build-and-push
|
|
||||||
if: startsWith(github.ref, 'refs/tags/')
|
|
||||||
environment:
|
|
||||||
name: production
|
|
||||||
url: https://docs.company.local
|
|
||||||
|
|
||||||
steps:
|
|
||||||
- name: Checkout code
|
|
||||||
uses: actions/checkout@v4
|
|
||||||
|
|
||||||
- name: Set up kubectl
|
|
||||||
uses: azure/setup-kubectl@v3
|
|
||||||
with:
|
|
||||||
version: 'latest'
|
|
||||||
|
|
||||||
- name: Configure kubectl
|
|
||||||
run: |
|
|
||||||
echo "${{ secrets.KUBE_CONFIG_PRODUCTION }}" > kubeconfig
|
|
||||||
export KUBECONFIG=kubeconfig
|
|
||||||
|
|
||||||
- name: Deploy to Kubernetes
|
|
||||||
run: |
|
|
||||||
export KUBECONFIG=kubeconfig
|
|
||||||
kubectl set image deployment/api api=${{ vars.PACKAGES_REGISTRY }}/${{ env.IMAGE_NAME }}/api:${{ github.ref_name }} -n datacenter-docs
|
|
||||||
kubectl set image deployment/chat chat=${{ vars.PACKAGES_REGISTRY }}/${{ env.IMAGE_NAME }}/chat:${{ github.ref_name }} -n datacenter-docs
|
|
||||||
kubectl set image deployment/worker worker=${{ vars.PACKAGES_REGISTRY }}/${{ env.IMAGE_NAME }}/worker:${{ github.ref_name }} -n datacenter-docs
|
|
||||||
kubectl rollout status deployment/api -n datacenter-docs --timeout=5m
|
|
||||||
kubectl rollout status deployment/chat -n datacenter-docs --timeout=5m
|
|
||||||
kubectl rollout status deployment/worker -n datacenter-docs --timeout=5m
|
|
||||||
|
|
||||||
- name: Smoke test
|
|
||||||
run: |
|
|
||||||
sleep 30
|
|
||||||
curl -f https://docs.company.local/health || exit 1
|
|
||||||
continue-on-error: true
|
|
||||||
|
|
||||||
generate-docs:
|
|
||||||
name: Generate Documentation
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
if: github.event.schedule == '0 */6 * * *' || github.ref == 'refs/heads/main'
|
|
||||||
|
|
||||||
steps:
|
|
||||||
- name: Checkout code
|
|
||||||
uses: actions/checkout@v4
|
|
||||||
|
|
||||||
- name: Set up Python
|
|
||||||
uses: actions/setup-python@v5
|
|
||||||
with:
|
|
||||||
python-version: ${{ env.PYTHON_VERSION }}
|
|
||||||
|
|
||||||
- name: Install Poetry
|
|
||||||
run: |
|
|
||||||
curl -sSL https://install.python-poetry.org | python3 -
|
|
||||||
echo "$HOME/.local/bin" >> $GITHUB_PATH
|
|
||||||
|
|
||||||
- name: Install dependencies
|
|
||||||
run: |
|
|
||||||
poetry config virtualenvs.in-project true
|
|
||||||
poetry install
|
|
||||||
|
|
||||||
- name: Generate documentation
|
|
||||||
env:
|
|
||||||
MCP_SERVER_URL: ${{ secrets.MCP_SERVER_URL }}
|
|
||||||
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
|
|
||||||
run: |
|
|
||||||
poetry run datacenter-docs generate-all
|
|
||||||
continue-on-error: true
|
|
||||||
|
|
||||||
- name: Upload documentation artifacts
|
|
||||||
uses: actions/upload-artifact@v4
|
|
||||||
with:
|
|
||||||
name: documentation
|
|
||||||
path: output/
|
|
||||||
retention-days: 30
|
|
||||||
|
|
||||||
- name: Commit and push if changed
|
|
||||||
run: |
|
|
||||||
git config --global user.name "Docs Bot"
|
|
||||||
git config --global user.email "bot@company.local"
|
|
||||||
git add output/
|
|
||||||
git diff --quiet && git diff --staged --quiet || (git commit -m "docs: Auto-generated documentation update [skip ci]" && git push)
|
|
||||||
continue-on-error: true
|
|
||||||
@@ -1,50 +0,0 @@
|
|||||||
# Deploy to Production Environment
|
|
||||||
|
|
||||||
name: Deploy Production
|
|
||||||
|
|
||||||
on:
|
|
||||||
push:
|
|
||||||
tags:
|
|
||||||
- 'v*'
|
|
||||||
|
|
||||||
env:
|
|
||||||
REGISTRY: ${{ vars.PACKAGES_REGISTRY }}
|
|
||||||
IMAGE_NAME: ${{ gitea.repository }}
|
|
||||||
|
|
||||||
jobs:
|
|
||||||
deploy-production:
|
|
||||||
name: Deploy to Production
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
environment:
|
|
||||||
name: production
|
|
||||||
url: https://docs.company.local
|
|
||||||
|
|
||||||
steps:
|
|
||||||
- name: Checkout code
|
|
||||||
uses: actions/checkout@v4
|
|
||||||
|
|
||||||
- name: Set up kubectl
|
|
||||||
uses: azure/setup-kubectl@v3
|
|
||||||
with:
|
|
||||||
version: 'latest'
|
|
||||||
|
|
||||||
- name: Configure kubectl
|
|
||||||
run: |
|
|
||||||
echo "${{ secrets.KUBE_CONFIG_PRODUCTION }}" > kubeconfig
|
|
||||||
export KUBECONFIG=kubeconfig
|
|
||||||
|
|
||||||
- name: Deploy to Kubernetes
|
|
||||||
run: |
|
|
||||||
export KUBECONFIG=kubeconfig
|
|
||||||
kubectl set image deployment/api api=${{ vars.PACKAGES_REGISTRY }}/${{ env.IMAGE_NAME }}/api:${{ github.ref_name }} -n datacenter-docs
|
|
||||||
kubectl set image deployment/chat chat=${{ vars.PACKAGES_REGISTRY }}/${{ env.IMAGE_NAME }}/chat:${{ github.ref_name }} -n datacenter-docs
|
|
||||||
kubectl set image deployment/worker worker=${{ vars.PACKAGES_REGISTRY }}/${{ env.IMAGE_NAME }}/worker:${{ github.ref_name }} -n datacenter-docs
|
|
||||||
kubectl rollout status deployment/api -n datacenter-docs --timeout=5m
|
|
||||||
kubectl rollout status deployment/chat -n datacenter-docs --timeout=5m
|
|
||||||
kubectl rollout status deployment/worker -n datacenter-docs --timeout=5m
|
|
||||||
|
|
||||||
- name: Smoke test
|
|
||||||
run: |
|
|
||||||
sleep 30
|
|
||||||
curl -f https://docs.company.local/health || exit 1
|
|
||||||
continue-on-error: true
|
|
||||||
@@ -1,45 +0,0 @@
|
|||||||
# Deploy to Staging Environment
|
|
||||||
|
|
||||||
name: Deploy Staging
|
|
||||||
|
|
||||||
on:
|
|
||||||
push:
|
|
||||||
branches: [ main ]
|
|
||||||
|
|
||||||
env:
|
|
||||||
REGISTRY: ${{ vars.PACKAGES_REGISTRY }}
|
|
||||||
IMAGE_NAME: ${{ gitea.repository }}
|
|
||||||
|
|
||||||
jobs:
|
|
||||||
deploy-staging:
|
|
||||||
name: Deploy to Staging
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
needs: [] # Will depend on build workflow when using workflow_run
|
|
||||||
environment:
|
|
||||||
name: staging
|
|
||||||
url: https://staging-docs.company.local
|
|
||||||
|
|
||||||
steps:
|
|
||||||
- name: Checkout code
|
|
||||||
uses: actions/checkout@v4
|
|
||||||
|
|
||||||
- name: Set up kubectl
|
|
||||||
uses: azure/setup-kubectl@v3
|
|
||||||
with:
|
|
||||||
version: 'latest'
|
|
||||||
|
|
||||||
- name: Configure kubectl
|
|
||||||
run: |
|
|
||||||
echo "${{ secrets.KUBE_CONFIG_STAGING }}" > kubeconfig
|
|
||||||
export KUBECONFIG=kubeconfig
|
|
||||||
|
|
||||||
- name: Deploy to Kubernetes
|
|
||||||
run: |
|
|
||||||
export KUBECONFIG=kubeconfig
|
|
||||||
kubectl set image deployment/api api=${{ vars.PACKAGES_REGISTRY }}/${{ env.IMAGE_NAME }}/api:${{ github.sha }} -n datacenter-docs
|
|
||||||
kubectl set image deployment/chat chat=${{ vars.PACKAGES_REGISTRY }}/${{ env.IMAGE_NAME }}/chat:${{ github.sha }} -n datacenter-docs
|
|
||||||
kubectl set image deployment/worker worker=${{ vars.PACKAGES_REGISTRY }}/${{ env.IMAGE_NAME }}/worker:${{ github.sha }} -n datacenter-docs
|
|
||||||
kubectl rollout status deployment/api -n datacenter-docs --timeout=5m
|
|
||||||
kubectl rollout status deployment/chat -n datacenter-docs --timeout=5m
|
|
||||||
kubectl rollout status deployment/worker -n datacenter-docs --timeout=5m
|
|
||||||
continue-on-error: true
|
|
||||||
@@ -1,60 +0,0 @@
|
|||||||
# Automated Documentation Generation
|
|
||||||
|
|
||||||
name: Generate Documentation
|
|
||||||
|
|
||||||
on:
|
|
||||||
schedule:
|
|
||||||
- cron: '0 */6 * * *' # Every 6 hours
|
|
||||||
workflow_dispatch: # Allow manual triggering
|
|
||||||
|
|
||||||
env:
|
|
||||||
POETRY_VERSION: 1.8.0
|
|
||||||
PYTHON_VERSION: "3.12"
|
|
||||||
|
|
||||||
jobs:
|
|
||||||
generate-docs:
|
|
||||||
name: Generate Documentation
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
|
|
||||||
steps:
|
|
||||||
- name: Checkout code
|
|
||||||
uses: actions/checkout@v4
|
|
||||||
|
|
||||||
- name: Set up Python
|
|
||||||
uses: actions/setup-python@v5
|
|
||||||
with:
|
|
||||||
python-version: ${{ env.PYTHON_VERSION }}
|
|
||||||
|
|
||||||
- name: Install Poetry
|
|
||||||
run: |
|
|
||||||
curl -sSL https://install.python-poetry.org | python3 -
|
|
||||||
echo "$HOME/.local/bin" >> $GITHUB_PATH
|
|
||||||
|
|
||||||
- name: Install dependencies
|
|
||||||
run: |
|
|
||||||
poetry config virtualenvs.in-project true
|
|
||||||
poetry install
|
|
||||||
|
|
||||||
- name: Generate documentation
|
|
||||||
env:
|
|
||||||
MCP_SERVER_URL: ${{ secrets.MCP_SERVER_URL }}
|
|
||||||
LLM_BASE_URL: ${{ secrets.LLM_BASE_URL }}
|
|
||||||
LLM_API_KEY: ${{ secrets.LLM_API_KEY }}
|
|
||||||
run: |
|
|
||||||
poetry run datacenter-docs generate-all
|
|
||||||
continue-on-error: true
|
|
||||||
|
|
||||||
- name: Upload documentation artifacts
|
|
||||||
uses: actions/upload-artifact@v4
|
|
||||||
with:
|
|
||||||
name: documentation
|
|
||||||
path: output/
|
|
||||||
retention-days: 30
|
|
||||||
|
|
||||||
- name: Commit and push if changed
|
|
||||||
run: |
|
|
||||||
git config --global user.name "Docs Bot"
|
|
||||||
git config --global user.email "bot@company.local"
|
|
||||||
git add output/
|
|
||||||
git diff --quiet && git diff --staged --quiet || (git commit -m "docs: Auto-generated documentation update [skip ci]" && git push)
|
|
||||||
continue-on-error: true
|
|
||||||
@@ -1,47 +0,0 @@
|
|||||||
# Code Quality - Linting and Type Checking
|
|
||||||
|
|
||||||
name: Lint
|
|
||||||
|
|
||||||
on:
|
|
||||||
push:
|
|
||||||
branches: [ main, develop ]
|
|
||||||
pull_request:
|
|
||||||
branches: [ main, develop ]
|
|
||||||
|
|
||||||
env:
|
|
||||||
POETRY_VERSION: 1.8.0
|
|
||||||
PYTHON_VERSION: "3.12"
|
|
||||||
|
|
||||||
jobs:
|
|
||||||
lint:
|
|
||||||
name: Lint Code
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
|
|
||||||
steps:
|
|
||||||
- name: Checkout code
|
|
||||||
uses: actions/checkout@v4
|
|
||||||
|
|
||||||
- name: Set up Python
|
|
||||||
uses: actions/setup-python@v5
|
|
||||||
with:
|
|
||||||
python-version: ${{ env.PYTHON_VERSION }}
|
|
||||||
cache: 'pip'
|
|
||||||
|
|
||||||
- name: Install Poetry
|
|
||||||
run: |
|
|
||||||
curl -sSL https://install.python-poetry.org | python3 -
|
|
||||||
echo "$HOME/.local/bin" >> $GITHUB_PATH
|
|
||||||
|
|
||||||
- name: Install dependencies
|
|
||||||
run: |
|
|
||||||
poetry config virtualenvs.in-project true
|
|
||||||
poetry install --no-root
|
|
||||||
|
|
||||||
- name: Run Black
|
|
||||||
run: poetry run black --check src/ tests/
|
|
||||||
|
|
||||||
- name: Run Ruff
|
|
||||||
run: poetry run ruff check src/ tests/
|
|
||||||
|
|
||||||
- name: Run MyPy
|
|
||||||
run: poetry run mypy src/
|
|
||||||
@@ -1,51 +0,0 @@
|
|||||||
# Security Scanning
|
|
||||||
|
|
||||||
name: Security
|
|
||||||
|
|
||||||
on:
|
|
||||||
push:
|
|
||||||
branches: [ main, develop ]
|
|
||||||
pull_request:
|
|
||||||
branches: [ main, develop ]
|
|
||||||
schedule:
|
|
||||||
- cron: '0 0 * * 1' # Weekly on Monday
|
|
||||||
|
|
||||||
env:
|
|
||||||
POETRY_VERSION: 1.8.0
|
|
||||||
PYTHON_VERSION: "3.12"
|
|
||||||
|
|
||||||
jobs:
|
|
||||||
security:
|
|
||||||
name: Security Scanning
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
|
|
||||||
steps:
|
|
||||||
- name: Checkout code
|
|
||||||
uses: actions/checkout@v4
|
|
||||||
|
|
||||||
- name: Set up Python
|
|
||||||
uses: actions/setup-python@v5
|
|
||||||
with:
|
|
||||||
python-version: ${{ env.PYTHON_VERSION }}
|
|
||||||
|
|
||||||
- name: Install Poetry
|
|
||||||
run: |
|
|
||||||
curl -sSL https://install.python-poetry.org | python3 -
|
|
||||||
echo "$HOME/.local/bin" >> $GITHUB_PATH
|
|
||||||
|
|
||||||
- name: Install dependencies
|
|
||||||
run: |
|
|
||||||
poetry config virtualenvs.in-project true
|
|
||||||
poetry install
|
|
||||||
|
|
||||||
- name: Run Bandit
|
|
||||||
run: |
|
|
||||||
poetry run bandit -r src/ -f json -o bandit-report.json
|
|
||||||
continue-on-error: true
|
|
||||||
|
|
||||||
- name: Upload Bandit report
|
|
||||||
uses: actions/upload-artifact@v4
|
|
||||||
with:
|
|
||||||
name: bandit-report
|
|
||||||
path: bandit-report.json
|
|
||||||
continue-on-error: true
|
|
||||||
@@ -1,71 +0,0 @@
|
|||||||
# Tests and Coverage
|
|
||||||
|
|
||||||
name: Test
|
|
||||||
|
|
||||||
on:
|
|
||||||
push:
|
|
||||||
branches: [ main, develop ]
|
|
||||||
pull_request:
|
|
||||||
branches: [ main, develop ]
|
|
||||||
|
|
||||||
env:
|
|
||||||
POETRY_VERSION: 1.8.0
|
|
||||||
PYTHON_VERSION: "3.12"
|
|
||||||
|
|
||||||
jobs:
|
|
||||||
test:
|
|
||||||
name: Run Tests
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
|
|
||||||
services:
|
|
||||||
mongodb:
|
|
||||||
image: mongo:7-jammy
|
|
||||||
ports:
|
|
||||||
- 27017:27017
|
|
||||||
options: >-
|
|
||||||
--health-cmd "mongosh --eval 'db.adminCommand(\"ping\")'"
|
|
||||||
--health-interval 10s
|
|
||||||
--health-timeout 5s
|
|
||||||
--health-retries 5
|
|
||||||
|
|
||||||
steps:
|
|
||||||
- name: Checkout code
|
|
||||||
uses: actions/checkout@v4
|
|
||||||
|
|
||||||
- name: Set up Python
|
|
||||||
uses: actions/setup-python@v5
|
|
||||||
with:
|
|
||||||
python-version: ${{ env.PYTHON_VERSION }}
|
|
||||||
cache: 'pip'
|
|
||||||
|
|
||||||
- name: Install Poetry
|
|
||||||
run: |
|
|
||||||
curl -sSL https://install.python-poetry.org | python3 -
|
|
||||||
echo "$HOME/.local/bin" >> $GITHUB_PATH
|
|
||||||
|
|
||||||
- name: Install dependencies
|
|
||||||
run: |
|
|
||||||
poetry config virtualenvs.in-project true
|
|
||||||
poetry install
|
|
||||||
|
|
||||||
- name: Run unit tests
|
|
||||||
env:
|
|
||||||
MONGODB_URL: mongodb://localhost:27017
|
|
||||||
MONGODB_DATABASE: testdb
|
|
||||||
run: |
|
|
||||||
poetry run pytest tests/unit -v --cov --cov-report=xml --cov-report=html
|
|
||||||
|
|
||||||
- name: Upload coverage
|
|
||||||
uses: codecov/codecov-action@v3
|
|
||||||
with:
|
|
||||||
files: ./coverage.xml
|
|
||||||
flags: unittests
|
|
||||||
name: codecov-umbrella
|
|
||||||
continue-on-error: true
|
|
||||||
|
|
||||||
- name: Archive coverage results
|
|
||||||
uses: actions/upload-artifact@v4
|
|
||||||
with:
|
|
||||||
name: coverage-report
|
|
||||||
path: htmlcov/
|
|
||||||
continue-on-error: true
|
|
||||||
Reference in New Issue
Block a user