dnviti
d6d44270ee
Some checks failed
Build / Build and Push Docker Images (worker) (push) Waiting to run
Deploy Staging / Deploy to Staging (push) Waiting to run
Lint / Lint Code (push) Has been cancelled
Security / Security Scanning (push) Has been cancelled
Build / Build and Push Docker Images (api) (push) Has started running
Build / Build and Push Docker Images (chat) (push) Has started running
Build / Build and Push Docker Images (frontend) (push) Has started running
Test / Run Tests (push) Has been cancelled
BREAKING CHANGE: Monolithic ci.yml split into focused pipeline files ## Pipeline Reorganization Split single CI/CD pipeline into 7 specialized workflows: 1. **lint.yml** - Code quality checks (Black, Ruff, MyPy) 2. **test.yml** - Test suite with coverage reporting 3. **security.yml** - Security scanning (Bandit) 4. **build.yml** - Docker image builds and registry push 5. **deploy-staging.yml** - Staging environment deployment 6. **deploy-production.yml** - Production deployment (tags only) 7. **docs-generation.yml** - Scheduled documentation generation ## Benefits - **Modularity**: Each pipeline has single responsibility - **Performance**: Workflows run independently, faster feedback - **Clarity**: Easier to understand and maintain - **Flexibility**: Trigger pipelines independently - **Debugging**: Isolated failures easier to diagnose ## Dockerfile Improvements - Fix FROM AS casing (was 'as', now 'AS') in all Dockerfiles - Resolves Docker build warnings - Improves consistency across build files ## Documentation - Added .gitea/workflows/README.md with: - Workflow descriptions and triggers - Dependency diagram - Environment variables reference - Troubleshooting guide - Best practices ## Migration Notes - Old monolithic pipeline backed up as ci.yml.old - All triggers preserved from original pipeline - No changes to build or deploy logic - Same environment variables and secrets required 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
175 lines
5.1 KiB
Markdown
175 lines
5.1 KiB
Markdown
# CI/CD Workflows
|
|
|
|
This directory contains separated CI/CD pipeline workflows for the Datacenter Documentation System.
|
|
|
|
## Workflow Organization
|
|
|
|
Each workflow is designed for a specific purpose and can run independently:
|
|
|
|
### 1. Code Quality (`lint.yml`)
|
|
**Triggers**: Push/PR to main/develop
|
|
|
|
Runs code quality checks:
|
|
- **Black**: Code formatting verification
|
|
- **Ruff**: Linting and code quality
|
|
- **MyPy**: Static type checking
|
|
|
|
**Purpose**: Ensure code quality standards before merge.
|
|
|
|
---
|
|
|
|
### 2. Testing (`test.yml`)
|
|
**Triggers**: Push/PR to main/develop
|
|
|
|
Runs test suite with coverage:
|
|
- Unit tests with pytest
|
|
- Integration tests (when available)
|
|
- Coverage reporting to CodeCov
|
|
- MongoDB service container for database tests
|
|
|
|
**Purpose**: Verify functionality and prevent regressions.
|
|
|
|
---
|
|
|
|
### 3. Security (`security.yml`)
|
|
**Triggers**: Push/PR to main/develop, Weekly schedule (Monday)
|
|
|
|
Security scanning:
|
|
- **Bandit**: Python code security analysis
|
|
- **Safety**: Dependency vulnerability scanning (future)
|
|
|
|
**Purpose**: Identify security vulnerabilities early.
|
|
|
|
---
|
|
|
|
### 4. Build (`build.yml`)
|
|
**Triggers**: Push to main, Version tags
|
|
|
|
Builds and pushes Docker images:
|
|
- **Components**: API, Chat, Worker, Frontend
|
|
- **Registry**: Gitea Container Registry
|
|
- **Caching**: BuildKit cache for faster builds
|
|
- **Tagging**: Branch names, SHAs, semantic versions
|
|
|
|
**Purpose**: Create deployable container images.
|
|
|
|
---
|
|
|
|
### 5. Deploy - Staging (`deploy-staging.yml`)
|
|
**Triggers**: Push to main (after successful build)
|
|
|
|
Deploys to staging environment:
|
|
- **Platform**: Kubernetes
|
|
- **Namespace**: datacenter-docs
|
|
- **Components**: API, Chat, Worker
|
|
- **Validation**: Rollout status checks
|
|
|
|
**Purpose**: Test changes in staging before production.
|
|
|
|
---
|
|
|
|
### 6. Deploy - Production (`deploy-production.yml`)
|
|
**Triggers**: Version tags (v*)
|
|
|
|
Deploys to production environment:
|
|
- **Platform**: Kubernetes
|
|
- **Namespace**: datacenter-docs
|
|
- **Components**: API, Chat, Worker
|
|
- **Validation**: Rollout status + smoke tests
|
|
|
|
**Purpose**: Deploy stable versions to production.
|
|
|
|
---
|
|
|
|
### 7. Documentation Generation (`docs-generation.yml`)
|
|
**Triggers**: Schedule (every 6 hours), Manual
|
|
|
|
Automated documentation generation:
|
|
- Collects infrastructure data
|
|
- Generates documentation via LLM
|
|
- Uploads artifacts
|
|
- Commits changes (if any)
|
|
|
|
**Purpose**: Keep documentation up-to-date automatically.
|
|
|
|
---
|
|
|
|
## Workflow Dependencies
|
|
|
|
```
|
|
┌─────────────┐
|
|
│ lint.yml │ ─┐
|
|
└─────────────┘ │
|
|
│
|
|
┌─────────────┐ │
|
|
│ test.yml │ ─┤
|
|
└─────────────┘ │
|
|
├──> ┌──────────────┐ ┌────────────────────┐
|
|
┌─────────────┐ │ │ build.yml │ ──> │ deploy-staging.yml │
|
|
│security.yml │ ─┘ └──────────────┘ └────────────────────┘
|
|
└─────────────┘ │
|
|
│ (on tags)
|
|
▼
|
|
┌──────────────────────┐
|
|
│deploy-production.yml │
|
|
└──────────────────────┘
|
|
|
|
┌──────────────────────┐
|
|
│docs-generation.yml │ (independent, scheduled)
|
|
└──────────────────────┘
|
|
```
|
|
|
|
## Environment Variables
|
|
|
|
### Required Secrets
|
|
- `USERNAME`: Registry username
|
|
- `TOKEN`: Registry access token
|
|
- `LLM_API_KEY`: LLM provider API key
|
|
- `LLM_BASE_URL`: LLM provider base URL
|
|
- `MCP_SERVER_URL`: Infrastructure API endpoint
|
|
- `KUBE_CONFIG_STAGING`: Staging Kubernetes config
|
|
- `KUBE_CONFIG_PRODUCTION`: Production Kubernetes config
|
|
|
|
### Required Variables
|
|
- `PACKAGES_REGISTRY`: Container registry URL (e.g., `git.commandware.com`)
|
|
|
|
## Running Workflows
|
|
|
|
### Automatic Triggers
|
|
All workflows run automatically based on their configured triggers.
|
|
|
|
### Manual Triggers
|
|
Some workflows support manual triggering via Gitea Actions UI:
|
|
- `docs-generation.yml` (workflow_dispatch)
|
|
|
|
## Best Practices
|
|
|
|
1. **Always pass lint and test** before merging to main
|
|
2. **Review security reports** weekly
|
|
3. **Test in staging** before creating version tags
|
|
4. **Use semantic versioning** for production releases (v1.2.3)
|
|
5. **Monitor deployment rollouts** in Kubernetes
|
|
|
|
## Troubleshooting
|
|
|
|
### Build Failures
|
|
- Check Docker build logs
|
|
- Verify poetry.lock is committed
|
|
- Ensure all dependencies are available
|
|
|
|
### Test Failures
|
|
- Review test output in job logs
|
|
- Check MongoDB service health
|
|
- Verify environment variables
|
|
|
|
### Deployment Failures
|
|
- Check Kubernetes cluster status
|
|
- Verify secrets are configured
|
|
- Review deployment rollout logs
|
|
|
|
## Maintenance
|
|
|
|
- Review and update workflow configurations quarterly
|
|
- Update action versions when new releases are available
|
|
- Monitor workflow execution times and optimize as needed
|