Some checks failed
CI/CD Pipeline / Run Tests (push) Has been cancelled
CI/CD Pipeline / Security Scanning (push) Has been cancelled
CI/CD Pipeline / Build and Push Docker Images (api) (push) Has been cancelled
CI/CD Pipeline / Build and Push Docker Images (chat) (push) Has been cancelled
CI/CD Pipeline / Build and Push Docker Images (frontend) (push) Has been cancelled
CI/CD Pipeline / Build and Push Docker Images (worker) (push) Has been cancelled
CI/CD Pipeline / Deploy to Staging (push) Has been cancelled
CI/CD Pipeline / Deploy to Production (push) Has been cancelled
CI/CD Pipeline / Generate Documentation (push) Has started running
CI/CD Pipeline / Lint Code (push) Has started running
Implement a scalable system for automatic documentation generation from infrastructure systems, preventing LLM context overload through template-driven sectioning. **New Features:** 1. **YAML Template System** (`templates/documentation/proxmox.yaml`) - Define documentation sections independently - Specify data requirements per section - Configure prompts, generation settings, and scheduling - Prevents LLM context overflow by sectioning data 2. **Template-Based Generator** (`src/datacenter_docs/generators/template_generator.py`) - Load and parse YAML templates - Generate documentation sections independently - Extract only required data for each section - Save sections individually to files and database - Combine sections with table of contents 3. **Celery Tasks** (`src/datacenter_docs/workers/documentation_tasks.py`) - `collect_and_generate_docs`: Collect data and generate docs - `generate_proxmox_docs`: Scheduled Proxmox documentation (daily at 2 AM) - `generate_all_docs`: Generate docs for all systems in parallel - `index_generated_docs`: Index generated docs into vector store for RAG - `full_docs_pipeline`: Complete workflow (collect → generate → index) 4. **Scheduled Jobs** (updated `celery_app.py`) - Daily Proxmox documentation generation - Every 6 hours: all systems documentation - Weekly: full pipeline with indexing - Proper task routing and rate limiting 5. **Test Script** (`scripts/test_proxmox_docs.py`) - End-to-end testing of documentation generation - Mock data collection from Proxmox - Template-based generation - File and database storage 6. **Configuration Updates** (`src/datacenter_docs/utils/config.py`) - Add port configuration fields for Docker services - Add MongoDB and Redis credentials - Support all required environment variables **Proxmox Documentation Sections:** - Infrastructure Overview (cluster, nodes, stats) - Virtual Machines Inventory - LXC Containers Inventory - Storage Configuration - Network Configuration - Maintenance Procedures **Benefits:** - Scalable to multiple infrastructure systems - Prevents LLM context window overflow - Independent section generation - Scheduled automatic updates - Vector store integration for RAG chat - Template-driven approach for consistency 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
222 lines
6.2 KiB
YAML
222 lines
6.2 KiB
YAML
# Proxmox Documentation Template
|
|
# Defines documentation sections to generate from Proxmox data
|
|
# Each section is generated independently to avoid LLM context overload
|
|
|
|
metadata:
|
|
name: "Proxmox Virtual Environment"
|
|
collector: "proxmox"
|
|
version: "1.0.0"
|
|
description: "Documentation template for Proxmox VE infrastructure"
|
|
|
|
# Documentation sections - each generates a separate markdown file
|
|
sections:
|
|
- id: "proxmox_overview"
|
|
title: "Proxmox Infrastructure Overview"
|
|
category: "infrastructure"
|
|
priority: 1
|
|
description: "High-level overview of Proxmox cluster and resources"
|
|
data_requirements:
|
|
- "cluster"
|
|
- "statistics"
|
|
- "nodes"
|
|
prompt_template: |
|
|
Generate comprehensive documentation for our Proxmox Virtual Environment cluster.
|
|
|
|
**Cluster Information:**
|
|
{cluster}
|
|
|
|
**Infrastructure Statistics:**
|
|
{statistics}
|
|
|
|
**Nodes:**
|
|
{nodes}
|
|
|
|
Create a well-structured markdown document that includes:
|
|
1. Cluster overview with key statistics
|
|
2. Node inventory and status
|
|
3. Resource allocation summary (CPU, RAM, Storage)
|
|
4. High availability status
|
|
5. Capacity planning insights
|
|
|
|
Use tables, bullet points, and clear sections. Include actual values from the data.
|
|
|
|
- id: "proxmox_vms"
|
|
title: "Virtual Machines Inventory"
|
|
category: "virtualization"
|
|
priority: 2
|
|
description: "Complete inventory of QEMU virtual machines"
|
|
data_requirements:
|
|
- "vms"
|
|
- "nodes"
|
|
prompt_template: |
|
|
Generate detailed documentation for all virtual machines in the Proxmox cluster.
|
|
|
|
**Virtual Machines:**
|
|
{vms}
|
|
|
|
**Nodes:**
|
|
{nodes}
|
|
|
|
Create documentation that includes:
|
|
1. VM inventory table (VMID, Name, Node, Status, vCPU, RAM, Disk)
|
|
2. VMs grouped by node
|
|
3. VMs grouped by status (running/stopped)
|
|
4. Resource allocation per VM
|
|
5. Naming conventions and patterns observed
|
|
6. Recommendations for VM placement and balancing
|
|
|
|
Use markdown tables and organize information clearly.
|
|
|
|
- id: "proxmox_containers"
|
|
title: "LXC Containers Inventory"
|
|
category: "virtualization"
|
|
priority: 3
|
|
description: "Complete inventory of LXC containers"
|
|
data_requirements:
|
|
- "containers"
|
|
- "nodes"
|
|
prompt_template: |
|
|
Generate detailed documentation for all LXC containers in the Proxmox cluster.
|
|
|
|
**Containers:**
|
|
{containers}
|
|
|
|
**Nodes:**
|
|
{nodes}
|
|
|
|
Create documentation that includes:
|
|
1. Container inventory table (VMID, Name, Node, Status, vCPU, RAM, Disk)
|
|
2. Containers grouped by node
|
|
3. Containers grouped by status (running/stopped)
|
|
4. Resource allocation per container
|
|
5. Use cases and patterns for containers vs VMs
|
|
6. Recommendations for container management
|
|
|
|
Use markdown tables and clear organization.
|
|
|
|
- id: "proxmox_storage"
|
|
title: "Storage Configuration"
|
|
category: "storage"
|
|
priority: 4
|
|
description: "Storage pools and allocation"
|
|
data_requirements:
|
|
- "storage"
|
|
- "statistics"
|
|
prompt_template: |
|
|
Generate comprehensive storage documentation for the Proxmox cluster.
|
|
|
|
**Storage Pools:**
|
|
{storage}
|
|
|
|
**Overall Statistics:**
|
|
{statistics}
|
|
|
|
Create documentation that includes:
|
|
1. Storage inventory table (Name, Type, Total, Used, Available, Usage %)
|
|
2. Storage types explained (local, NFS, Ceph, etc.)
|
|
3. Content types per storage (images, ISO, containers)
|
|
4. Storage capacity analysis
|
|
5. Performance considerations
|
|
6. Backup storage recommendations
|
|
7. Capacity planning and alerts
|
|
|
|
Use markdown tables, charts (if possible), and clear sections.
|
|
|
|
- id: "proxmox_networking"
|
|
title: "Network Configuration"
|
|
category: "network"
|
|
priority: 5
|
|
description: "Network bridges and configuration"
|
|
data_requirements:
|
|
- "networks"
|
|
- "nodes"
|
|
prompt_template: |
|
|
Generate network configuration documentation for the Proxmox cluster.
|
|
|
|
**Network Interfaces:**
|
|
{networks}
|
|
|
|
**Nodes:**
|
|
{nodes}
|
|
|
|
Create documentation that includes:
|
|
1. Network bridges inventory (Bridge, Type, CIDR, Ports, Purpose)
|
|
2. Network topology diagram (text-based or description)
|
|
3. VLAN configuration if present
|
|
4. Network purposes (management, VM, storage, etc.)
|
|
5. Best practices for network separation
|
|
6. Troubleshooting guides for common network issues
|
|
|
|
Use markdown tables and clear explanations.
|
|
|
|
- id: "proxmox_maintenance"
|
|
title: "Maintenance Procedures"
|
|
category: "operations"
|
|
priority: 6
|
|
description: "Standard maintenance and operational procedures"
|
|
data_requirements:
|
|
- "nodes"
|
|
- "cluster"
|
|
- "vms"
|
|
- "containers"
|
|
prompt_template: |
|
|
Generate operational and maintenance documentation for the Proxmox cluster.
|
|
|
|
**Cluster Info:**
|
|
{cluster}
|
|
|
|
**Nodes:**
|
|
{nodes}
|
|
|
|
Based on the cluster configuration, create documentation that includes:
|
|
1. **Backup Procedures**
|
|
- VM/Container backup strategies
|
|
- Configuration backup
|
|
- Retention policies
|
|
|
|
2. **Update Procedures**
|
|
- Proxmox VE updates
|
|
- Kernel updates
|
|
- Rolling updates for HA clusters
|
|
|
|
3. **Monitoring**
|
|
- Key metrics to monitor
|
|
- Alert thresholds
|
|
- Dashboard recommendations
|
|
|
|
4. **Common Tasks**
|
|
- Creating VMs/Containers
|
|
- Migration procedures
|
|
- Storage management
|
|
- Snapshot management
|
|
|
|
5. **Troubleshooting**
|
|
- Common issues and solutions
|
|
- Log locations
|
|
- Recovery procedures
|
|
|
|
6. **Emergency Contacts**
|
|
- Escalation procedures
|
|
- Vendor support information
|
|
|
|
Make it practical and actionable for operations team.
|
|
|
|
# Generation settings
|
|
generation:
|
|
max_tokens: 4000
|
|
temperature: 0.7
|
|
language: "en" # Default language, can be overridden
|
|
|
|
# Output configuration
|
|
output:
|
|
directory: "output/proxmox"
|
|
filename_pattern: "{section_id}.md"
|
|
save_to_database: true
|
|
save_to_file: true
|
|
|
|
# Scheduling (for Celery tasks)
|
|
schedule:
|
|
enabled: true
|
|
cron: "0 2 * * *" # Daily at 2 AM
|
|
timezone: "UTC"
|