Initial commit: LLM Automation Docs & Remediation Engine v2.0
Features: - Automated datacenter documentation generation - MCP integration for device connectivity - Auto-remediation engine with safety checks - Multi-factor reliability scoring (0-100%) - Human feedback learning loop - Pattern recognition and continuous improvement - Agentic chat support with AI - API for ticket resolution - Frontend React with Material-UI - CI/CD pipelines (GitLab + Gitea) - Docker & Kubernetes deployment - Complete documentation and guides v2.0 Highlights: - Auto-remediation with write operations (disabled by default) - Reliability calculator with 4-factor scoring - Human feedback system for continuous learning - Pattern-based progressive automation - Approval workflow for critical actions - Full audit trail and rollback capability
This commit is contained in:
LLM Automation System
511
README_WEB.md
Normal file
511
README_WEB.md
Normal file
@@ -0,0 +1,511 @@
|
||||
# 🌐 Sistema Web e MCP - Documentazione Datacenter
|
||||
|
||||
Sistema completo per pubblicazione web della documentazione datacenter con API REST e MCP Server per connessioni LLM alle infrastrutture.
|
||||
|
||||
## 📦 Componenti
|
||||
|
||||
### 1. FastAPI Documentation Server
|
||||
- **Porta**: 8000
|
||||
- **Funzione**: Serve documentazione MkDocs compilata + API REST
|
||||
- **Features**:
|
||||
- Documentazione web responsive
|
||||
- API REST per accesso programmatico
|
||||
- Ottimizzazione per LLM
|
||||
- Search full-text
|
||||
- Statistics e metadata
|
||||
|
||||
### 2. MCP Server
|
||||
- **Porta**: 8001
|
||||
- **Funzione**: Model Context Protocol - Connessioni infrastruttura
|
||||
- **Features**:
|
||||
- SSH execution
|
||||
- SNMP queries
|
||||
- API REST integration
|
||||
- VMware, Cisco, storage shortcuts
|
||||
- Audit logging
|
||||
|
||||
### 3. MkDocs Static Site
|
||||
- **Framework**: Material for MkDocs
|
||||
- **Build**: Automatico via CI/CD
|
||||
- **Features**:
|
||||
- Responsive design
|
||||
- Dark mode
|
||||
- Search integrata
|
||||
- Git revision dates
|
||||
- Navigation ottimizzata
|
||||
|
||||
### 4. Nginx Reverse Proxy
|
||||
- **Porta**: 80 (HTTP) → 443 (HTTPS)
|
||||
- **Funzione**: SSL termination, caching, rate limiting
|
||||
- **Features**:
|
||||
- HTTPS con TLS 1.2+
|
||||
- Gzip compression
|
||||
- Static file caching
|
||||
- Security headers
|
||||
|
||||
## 🚀 Quick Start
|
||||
|
||||
### Prerequisiti
|
||||
```bash
|
||||
- Docker & Docker Compose
|
||||
- Git
|
||||
- Accesso management network
|
||||
```
|
||||
|
||||
### Setup Iniziale
|
||||
|
||||
1. **Clone repository**
|
||||
```bash
|
||||
git clone https://github.com/company/datacenter-docs.git
|
||||
cd datacenter-docs
|
||||
```
|
||||
|
||||
2. **Configura credenziali**
|
||||
```bash
|
||||
# Crea file MCP config
|
||||
cp config/mcp_config.example.json config/mcp_config.json
|
||||
# Edita con credenziali reali
|
||||
vim config/mcp_config.json
|
||||
|
||||
# Crea .env per Docker
|
||||
cat > .env << 'EOF'
|
||||
VCENTER_PASSWORD=your_password
|
||||
SWITCH_PASSWORD=your_password
|
||||
STORAGE_API_KEY=your_api_key
|
||||
EOF
|
||||
```
|
||||
|
||||
3. **Build e avvia servizi**
|
||||
```bash
|
||||
# Build documentazione
|
||||
./scripts/build-docs.sh
|
||||
|
||||
# Avvia con Docker Compose
|
||||
docker-compose up -d
|
||||
|
||||
# Verifica health
|
||||
curl http://localhost:8000/health
|
||||
curl http://localhost:8001/methods
|
||||
```
|
||||
|
||||
4. **Accedi alla documentazione**
|
||||
```
|
||||
http://localhost:8000/docs/
|
||||
http://localhost:8000/api/docs (API Swagger)
|
||||
http://localhost:8001/docs (MCP Swagger)
|
||||
```
|
||||
|
||||
## 📁 Struttura File
|
||||
|
||||
```
|
||||
datacenter-docs/
|
||||
├── api/ # FastAPI application
|
||||
│ ├── main.py # Main FastAPI app
|
||||
│ └── requirements-api.txt # Python dependencies
|
||||
├── mcp-server/ # MCP Server
|
||||
│ └── server.py # MCP implementation
|
||||
├── docs/ # MkDocs source
|
||||
│ ├── index.md # Homepage
|
||||
│ ├── sections/ # Documentation sections
|
||||
│ └── api/ # API documentation
|
||||
├── templates/ # Template documentazione
|
||||
├── nginx/ # Nginx configuration
|
||||
│ └── nginx.conf
|
||||
├── scripts/ # Utility scripts
|
||||
│ ├── build-docs.sh
|
||||
│ └── deploy.sh
|
||||
├── .github/workflows/ # CI/CD pipelines
|
||||
│ └── build-deploy.yml
|
||||
├── config/ # Configuration files
|
||||
│ └── mcp_config.json
|
||||
├── mkdocs.yml # MkDocs configuration
|
||||
├── Dockerfile # Multi-stage Dockerfile
|
||||
├── docker-compose.yml # Docker Compose config
|
||||
└── docker-entrypoint.sh # Container entrypoint
|
||||
```
|
||||
|
||||
## 🔄 Workflow Automazione
|
||||
|
||||
### 1. Generazione Documentazione
|
||||
```bash
|
||||
# LLM genera/aggiorna template
|
||||
python3 main.py --section 01
|
||||
|
||||
# Commit su Git
|
||||
git add templates/
|
||||
git commit -m "docs: update infrastructure section"
|
||||
git push origin main
|
||||
```
|
||||
|
||||
### 2. CI/CD Pipeline
|
||||
```
|
||||
Push to main
|
||||
↓
|
||||
GitHub Actions triggered
|
||||
↓
|
||||
├─ Lint & Validate
|
||||
├─ Build MkDocs
|
||||
├─ Build Docker Image
|
||||
├─ Security Scan
|
||||
└─ Deploy to Production
|
||||
↓
|
||||
Documentation live!
|
||||
```
|
||||
|
||||
### 3. Accesso Documentazione
|
||||
```
|
||||
User → Nginx → FastAPI → MkDocs Site
|
||||
↓
|
||||
API REST
|
||||
↓
|
||||
LLM-optimized
|
||||
```
|
||||
|
||||
## 🔌 API Usage
|
||||
|
||||
### Python Client Example
|
||||
```python
|
||||
import requests
|
||||
|
||||
# Get all sections
|
||||
r = requests.get('http://localhost:8000/api/v1/sections')
|
||||
sections = r.json()
|
||||
|
||||
for section in sections:
|
||||
print(f"{section['title']}: {section['token_estimate']} tokens")
|
||||
|
||||
# Get specific section
|
||||
r = requests.get('http://localhost:8000/api/v1/sections/02_networking')
|
||||
content = r.json()
|
||||
print(content['content'])
|
||||
|
||||
# LLM-optimized content
|
||||
r = requests.get('http://localhost:8000/api/v1/llm-optimized/02_networking')
|
||||
llm_data = r.json()
|
||||
print(f"Ready for LLM: {llm_data['token_count']} tokens")
|
||||
```
|
||||
|
||||
### cURL Examples
|
||||
```bash
|
||||
# Health check
|
||||
curl http://localhost:8000/health
|
||||
|
||||
# Get summary
|
||||
curl http://localhost:8000/api/v1/summary | jq
|
||||
|
||||
# Search
|
||||
curl "http://localhost:8000/api/v1/search?q=vmware" | jq
|
||||
|
||||
# Get section as HTML
|
||||
curl "http://localhost:8000/api/v1/sections/03_server_virtualizzazione?format=html"
|
||||
```
|
||||
|
||||
## 🤖 MCP Usage
|
||||
|
||||
### Python MCP Client
|
||||
```python
|
||||
import asyncio
|
||||
import requests
|
||||
|
||||
async def query_infrastructure():
|
||||
base_url = 'http://localhost:8001'
|
||||
|
||||
# List available methods
|
||||
r = requests.get(f'{base_url}/methods')
|
||||
print(r.json())
|
||||
|
||||
# Execute SSH command
|
||||
r = requests.post(f'{base_url}/execute/ssh', json={
|
||||
'connection_name': 'switch-core-01',
|
||||
'command': 'show version'
|
||||
})
|
||||
result = r.json()
|
||||
print(f"Output: {result['output']}")
|
||||
|
||||
# SNMP query
|
||||
r = requests.post(f'{base_url}/execute/snmp/get', json={
|
||||
'connection_name': 'ups-01',
|
||||
'oid': '.1.3.6.1.2.1.33.1.2.1.0'
|
||||
})
|
||||
ups_status = r.json()
|
||||
print(f"UPS Status: {ups_status['output']}")
|
||||
|
||||
asyncio.run(query_infrastructure())
|
||||
```
|
||||
|
||||
### Available MCP Methods
|
||||
- `ssh_execute` - Execute commands via SSH
|
||||
- `ssh_get_config` - Get device configurations
|
||||
- `snmp_get` - SNMP GET query
|
||||
- `snmp_walk` - SNMP WALK query
|
||||
- `api_request` - Generic API call
|
||||
- `vmware_get_vms` - Get VMware VMs
|
||||
- `vmware_get_hosts` - Get ESXi hosts
|
||||
- `cisco_get_interfaces` - Cisco interface status
|
||||
- `ups_get_status` - UPS status via SNMP
|
||||
|
||||
## 🔐 Security
|
||||
|
||||
### Access Control
|
||||
```yaml
|
||||
Documentation (port 8000):
|
||||
- Public read access (internal network)
|
||||
- API key for external access
|
||||
|
||||
MCP Server (port 8001):
|
||||
- Internal network only
|
||||
- No external exposure
|
||||
- Audit logging enabled
|
||||
- Read-only operations
|
||||
```
|
||||
|
||||
### Secrets Management
|
||||
```bash
|
||||
# Use environment variables
|
||||
export VCENTER_PASSWORD="..."
|
||||
export SWITCH_PASSWORD="..."
|
||||
|
||||
# Or use Docker secrets
|
||||
docker secret create vcenter_pass vcenter_password.txt
|
||||
|
||||
# Or use HashiCorp Vault
|
||||
vault kv get -field=password datacenter/vcenter
|
||||
```
|
||||
|
||||
### Network Security
|
||||
```bash
|
||||
# Firewall rules
|
||||
# Allow: Management network → MCP Server
|
||||
# Allow: Internal network → Documentation
|
||||
# Deny: External → MCP Server
|
||||
# Allow: External → Documentation (with auth)
|
||||
```
|
||||
|
||||
## 📊 Monitoring
|
||||
|
||||
### Health Checks
|
||||
```bash
|
||||
# FastAPI health
|
||||
curl http://localhost:8000/health
|
||||
|
||||
# MCP health
|
||||
curl http://localhost:8001/methods
|
||||
|
||||
# Docker health
|
||||
docker ps
|
||||
docker-compose ps
|
||||
```
|
||||
|
||||
### Logs
|
||||
```bash
|
||||
# Application logs
|
||||
docker-compose logs -f docs-server
|
||||
|
||||
# Nginx logs
|
||||
docker-compose logs -f nginx
|
||||
|
||||
# Specific service
|
||||
docker-compose logs -f docs-server | grep ERROR
|
||||
```
|
||||
|
||||
### Metrics
|
||||
```bash
|
||||
# Documentation statistics
|
||||
curl http://localhost:8000/api/v1/stats | jq
|
||||
|
||||
# Response times
|
||||
curl -w "@curl-format.txt" -o /dev/null -s http://localhost:8000/health
|
||||
```
|
||||
|
||||
## 🛠️ Development
|
||||
|
||||
### Local Development
|
||||
```bash
|
||||
# Install dependencies
|
||||
pip install -r requirements.txt
|
||||
pip install -r api/requirements-api.txt
|
||||
|
||||
# Run FastAPI locally
|
||||
cd api
|
||||
uvicorn main:app --reload --port 8000
|
||||
|
||||
# Run MCP server locally
|
||||
cd mcp-server
|
||||
uvicorn server:mcp_app --reload --port 8001
|
||||
|
||||
# Build docs locally
|
||||
mkdocs serve
|
||||
```
|
||||
|
||||
### Testing
|
||||
```bash
|
||||
# Run tests
|
||||
pytest tests/ -v
|
||||
|
||||
# Coverage
|
||||
pytest tests/ --cov=api --cov=mcp-server --cov-report=html
|
||||
|
||||
# Linting
|
||||
flake8 api/ mcp-server/
|
||||
black --check api/ mcp-server/
|
||||
```
|
||||
|
||||
## 🚢 Deployment
|
||||
|
||||
### Production Deployment
|
||||
```bash
|
||||
# Via script
|
||||
./scripts/deploy.sh
|
||||
|
||||
# Manual
|
||||
docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d
|
||||
|
||||
# Verify
|
||||
curl https://docs.datacenter.local/health
|
||||
```
|
||||
|
||||
### Update Documentation
|
||||
```bash
|
||||
# Pull latest
|
||||
git pull origin main
|
||||
|
||||
# Rebuild
|
||||
docker-compose build docs-server
|
||||
|
||||
# Rolling update
|
||||
docker-compose up -d --no-deps docs-server
|
||||
```
|
||||
|
||||
### Rollback
|
||||
```bash
|
||||
# Rollback to previous image
|
||||
docker-compose down
|
||||
docker-compose up -d docs-server:previous-tag
|
||||
|
||||
# Or restore from backup
|
||||
cp -r backup/docs/* docs/
|
||||
docker-compose restart docs-server
|
||||
```
|
||||
|
||||
## 📝 Configuration
|
||||
|
||||
### Environment Variables
|
||||
```bash
|
||||
# Application
|
||||
ENVIRONMENT=production
|
||||
LOG_LEVEL=info
|
||||
|
||||
# MCP Connections
|
||||
VCENTER_PASSWORD=xxx
|
||||
SWITCH_PASSWORD=xxx
|
||||
STORAGE_API_KEY=xxx
|
||||
|
||||
# Optional
|
||||
REDIS_URL=redis://localhost:6379
|
||||
DATABASE_URL=postgresql://user:pass@localhost/db
|
||||
```
|
||||
|
||||
### MkDocs Configuration
|
||||
Edit `mkdocs.yml`:
|
||||
```yaml
|
||||
site_name: Your Site Name
|
||||
theme:
|
||||
name: material
|
||||
palette:
|
||||
primary: indigo
|
||||
nav:
|
||||
- Home: index.md
|
||||
# ...
|
||||
```
|
||||
|
||||
### Nginx Configuration
|
||||
Edit `nginx/nginx.conf`:
|
||||
```nginx
|
||||
# Rate limiting
|
||||
limit_req_zone $binary_remote_addr zone=api_limit:10m rate=10r/s;
|
||||
|
||||
# SSL certificates
|
||||
ssl_certificate /etc/nginx/ssl/cert.pem;
|
||||
ssl_certificate_key /etc/nginx/ssl/key.pem;
|
||||
```
|
||||
|
||||
## 🔍 Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
**Port già in uso**
|
||||
```bash
|
||||
# Check what's using port
|
||||
sudo lsof -i :8000
|
||||
sudo lsof -i :8001
|
||||
|
||||
# Stop conflicting service
|
||||
sudo systemctl stop service_name
|
||||
```
|
||||
|
||||
**Docker build failed**
|
||||
```bash
|
||||
# Clean build
|
||||
docker-compose build --no-cache docs-server
|
||||
|
||||
# Check logs
|
||||
docker-compose logs docs-server
|
||||
```
|
||||
|
||||
**MCP connection errors**
|
||||
```bash
|
||||
# Test connectivity
|
||||
telnet switch.domain.local 22
|
||||
snmpget -v2c -c public ups.domain.local .1.3.6.1.2.1.1.1.0
|
||||
|
||||
# Check config
|
||||
cat config/mcp_config.json | jq
|
||||
|
||||
# Test connection
|
||||
curl -X GET http://localhost:8001/test/switch-core-01
|
||||
```
|
||||
|
||||
**Documentation not updating**
|
||||
```bash
|
||||
# Rebuild docs
|
||||
./scripts/build-docs.sh
|
||||
|
||||
# Force rebuild
|
||||
docker-compose down
|
||||
docker-compose up -d --build
|
||||
|
||||
# Check pipeline
|
||||
# Go to GitHub Actions and check logs
|
||||
```
|
||||
|
||||
## 📚 Additional Resources
|
||||
|
||||
- [MkDocs Documentation](https://www.mkdocs.org/)
|
||||
- [Material for MkDocs](https://squidfunk.github.io/mkdocs-material/)
|
||||
- [FastAPI Documentation](https://fastapi.tiangolo.com/)
|
||||
- [Docker Compose](https://docs.docker.com/compose/)
|
||||
|
||||
## 🤝 Contributing
|
||||
|
||||
1. Fork repository
|
||||
2. Create feature branch
|
||||
3. Make changes
|
||||
4. Test locally
|
||||
5. Submit pull request
|
||||
|
||||
## 📞 Support
|
||||
|
||||
- **Email**: automation-team@company.com
|
||||
- **Issues**: https://github.com/company/datacenter-docs/issues
|
||||
- **Wiki**: https://github.com/company/datacenter-docs/wiki
|
||||
|
||||
## 📄 License
|
||||
|
||||
Internal use only - Company Proprietary
|
||||
|
||||
---
|
||||
|
||||
**Sistema Web e MCP per Documentazione Datacenter**
|
||||
**Versione**: 1.0.0
|
||||
**Maintainer**: Automation Team
|
||||
**Last Update**: 2025-01-XX
|
||||
Reference in New Issue
Block a user