# π Automated Infrastructure Documentation System
Sistema automatizzato per la generazione e mantenimento della documentazione tecnica dell'infrastruttura aziendale tramite LLM locale con validazione umana e pubblicazione GitOps.
[](https://opensource.org/licenses/MIT)
[](https://www.python.org/downloads/)
[](https://redis.io/)
## π Indice
- [Overview](#overview)
- [Architettura](#architettura)
- [Schema Architetturale](#schema-architetturale)
- [Schema Tecnico](#schema-tecnico)
- [Contatti](#contatti)
## π― Overview
Sistema progettato per **automatizzare la creazione e l'aggiornamento della documentazione tecnica** di sistemi infrastrutturali complessi (VMware, Kubernetes, Linux, Cisco, ecc.) utilizzando un Large Language Model locale (Qwen).
### Caratteristiche Principali
- β
**Raccolta dati asincrona** da molteplici sistemi infrastrutturali
- β
**Isolamento di sicurezza**: LLM non accede mai ai sistemi live
- β
**Change Detection**: Documentazione generata solo su modifiche rilevate
- β
**Redis Cache** per storage dati e performance
- β
**LLM locale on-premise** (Qwen) tramite MCP Server
- β
**Human-in-the-loop validation** con workflow GitOps
- β
**CI/CD automatizzato** per pubblicazione
## ποΈ Architettura
Il sistema Γ¨ suddiviso in **3 flussi principali**:
1. **Raccolta Dati (Background)**: Connettori interrogano periodicamente i sistemi infrastrutturali tramite API e aggiornano Redis
2. **Change Detection**: Sistema di rilevamento modifiche che attiva la generazione documentazione solo quando necessario
3. **Generazione e Pubblicazione (Triggered)**: LLM locale (Qwen) genera markdown leggendo da Redis, seguito da review umana e deploy automatico
> **Principio di Sicurezza**: L'LLM non ha mai accesso diretto ai sistemi infrastrutturali. Tutti i dati sono letti da Redis.
> **Principio di Efficienza**: La documentazione viene generata solo quando il sistema rileva modifiche nella configurazione infrastrutturale.
---
## π Schema Architetturale
### Management View
Schema semplificato per presentazioni executive e management.
```mermaid
graph TB
%% Styling
classDef infrastructure fill:#e1f5ff,stroke:#01579b,stroke-width:3px,color:#333
classDef cache fill:#f3e5f5,stroke:#4a148c,stroke-width:3px,color:#333
classDef change fill:#fff3e0,stroke:#e65100,stroke-width:3px,color:#333
classDef llm fill:#e8f5e9,stroke:#1b5e20,stroke-width:3px,color:#333
classDef git fill:#fce4ec,stroke:#880e4f,stroke-width:3px,color:#333
classDef human fill:#fff9c4,stroke:#f57f17,stroke-width:3px,color:#333
%% ========================================
%% FLUSSO 1: RACCOLTA DATI (Background)
%% ========================================
INFRA[("π’ SISTEMI
INFRASTRUTTURALI
VMware | K8s | Linux | Cisco")]:::infrastructure
CONN["π CONNETTORI
Polling Automatico"]:::infrastructure
REDIS[("πΎ REDIS CACHE
Configurazione
Infrastruttura")]:::cache
INFRA -->|"API Polling
Continuo"| CONN
CONN -->|"Update
Configurazione"| REDIS
%% ========================================
%% CHANGE DETECTION
%% ========================================
CHANGE["π CHANGE DETECTOR
Rileva Modifiche
Configurazione"]:::change
REDIS -->|"Monitor
Changes"| CHANGE
%% ========================================
%% FLUSSO 2: GENERAZIONE DOCUMENTAZIONE (Triggered)
%% ========================================
TRIGGER["β‘ TRIGGER
Solo se modifiche"]:::change
USER["π€ UTENTE
Richiesta Manuale"]:::human
LLM["π€ LLM ENGINE
Qwen (Locale)"]:::llm
MCP["π§ MCP SERVER
API Control Platform"]:::llm
DOC["π DOCUMENTO
Markdown Generato"]:::llm
CHANGE -->|"Modifiche
Rilevate"| TRIGGER
USER -.->|"Opzionale"| TRIGGER
TRIGGER -->|"Avvia
Generazione"| LLM
LLM -->|"Tool Call"| MCP
MCP -->|"Query"| REDIS
REDIS -->|"Dati Config"| MCP
MCP -->|"Context"| LLM
LLM -->|"Genera"| DOC
%% ========================================
%% FLUSSO 3: VALIDAZIONE E PUBBLICAZIONE
%% ========================================
GIT["π¦ GITLAB
Repository"]:::git
PR["π PULL REQUEST
Review Automatica"]:::git
TECH["π¨βπΌ TEAM TECNICO
Validazione Umana"]:::human
PIPELINE["β‘ CI/CD PIPELINE
GitLab Runner"]:::git
MKDOCS["π MKDOCS
Static Site Generator"]:::git
WEB["π DOCUMENTAZIONE
GitLab Pages
(Pubblicata)"]:::git
DOC -->|"Push +
Branch"| GIT
GIT -->|"Crea"| PR
PR -->|"Notifica"| TECH
TECH -->|"Approva +
Merge"| GIT
GIT -->|"Trigger"| PIPELINE
PIPELINE -->|"Build"| MKDOCS
MKDOCS -->|"Deploy"| WEB
%% ========================================
%% ANNOTAZIONI
%% ========================================
SECURITY["π SICUREZZA
LLM isolato dai sistemi live"]:::human
EFFICIENCY["β‘ EFFICIENZA
Doc generata solo
su modifiche"]:::change
LLM -.->|"NESSUN
ACCESSO"| INFRA
SECURITY -.-> LLM
EFFICIENCY -.-> CHANGE
```
---
## π§ Schema Tecnico
### Implementation View
Schema dettagliato per il team tecnico con specifiche implementative.
```mermaid
graph TB
%% Styling tecnico
classDef infra fill:#e1f5ff,stroke:#01579b,stroke-width:2px,color:#333,font-size:11px
classDef connector fill:#e3f2fd,stroke:#1565c0,stroke-width:2px,color:#333,font-size:11px
classDef cache fill:#f3e5f5,stroke:#4a148c,stroke-width:2px,color:#333,font-size:11px
classDef change fill:#fff3e0,stroke:#e65100,stroke-width:2px,color:#333,font-size:11px
classDef llm fill:#e8f5e9,stroke:#1b5e20,stroke-width:2px,color:#333,font-size:11px
classDef git fill:#fce4ec,stroke:#880e4f,stroke-width:2px,color:#333,font-size:11px
classDef monitor fill:#fff8e1,stroke:#f57f17,stroke-width:2px,color:#333,font-size:11px
%% =====================================
%% LAYER 1: SISTEMI SORGENTE
%% =====================================
subgraph SOURCES["π’ INFRASTRUCTURE SOURCES"]
VCENTER["VMware vCenter
API: vSphere REST 7.0+
Port: 443/HTTPS
Auth: API Token"]:::infra
K8S_API["Kubernetes API
API: v1.28+
Port: 6443/HTTPS
Auth: ServiceAccount + RBAC"]:::infra
LINUX["Linux Servers
Protocol: SSH/Ansible
Port: 22
Auth: SSH Keys"]:::infra
CISCO["Cisco Devices
Protocol: NETCONF/RESTCONF
Port: 830/443
Auth: AAA"]:::infra
end
%% =====================================
%% LAYER 2: CONNETTORI
%% =====================================
subgraph CONNECTORS["π DATA COLLECTORS (Python/Go)"]
CONN_VM["VMware Collector
Lang: Python 3.11
Lib: pyvmomi
Schedule: */15 * * * *
Output: JSON β Redis"]:::connector
CONN_K8S["K8s Collector
Lang: Python 3.11
Lib: kubernetes-client
Schedule: */5 * * * *
Resources: pods,svc,ing,deploy"]:::connector
CONN_LNX["Linux Collector
Lang: Python 3.11
Lib: paramiko/ansible
Schedule: */30 * * * *
Data: sysinfo,packages,services"]:::connector
CONN_CSC["Cisco Collector
Lang: Python 3.11
Lib: ncclient
Schedule: */30 * * * *
Data: interfaces,routing,vlans"]:::connector
end
VCENTER -->|"GET /api/vcenter/vm"| CONN_VM
K8S_API -->|"kubectl proxy
API calls"| CONN_K8S
LINUX -->|"SSH batch
commands"| CONN_LNX
CISCO -->|"NETCONF
get-config"| CONN_CSC
%% =====================================
%% LAYER 3: REDIS STORAGE
%% =====================================
subgraph STORAGE["πΎ REDIS CLUSTER"]
REDIS_CLUSTER["Redis Cluster
Mode: Cluster (6 nodes)
Port: 6379
Persistence: RDB + AOF
Memory: 64GB
Eviction: allkeys-lru"]:::cache
REDIS_KEYS["Key Structure:
β’ vmware:vcenter-id:vms:hash
β’ k8s:cluster:namespace:resource:hash
β’ linux:hostname:info:hash
β’ cisco:device-id:config:hash
β’ changelog:timestamp:diff
TTL: 30d for data, 90d for changelog"]:::cache
end
CONN_VM -->|"HSET/HMSET
+ Hash Storage"| REDIS_CLUSTER
CONN_K8S -->|"HSET/HMSET
+ Hash Storage"| REDIS_CLUSTER
CONN_LNX -->|"HSET/HMSET
+ Hash Storage"| REDIS_CLUSTER
CONN_CSC -->|"HSET/HMSET
+ Hash Storage"| REDIS_CLUSTER
REDIS_CLUSTER --> REDIS_KEYS
%% =====================================
%% LAYER 4: CHANGE DETECTION
%% =====================================
subgraph CHANGE_DETECTION["π CHANGE DETECTION SYSTEM"]
DETECTOR["Change Detector Service
Lang: Python 3.11
Lib: redis-py
Algorithm: Hash comparison
Check interval: */5 * * * *"]:::change
DIFF_ENGINE["Diff Engine
β’ Deep object comparison
β’ JSON diff generation
β’ Change classification
β’ Severity assessment"]:::change
CHANGE_LOG["Change Log Store
Key: changelog:*
Data: diff JSON + metadata
Indexed by: timestamp, resource"]:::change
NOTIFIER["Change Notifier
β’ Webhook triggers
β’ Slack notifications
β’ Event emission
Target: LLM trigger"]:::change
end
REDIS_CLUSTER -->|"Monitor
key changes"| DETECTOR
DETECTOR --> DIFF_ENGINE
DIFF_ENGINE -->|"Store diff"| CHANGE_LOG
CHANGE_LOG --> REDIS_CLUSTER
DIFF_ENGINE -->|"Notify if
significant"| NOTIFIER
%% =====================================
%% LAYER 5: LLM TRIGGER & GENERATION
%% =====================================
subgraph TRIGGER_SYSTEM["β‘ TRIGGER SYSTEM"]
TRIGGER_SVC["Trigger Service
Lang: Python 3.11
Listen: Webhook + Redis Pub/Sub
Debounce: 5 min
Batch: multiple changes"]:::change
QUEUE["Generation Queue
Type: Redis List
Priority: High/Medium/Low
Processing: FIFO"]:::change
end
NOTIFIER -->|"Trigger event"| TRIGGER_SVC
TRIGGER_SVC -->|"Enqueue
generation task"| QUEUE
subgraph LLM_LAYER["π€ AI GENERATION LAYER"]
LLM_ENGINE["LLM Engine
Model: Qwen (Locale)
API: Ollama/vLLM/LM Studio
Port: 11434
Temp: 0.3
Max Tokens: 4096
Timeout: 120s"]:::llm
MCP_SERVER["MCP Server
Lang: TypeScript/Node.js
Port: 3000
Protocol: JSON-RPC 2.0
Auth: JWT tokens"]:::llm
MCP_TOOLS["MCP Tools:
β’ getVMwareInventory(vcenter)
β’ getK8sResources(cluster,ns,type)
β’ getLinuxSystemInfo(hostname)
β’ getCiscoConfig(device,section)
β’ getChangelog(start,end,resource)
Return: JSON + Metadata"]:::llm
end
QUEUE -->|"Dequeue
task"| LLM_ENGINE
LLM_ENGINE <-->|"Tool calls
JSON-RPC"| MCP_SERVER
MCP_SERVER --> MCP_TOOLS
MCP_TOOLS -->|"HGETALL/MGET
Read data"| REDIS_CLUSTER
REDIS_CLUSTER -->|"Config data
+ Changelog"| MCP_TOOLS
MCP_TOOLS -->|"Structured Data
+ Context"| LLM_ENGINE
subgraph OUTPUT["π DOCUMENT GENERATION"]
TEMPLATE["Template Engine
Format: Jinja2
Templates: markdown/*.j2
Variables: from LLM"]:::llm
MARKDOWN["Markdown Output
Format: CommonMark
Metadata: YAML frontmatter
Change summary included
Assets: diagrams in mermaid"]:::llm
VALIDATOR["Doc Validator
β’ Markdown linting
β’ Link checking
β’ Schema validation
β’ Change verification"]:::llm
end
LLM_ENGINE --> TEMPLATE
TEMPLATE --> MARKDOWN
MARKDOWN --> VALIDATOR
%% =====================================
%% LAYER 6: GITOPS
%% =====================================
subgraph GITOPS["π GITOPS WORKFLOW"]
GIT_REPO["GitLab Repository
URL: gitlab.com/docs/infra
Branch strategy: main + feature/*
Protected: main (require approval)"]:::git
GIT_API["GitLab API
API: v4
Auth: Project Access Token
Permissions: api, write_repo"]:::git
PR_AUTO["Automated PR Creator
Lang: Python 3.11
Lib: python-gitlab
Template: .gitlab/merge_request.md
Include: change summary"]:::git
end
VALIDATOR -->|"git add/commit/push"| GIT_REPO
GIT_REPO <--> GIT_API
GIT_API --> PR_AUTO
REVIEWER["π¨βπΌ Technical Reviewer
Role: Maintainer/Owner
Review: diff + validation
Check: change correlation
Approve: required (min 1)"]:::monitor
PR_AUTO -->|"Notification
Email + Slack"| REVIEWER
REVIEWER -->|"Merge to main"| GIT_REPO
%% =====================================
%% LAYER 7: CI/CD & PUBLISH
%% =====================================
subgraph CICD["β‘ CI/CD PIPELINE"]
GITLAB_CI["GitLab CI/CD
Runner: docker
Image: python:3.11-alpine
Stages: build, test, deploy"]:::git
PIPELINE_JOBS["Pipeline Jobs:
1. lint (markdownlint-cli)
2. build (mkdocs build)
3. test (link-checker)
4. deploy (rsync/s3)"]:::git
MKDOCS_CFG["MkDocs Config
Theme: material
Plugins: search, tags, mermaid
Extensions: admonition, codehilite"]:::git
end
GIT_REPO -->|"on: push to main
Webhook trigger"| GITLAB_CI
GITLAB_CI --> PIPELINE_JOBS
PIPELINE_JOBS --> MKDOCS_CFG
subgraph PUBLISH["π PUBLICATION"]
STATIC_SITE["Static Site
Generator: MkDocs
Output: HTML/CSS/JS
Assets: optimized images"]:::git
CDN["GitLab Pages / S3 + CloudFront
URL: docs.company.com
SSL: Let's Encrypt
Cache: 1h"]:::git
SEARCH["Search Index
Engine: Algolia/Meilisearch
Update: on publish
API: REST"]:::git
end
MKDOCS_CFG -->|"mkdocs build
--strict"| STATIC_SITE
STATIC_SITE --> CDN
STATIC_SITE --> SEARCH
%% =====================================
%% LAYER 8: MONITORING & OBSERVABILITY
%% =====================================
subgraph OBSERVABILITY["π MONITORING & LOGGING"]
PROMETHEUS["Prometheus
Metrics: collector updates, changes detected
Scrape: 30s
Retention: 15d"]:::monitor
GRAFANA["Grafana Dashboards
β’ Collector status
β’ Redis performance
β’ Change detection rate
β’ LLM response times
β’ Pipeline success rate"]:::monitor
ELK["ELK Stack
Logs: all components
Index: daily rotation
Retention: 30d"]:::monitor
ALERTS["Alerting
β’ Collector failures
β’ Redis issues
β’ Change detection errors
β’ Pipeline failures
Channel: Slack + PagerDuty"]:::monitor
end
CONN_VM -.->|"metrics"| PROMETHEUS
CONN_K8S -.->|"metrics"| PROMETHEUS
REDIS_CLUSTER -.->|"metrics"| PROMETHEUS
DETECTOR -.->|"metrics"| PROMETHEUS
MCP_SERVER -.->|"metrics"| PROMETHEUS
GITLAB_CI -.->|"metrics"| PROMETHEUS
PROMETHEUS --> GRAFANA
CONN_VM -.->|"logs"| ELK
DETECTOR -.->|"logs"| ELK
MCP_SERVER -.->|"logs"| ELK
GITLAB_CI -.->|"logs"| ELK
GRAFANA --> ALERTS
%% =====================================
%% SECURITY & EFFICIENCY ANNOTATIONS
%% =====================================
SEC1["π SECURITY:
β’ All APIs use TLS 1.3
β’ Secrets in Vault/K8s Secrets
β’ Network: private VPC
β’ LLM has NO direct access"]:::monitor
SEC2["π AUTHENTICATION:
β’ API Tokens rotated 90d
β’ RBAC enforced
β’ Audit logs enabled
β’ MFA required for Git"]:::monitor
EFF1["β‘ EFFICIENCY:
β’ Doc generation only on changes
β’ Debounce prevents spam
β’ Hash-based change detection
β’ Batch processing"]:::change
SEC1 -.-> MCP_SERVER
SEC2 -.-> GIT_REPO
EFF1 -.-> DETECTOR
```
---
## π§ Contatti
- **Team**: Infrastructure Documentation Team
- **Email**: infra-docs@company.com
- **GitLab**: https://gitlab.com/company/infra-docs-automation
---
**Versione**: 1.0.0
**Ultimo aggiornamento**: 2025-10-28