llm-automation-docs-and-remediation-engine/MONGODB_GUIDE.md

🍃 MongoDB Migration Guide

Perché MongoDB?

Il sistema è stato aggiornato per utilizzare MongoDB 7.0 invece di PostgreSQL per i seguenti motivi:

✅ Vantaggi per questo Use Case

1. Schema Flessibile

   • Ticket metadata variabili senza migration
   • Facile aggiunta di nuovi campi
   • Supporto nativo per documenti JSON complessi

2. Performance

   • Ottime performance per operazioni di lettura
   • Aggregation pipeline potente per analytics
   • Indexing flessibile su campi nested

3. Scalabilità

   • Horizontal scaling nativo (sharding)
   • Replica set per high availability
   • Auto-failover integrato

4. Document-Oriented

   • Match perfetto per ticket system
   • Metadata JSON nativi
   • Embedding di related docs senza JOIN

5. Vector Search (Future)

   • MongoDB Atlas Vector Search integrato
   • Possibilità di sostituire ChromaDB
   • Unified database per docs + vectors

6. Developer Experience

   • Beanie ODM moderno con Pydantic
   • Async/await nativo con Motor
   • Type hints e validazione

🔄 Architettura Database

Collezioni Principali

datacenter_docs/
├── tickets                    # Ticket e risoluzioni
├── documentation_sections     # Metadata sezioni doc
├── chat_sessions             # Conversazioni chat
├── system_metrics            # Metriche sistema
└── audit_logs                # Audit trail

Schema Ticket (Example)

{
  "_id": ObjectId("..."),
  "ticket_id": "INC-12345",
  "title": "Network connectivity issue",
  "description": "Cannot ping 10.0.20.5 from VLAN 100",
  "priority": "high",
  "category": "network",
  "status": "resolved",
  "resolution": "Check VLAN configuration...",
  "suggested_actions": [
    "Verify VLAN 100 on switch",
    "Check inter-VLAN routing"
  ],
  "related_docs": [
    {
      "section": "networking",
      "content": "VLAN configuration...",
      "source": "/docs/02_networking.md"
    }
  ],
  "confidence_score": 0.92,
  "processing_time": 2.34,
  "metadata": {
    "source_system": "ServiceNow",
    "tags": ["network", "vlan", "connectivity"],
    "custom_field": "any value"
  },
  "created_at": ISODate("2025-01-15T10:30:00Z"),
  "updated_at": ISODate("2025-01-15T10:30:02Z")
}

🚀 Migration da PostgreSQL

Step 1: Export dati esistenti (se presenti)

# Export tickets da PostgreSQL
psql -U docs_user -d datacenter_docs -c \
  "COPY (SELECT * FROM tickets) TO '/tmp/tickets.csv' CSV HEADER"

Step 2: Import in MongoDB

import pandas as pd
from motor.motor_asyncio import AsyncIOMotorClient
import asyncio

async def migrate():
    # Leggi CSV
    df = pd.read_csv('/tmp/tickets.csv')
    
    # Connetti MongoDB
    client = AsyncIOMotorClient('mongodb://admin:password@localhost:27017')
    db = client.datacenter_docs
    
    # Insert documents
    tickets = df.to_dict('records')
    await db.tickets.insert_many(tickets)
    
    print(f"Migrated {len(tickets)} tickets")

asyncio.run(migrate())

Step 3: Verifica

# Connetti a MongoDB
mongosh mongodb://admin:password@localhost:27017

use datacenter_docs

# Conta documenti
db.tickets.countDocuments()

# Query esempio
db.tickets.find({status: "resolved"}).limit(5)

📦 Setup Locale

Docker Compose

# Start MongoDB
docker-compose up -d mongodb redis

# Verifica connessione
docker-compose exec mongodb mongosh \
  -u admin -p password --authenticationDatabase admin

# Test query
use datacenter_docs
db.tickets.find().limit(1)

Kubernetes

# Deploy MongoDB StatefulSet
kubectl apply -f deploy/kubernetes/mongodb.yaml

# Wait for pods
kubectl get pods -n datacenter-docs -w

# Initialize replica set
kubectl apply -f deploy/kubernetes/mongodb.yaml

# Verify
kubectl exec -n datacenter-docs mongodb-0 -- \
  mongosh -u admin -p password --authenticationDatabase admin \
  --eval "rs.status()"

🔧 Configurazione

Connection String

# Development (local)
MONGODB_URL=mongodb://admin:password@localhost:27017

# Docker Compose
MONGODB_URL=mongodb://admin:password@mongodb:27017

# Kubernetes (single node)
MONGODB_URL=mongodb://admin:password@mongodb.datacenter-docs.svc.cluster.local:27017

# Kubernetes (replica set)
MONGODB_URL=mongodb://admin:password@mongodb-0.mongodb.datacenter-docs.svc.cluster.local:27017,mongodb-1.mongodb.datacenter-docs.svc.cluster.local:27017,mongodb-2.mongodb.datacenter-docs.svc.cluster.local:27017/?replicaSet=rs0

Environment Variables

# MongoDB
MONGODB_URL=mongodb://admin:password@mongodb:27017
MONGODB_DATABASE=datacenter_docs

# MongoDB Root (for admin operations)
MONGO_ROOT_USER=admin
MONGO_ROOT_PASSWORD=secure_password

🔐 Security

Authentication

# Create application user
mongosh -u admin -p password --authenticationDatabase admin

use datacenter_docs

db.createUser({
  user: "docs_app",
  pwd: "app_password",
  roles: [
    { role: "readWrite", db: "datacenter_docs" }
  ]
})

# Use app user in connection string
MONGODB_URL=mongodb://docs_app:app_password@mongodb:27017/datacenter_docs

Encryption at Rest

# docker-compose.yml
mongodb:
  command: 
    - --enableEncryption
    - --encryptionKeyFile=/data/mongodb-keyfile
  volumes:
    - ./mongodb-keyfile:/data/mongodb-keyfile:ro

TLS/SSL

# Generate certificates
openssl req -newkey rsa:2048 -nodes -keyout mongodb.key \
  -x509 -days 365 -out mongodb.crt

# Configure MongoDB
mongodb:
  command:
    - --tlsMode=requireTLS
    - --tlsCertificateKeyFile=/etc/ssl/mongodb.pem

📊 Indexing Strategy

Automatic Indexes (via Beanie)

class Ticket(Document):
    ticket_id: Indexed(str, unique=True)  # Unique index
    status: str                            # Indexed in Settings
    
    class Settings:
        indexes = [
            "status",
            "category",
            [("status", 1), ("created_at", -1)],  # Compound
        ]

Custom Indexes

// Text search
db.tickets.createIndex({
  title: "text",
  description: "text",
  resolution: "text"
})

// Geospatial (future use)
db.locations.createIndex({ location: "2dsphere" })

// TTL index (auto-delete old docs)
db.chat_sessions.createIndex(
  { last_activity: 1 },
  { expireAfterSeconds: 2592000 }  // 30 days
)

🔍 Query Examples

Python (Beanie)

from datacenter_docs.api.models import Ticket

# Find by status
tickets = await Ticket.find(Ticket.status == "resolved").to_list()

# Complex query
from datetime import datetime, timedelta
recent = datetime.now() - timedelta(days=7)

tickets = await Ticket.find(
    Ticket.status == "resolved",
    Ticket.confidence_score > 0.8,
    Ticket.created_at > recent
).sort(-Ticket.created_at).to_list()

# Aggregation
pipeline = [
    {"$group": {
        "_id": "$category",
        "count": {"$sum": 1},
        "avg_confidence": {"$avg": "$confidence_score"}
    }},
    {"$sort": {"count": -1}}
]
result = await Ticket.aggregate(pipeline).to_list()

MongoDB Shell

// Find resolved tickets
db.tickets.find({ status: "resolved" })

// Complex aggregation
db.tickets.aggregate([
  { $match: { status: "resolved" } },
  { $group: {
      _id: "$category",
      total: { $sum: 1 },
      avg_confidence: { $avg: "$confidence_score" },
      avg_time: { $avg: "$processing_time" }
  }},
  { $sort: { total: -1 } }
])

// Text search
db.tickets.find({
  $text: { $search: "network connectivity" }
})

📈 Performance Optimization

Indexes

// Explain query
db.tickets.find({ status: "resolved" }).explain("executionStats")

// Check index usage
db.tickets.aggregate([
  { $indexStats: {} }
])

Connection Pooling

# config.py
MONGODB_URL = "mongodb://user:pass@host:27017/?maxPoolSize=50"

Read Preference

# For read-heavy workloads with replica set
from pymongo import ReadPreference

client = AsyncIOMotorClient(
    MONGODB_URL,
    readPreference=ReadPreference.SECONDARY_PREFERRED
)

🛠️ Maintenance

Backup

# Full backup
mongodump --uri="mongodb://admin:password@localhost:27017" \
  --authenticationDatabase=admin \
  --out=/backup/$(date +%Y%m%d)

# Restore
mongorestore --uri="mongodb://admin:password@localhost:27017" \
  --authenticationDatabase=admin \
  /backup/20250115

Monitoring

// Database stats
db.stats()

// Collection stats
db.tickets.stats()

// Current operations
db.currentOp()

// Server status
db.serverStatus()

Cleanup

// Remove old chat sessions
db.chat_sessions.deleteMany({
  last_activity: { $lt: new Date(Date.now() - 30*24*60*60*1000) }
})

// Compact collection
db.runCommand({ compact: "tickets" })

🔄 Replica Set (Production)

Setup

# Initialize replica set
rs.initiate({
  _id: "rs0",
  members: [
    { _id: 0, host: "mongodb-0:27017" },
    { _id: 1, host: "mongodb-1:27017" },
    { _id: 2, host: "mongodb-2:27017" }
  ]
})

# Check status
rs.status()

# Add member
rs.add("mongodb-3:27017")

Connection String

MONGODB_URL=mongodb://user:pass@mongodb-0:27017,mongodb-1:27017,mongodb-2:27017/?replicaSet=rs0&w=majority

📚 References

• MongoDB Manual
• Motor Documentation
• Beanie ODM
• MongoDB Best Practices

---

MongoDB Version: 7.0
Driver: Motor (Async)
ODM: Beanie
Python: 3.10+