feat: add Proxmox VE API authentication and real data collection
Some checks failed
CI/CD Pipeline / Run Tests (push) Has been skipped
CI/CD Pipeline / Security Scanning (push) Has been skipped
CI/CD Pipeline / Lint Code (push) Failing after 7m32s
CI/CD Pipeline / Build and Push Docker Images (api) (push) Has been skipped
CI/CD Pipeline / Build and Push Docker Images (chat) (push) Has been skipped
CI/CD Pipeline / Build and Push Docker Images (frontend) (push) Has been skipped
CI/CD Pipeline / Generate Documentation (push) Failing after 7m37s
CI/CD Pipeline / Build and Push Docker Images (worker) (push) Has been skipped
CI/CD Pipeline / Deploy to Staging (push) Has been skipped
CI/CD Pipeline / Deploy to Production (push) Has been skipped
Some checks failed
CI/CD Pipeline / Run Tests (push) Has been skipped
CI/CD Pipeline / Security Scanning (push) Has been skipped
CI/CD Pipeline / Lint Code (push) Failing after 7m32s
CI/CD Pipeline / Build and Push Docker Images (api) (push) Has been skipped
CI/CD Pipeline / Build and Push Docker Images (chat) (push) Has been skipped
CI/CD Pipeline / Build and Push Docker Images (frontend) (push) Has been skipped
CI/CD Pipeline / Generate Documentation (push) Failing after 7m37s
CI/CD Pipeline / Build and Push Docker Images (worker) (push) Has been skipped
CI/CD Pipeline / Deploy to Staging (push) Has been skipped
CI/CD Pipeline / Deploy to Production (push) Has been skipped
Implement complete Proxmox API integration with support for both password and API token authentication, replacing mock data with real infrastructure data. **Authentication Features:** 1. **Dual Authentication Support** - API Token authentication (recommended, more secure) - Username + Password authentication (fallback) - Automatic fallback to mock data if not configured 2. **Configuration** (`src/datacenter_docs/utils/config.py`) - PROXMOX_HOST: Server hostname/IP - PROXMOX_PORT: API port (default 8006) - PROXMOX_USER: Username with realm (e.g., root@pam) - PROXMOX_PASSWORD: Password authentication - PROXMOX_TOKEN_NAME: API token name - PROXMOX_TOKEN_VALUE: API token secret - PROXMOX_VERIFY_SSL: SSL certificate verification - PROXMOX_TIMEOUT: API request timeout 3. **Real Data Collection** (`src/datacenter_docs/collectors/proxmox_collector.py`) - VMs: Iterate all nodes, collect QEMU VMs with full details - Containers: Collect LXC containers from all nodes - Nodes: Cluster node information and status - Cluster: Cluster configuration and quorum status - Storage: Storage pools with usage statistics - Networks: Network interfaces from all nodes - Automatic fallback to mock data on errors 4. **Comprehensive Documentation** (`docs/PROXMOX_SETUP.md`) - Step-by-step API token creation guide - Permission setup (PVEAuditor role) - Security best practices - Troubleshooting guide - Example configurations 5. **Environment Template** (`.env.example`) - Detailed Proxmox configuration section - Inline documentation for both auth methods - Security recommendations **Security Features:** - Supports read-only PVEAuditor role - API tokens preferred over passwords - SSL verification configurable - Graceful degradation (uses mock data if API unavailable) - No credentials in code (environment variables only) **How to Configure:** ```bash # Method 1: API Token (Recommended) PROXMOX_HOST=proxmox.company.com PROXMOX_USER=automation@pam PROXMOX_TOKEN_NAME=docs-collector PROXMOX_TOKEN_VALUE=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx # Method 2: Password PROXMOX_HOST=proxmox.company.com PROXMOX_USER=root@pam PROXMOX_PASSWORD=your-secure-password ``` See `docs/PROXMOX_SETUP.md` for complete setup instructions. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
252
docs/PROXMOX_SETUP.md
Normal file
252
docs/PROXMOX_SETUP.md
Normal file
@@ -0,0 +1,252 @@
|
||||
# Proxmox VE API Integration Setup
|
||||
|
||||
This guide explains how to configure the Proxmox collector to connect to your Proxmox VE cluster and collect infrastructure data.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Proxmox VE 7.0 or later
|
||||
- Network access to Proxmox API (default port 8006)
|
||||
- User account with appropriate permissions
|
||||
|
||||
## Authentication Methods
|
||||
|
||||
The Proxmox collector supports two authentication methods:
|
||||
|
||||
### Method 1: API Token (RECOMMENDED)
|
||||
|
||||
API tokens are more secure than passwords and can be easily revoked. They also support fine-grained permissions.
|
||||
|
||||
**Advantages:**
|
||||
- More secure (no password exposure)
|
||||
- Can be revoked independently
|
||||
- Fine-grained permissions
|
||||
- Best practice for automation
|
||||
|
||||
**Setup Steps:**
|
||||
|
||||
1. **Login to Proxmox Web UI**
|
||||
- Navigate to `https://your-proxmox-host:8006`
|
||||
- Login with your credentials
|
||||
|
||||
2. **Navigate to API Tokens**
|
||||
- Click on **Datacenter** in the left sidebar
|
||||
- Go to **Permissions** → **API Tokens**
|
||||
|
||||
3. **Create New Token**
|
||||
- Click **Add** button
|
||||
- Configure the token:
|
||||
- **User**: Select or create a user (e.g., `automation@pam`)
|
||||
- **Token ID**: Give it a descriptive name (e.g., `docs-collector`)
|
||||
- **Privilege Separation**: Leave UNCHECKED for read-only access
|
||||
- Click **Add**
|
||||
|
||||
4. **Copy Token Secret**
|
||||
- ⚠️ **IMPORTANT**: Copy the token secret immediately!
|
||||
- It will only be shown once and cannot be retrieved later
|
||||
- Format: `xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx`
|
||||
|
||||
5. **Set Permissions**
|
||||
- Go to **Datacenter** → **Permissions**
|
||||
- Click **Add** → **API Token Permission**
|
||||
- Configure:
|
||||
- **Path**: `/` (root, for full access)
|
||||
- **API Token**: Select your token (e.g., `automation@pam!docs-collector`)
|
||||
- **Role**: `PVEAuditor` (read-only access)
|
||||
- Click **Add**
|
||||
|
||||
6. **Configure Environment Variables**
|
||||
```bash
|
||||
# In your .env file:
|
||||
PROXMOX_HOST=proxmox.example.com
|
||||
PROXMOX_PORT=8006
|
||||
PROXMOX_USER=automation@pam
|
||||
PROXMOX_TOKEN_NAME=docs-collector
|
||||
PROXMOX_TOKEN_VALUE=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
|
||||
PROXMOX_VERIFY_SSL=false # Set to true if using valid SSL cert
|
||||
```
|
||||
|
||||
### Method 2: Username + Password
|
||||
|
||||
Use this method only if API tokens are not available.
|
||||
|
||||
**Setup Steps:**
|
||||
|
||||
1. **Create Dedicated User (Optional but Recommended)**
|
||||
- Go to **Datacenter** → **Permissions** → **Users**
|
||||
- Click **Add**
|
||||
- Configure:
|
||||
- **User name**: `automation`
|
||||
- **Realm**: `pam` or `pve`
|
||||
- **Password**: Set a strong password
|
||||
- Click **Add**
|
||||
|
||||
2. **Set Permissions**
|
||||
- Go to **Datacenter** → **Permissions**
|
||||
- Click **Add** → **User Permission**
|
||||
- Configure:
|
||||
- **Path**: `/`
|
||||
- **User**: Select your user (e.g., `automation@pam`)
|
||||
- **Role**: `PVEAuditor`
|
||||
- Click **Add**
|
||||
|
||||
3. **Configure Environment Variables**
|
||||
```bash
|
||||
# In your .env file:
|
||||
PROXMOX_HOST=proxmox.example.com
|
||||
PROXMOX_PORT=8006
|
||||
PROXMOX_USER=automation@pam
|
||||
PROXMOX_PASSWORD=your-secure-password
|
||||
PROXMOX_VERIFY_SSL=false # Set to true if using valid SSL cert
|
||||
```
|
||||
|
||||
## Roles and Permissions
|
||||
|
||||
The collector needs **read-only access** to gather infrastructure data.
|
||||
|
||||
### Recommended Role: PVEAuditor
|
||||
|
||||
The `PVEAuditor` role provides read-only access to:
|
||||
- Virtual Machines (QEMU)
|
||||
- Containers (LXC)
|
||||
- Nodes and Cluster Status
|
||||
- Storage Information
|
||||
- Network Configuration
|
||||
|
||||
This role is **perfect for documentation collection** as it cannot modify anything.
|
||||
|
||||
### Custom Role (Optional)
|
||||
|
||||
If you want even more restricted access, create a custom role:
|
||||
|
||||
1. Go to **Datacenter** → **Permissions** → **Roles**
|
||||
2. Click **Create**
|
||||
3. Name it `DocCollector`
|
||||
4. Select these privileges:
|
||||
- `VM.Audit`
|
||||
- `Datastore.Audit`
|
||||
- `Sys.Audit`
|
||||
5. Use this custom role when assigning permissions
|
||||
|
||||
## Testing the Connection
|
||||
|
||||
After configuring credentials, test the connection:
|
||||
|
||||
```bash
|
||||
# Using the test script
|
||||
python scripts/test_proxmox_docs.py
|
||||
```
|
||||
|
||||
You should see:
|
||||
```
|
||||
✓ Connected to Proxmox VE version X.Y
|
||||
Collected N VMs from M nodes
|
||||
```
|
||||
|
||||
If using mock data, you'll see:
|
||||
```
|
||||
Proxmox host not configured, using mock data for development
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Connection Refused
|
||||
|
||||
**Error**: `Connection failed: [Errno 111] Connection refused`
|
||||
|
||||
**Solutions**:
|
||||
- Check `PROXMOX_HOST` is correct
|
||||
- Ensure port 8006 is accessible
|
||||
- Check firewall rules
|
||||
- Verify Proxmox API is running: `systemctl status pveproxy`
|
||||
|
||||
### Authentication Failed
|
||||
|
||||
**Error**: `Connection failed: 401 Unauthorized`
|
||||
|
||||
**Solutions**:
|
||||
- Verify credentials are correct
|
||||
- Check user/token has required permissions
|
||||
- Ensure realm is correct (`@pam` or `@pve`)
|
||||
- For tokens: verify token ID matches exactly
|
||||
|
||||
### SSL Certificate Errors
|
||||
|
||||
**Error**: `SSL: CERTIFICATE_VERIFY_FAILED`
|
||||
|
||||
**Solutions**:
|
||||
1. **Quick Fix (Development)**: Set `PROXMOX_VERIFY_SSL=false`
|
||||
2. **Production Fix**:
|
||||
- Install valid SSL certificate on Proxmox
|
||||
- Or add Proxmox CA to trusted certificates
|
||||
- Set `PROXMOX_VERIFY_SSL=true`
|
||||
|
||||
### Permission Denied
|
||||
|
||||
**Error**: `403 Forbidden` or `Permission denied`
|
||||
|
||||
**Solutions**:
|
||||
- Check user/token has `PVEAuditor` role
|
||||
- Verify permission is set on path `/`
|
||||
- For tokens: ensure "Privilege Separation" is UNCHECKED
|
||||
|
||||
### Timeout Errors
|
||||
|
||||
**Error**: `Connection timeout after 30s`
|
||||
|
||||
**Solutions**:
|
||||
- Increase timeout: `PROXMOX_TIMEOUT=60`
|
||||
- Check network latency
|
||||
- Verify Proxmox server is not overloaded
|
||||
|
||||
## Security Best Practices
|
||||
|
||||
1. **Use API Tokens** instead of passwords
|
||||
2. **Use Read-Only Role** (`PVEAuditor`)
|
||||
3. **Dedicated User** for automation (not root)
|
||||
4. **Valid SSL Certificates** in production (`PROXMOX_VERIFY_SSL=true`)
|
||||
5. **Rotate Tokens** periodically
|
||||
6. **Audit Logs** regularly to monitor API access
|
||||
7. **Network Segmentation** - restrict access to Proxmox API
|
||||
8. **Secret Management** - use vault for credentials (not .env in production)
|
||||
|
||||
## Example Configuration
|
||||
|
||||
### Development Environment
|
||||
```bash
|
||||
# .env
|
||||
PROXMOX_HOST=192.168.1.100
|
||||
PROXMOX_PORT=8006
|
||||
PROXMOX_USER=automation@pam
|
||||
PROXMOX_TOKEN_NAME=docs-dev
|
||||
PROXMOX_TOKEN_VALUE=12345678-1234-1234-1234-123456789012
|
||||
PROXMOX_VERIFY_SSL=false
|
||||
PROXMOX_TIMEOUT=30
|
||||
```
|
||||
|
||||
### Production Environment
|
||||
```bash
|
||||
# .env (use secrets manager in real production!)
|
||||
PROXMOX_HOST=proxmox.company.com
|
||||
PROXMOX_PORT=8006
|
||||
PROXMOX_USER=automation@pam
|
||||
PROXMOX_TOKEN_NAME=docs-prod
|
||||
PROXMOX_TOKEN_VALUE=87654321-4321-4321-4321-210987654321
|
||||
PROXMOX_VERIFY_SSL=true
|
||||
PROXMOX_TIMEOUT=60
|
||||
```
|
||||
|
||||
## Additional Resources
|
||||
|
||||
- [Proxmox VE API Documentation](https://pve.proxmox.com/pve-docs/api-viewer/)
|
||||
- [Proxmox User Management](https://pve.proxmox.com/wiki/User_Management)
|
||||
- [Proxmox API Tokens](https://pve.proxmox.com/wiki/Proxmox_VE_API#API_Tokens)
|
||||
- [proxmoxer Python Library](https://github.com/proxmoxer/proxmoxer)
|
||||
|
||||
## Support
|
||||
|
||||
If you encounter issues:
|
||||
1. Check this documentation
|
||||
2. Review logs: `docker-compose logs chat worker`
|
||||
3. Test with mock data first
|
||||
4. Verify Proxmox API access independently
|
||||
5. Open an issue on GitHub with logs
|
||||
Reference in New Issue
Block a user