llm-automation-docs-and-remediation-engine/scripts/README.md

# Scripts Directory

This directory contains utility scripts for the Datacenter Documentation project.

---

## 🔍 test-ci-pipeline.sh

**Local CI/CD Pipeline Validation Script**

### Description

Simulates the complete GitLab CI/CD pipeline locally before pushing code to the repository. This script runs all the same checks that would run in GitHub Actions, GitLab CI, or Gitea Actions.

### Usage

```bash
# Run from project root
bash scripts/test-ci-pipeline.sh

# Or make it executable and run directly
chmod +x scripts/test-ci-pipeline.sh
./scripts/test-ci-pipeline.sh
```

### Pipeline Stages

The script executes the following stages in order:

#### 1. **LINT** Stage
- **Black**: Code formatting check
- **Ruff**: Linting and code quality
- **MyPy**: Type checking (strict mode)

#### 2. **TEST** Stage
- **Unit Tests**: Runs pytest with coverage
- **Security Scan**: Bandit (if installed)

#### 3. **BUILD** Stage
- **Poetry Check**: Validates `pyproject.toml` configuration
- **Dependency Resolution**: Tests if all dependencies can be installed
- **Docker Validation**: Checks Dockerfile syntax

#### 4. **INTEGRATION** Stage (Optional)
- **API Health Check**: Tests if local API is running

### Output

The script provides:
- ✅ **Color-coded output** for easy readability
- 📊 **Real-time progress** for each job
- 📄 **Summary report** at the end
- 📝 **Written report** saved to `ci-pipeline-report-TIMESTAMP.txt`

### Example Output

```
╔═══════════════════════════════════════════════════════╗
║       LOCAL CI/CD PIPELINE SIMULATION                ║
║       GitLab CI Pipeline                             ║
╚═══════════════════════════════════════════════════════╝

=====================================
STAGE: LINT
=====================================

>>> JOB: lint:black
Running: poetry run black --check src/ tests/
✅ PASSED: Black code formatting

>>> JOB: lint:ruff
Running: poetry run ruff check src/ tests/
✅ PASSED: Ruff linting

>>> JOB: lint:mypy
Running: poetry run mypy src/
✅ PASSED: MyPy type checking

...

╔═══════════════════════════════════════════════════════╗
║          ✅ PIPELINE PASSED SUCCESSFULLY ✅           ║
╚═══════════════════════════════════════════════════════╝

Total Tests:  8
Passed:       8
Failed:       0
Duration:     6s
```

### Exit Codes

- **0**: All checks passed ✅
- **1**: One or more checks failed ❌

### Requirements

- **Poetry**: For dependency management
- **Python 3.12+**: As specified in `pyproject.toml`
- **Docker/Podman** (optional): For Docker validation stage
- **MongoDB** (optional): For integration tests

### When to Run

Run this script:
- ✅ **Before every commit** to ensure code quality
- ✅ **Before creating a pull request**
- ✅ **After making significant changes**
- ✅ **To verify CI/CD pipeline compatibility**

### Integration with Git

You can add this as a Git pre-push hook:

```bash
#!/bin/bash
# .git/hooks/pre-push

echo "Running CI pipeline validation..."
bash scripts/test-ci-pipeline.sh

if [ $? -ne 0 ]; then
    echo "❌ CI pipeline validation failed. Push aborted."
    exit 1
fi

echo "✅ CI pipeline validation passed. Proceeding with push..."
exit 0
```

### Continuous Integration Compatibility

This script simulates:
- ✅ **GitHub Actions** (`.github/workflows/build-deploy.yml`)
- ✅ **GitLab CI** (`.gitlab-ci.yml`)
- ✅ **Gitea Actions** (`.gitea/workflows/ci.yml`)

All checks performed locally will also pass in the actual CI/CD platforms.

---

## 📝 Report Files

After running the validation script, you'll find:

- **`ci-pipeline-report-TIMESTAMP.txt`**: Plain text summary
- **`CI_VALIDATION_REPORT.md`**: Comprehensive markdown report with details

---

## 🚀 Quick Start

```bash
# First time setup
poetry install

# Run validation
bash scripts/test-ci-pipeline.sh

# If all passes, commit and push
git add .
git commit -m "your commit message"
git push
```

---

## 🔧 Troubleshooting

### "poetry: command not found"
Install Poetry: https://python-poetry.org/docs/#installation

### "Black would reformat X files"
Run: `poetry run black src/ tests/`

### "Ruff found X errors"
Run: `poetry run ruff check --fix src/ tests/`

### "MyPy found X errors"
Fix type errors or add type ignores where appropriate.

### Docker validation fails
Ensure Docker or Podman is installed:
- **Ubuntu/Debian**: `sudo apt install docker.io`
- **Fedora**: `sudo dnf install podman podman-compose`

---

## 📚 Additional Resources

- [CLAUDE.md](../CLAUDE.md) - Project documentation for AI assistants
- [README.md](../README.md) - Project overview
- [TODO.md](../TODO.md) - Development roadmap
- [CI_VALIDATION_REPORT.md](../CI_VALIDATION_REPORT.md) - Latest validation report