How benjaminnhnn/building-a-safe-and-verifiable-ai-agent-framework-for-infrastructure-operations works
The documented product is an AWS-hosted AIOps platform: monitoring signals become queued incidents, receive deterministic/runbook or RAG-assisted analysis, reach administrators through Telegram, and are rechecked. The tree also contains a safety-gated Moodle incident workflow and synthetic probe. README-described demo-web services are absent from this tree, and sampled excerpts do not expose every worker/API call, so uncertain wiring is omitted.
Main entry points: main.py Agent API · tasks.py Incident processing · rag_engine.py RAG engine · prometheus.yml Prometheus
Components 19
Monitoring and Alerting
- Prometheus
platform-config/prometheus.yml - Blackbox probes
platform-config/blackbox.yml - Service monitor
agent_src/monitoring/service_monitor.py - Log watcher
agent_src/log_watcher.py - Alertmanager
ansible/config/alertmanager.yml
Incident Agent
- Agent API
agent_src/core/main.py - Redis queue and cooldowns
release/docker-compose.production.yml - Celery worker
agent_src/core/celery_app.py - Incident processing
agent_src/core/tasks.py
Knowledge and Feedback
- RAG engine
agent_src/core/rag_engine.py - Runbooks
agent_src/config/knowledge_base - ChromaDB memory
agent_src/vector_db
Moodle Operations
- Moodle incident workflow
agent_src/core/moodle_pipeline.py - Synthetic transaction
automation/moodle-synthetic-transaction.py - Moodle probe endpoint
moodle/synthetic-transaction.php
Other components
- Gemini
- Telegram
- Administrator
- Moodle staging
Connections 20
- Blackbox probes → Prometheus: probe metrics
- Prometheus → Alertmanager: dispatch alerts
- Service monitor → Agent API: posts alerts
- Log watcher → Agent API: posts alerts
- Alertmanager → Agent API: posts webhooks
- Agent API → Redis queue and cooldowns: deduplicates and enqueues
- Agent API → Celery worker: dispatches task
- Celery worker → Incident processing: runs task
- Incident processing → RAG engine: retrieves context
- RAG engine → Runbooks: ingests runbooks
- RAG engine → ChromaDB memory: queries and stores
- Incident processing → Gemini: requests analysis
- Incident processing → Telegram: sends report
- Telegram → Administrator: notifies
- Administrator → Agent API: submits feedback
- Incident processing → Prometheus: verifies incident
- Moodle incident workflow → Moodle staging: executes gated action
- Moodle incident workflow → Moodle staging: verifies runtime
- Synthetic transaction → Moodle probe endpoint: runs transaction
- Moodle probe endpoint → Moodle staging: tests staging state