# benjaminnhnn/building-a-safe-and-verifiable-ai-agent-framework-for-infrastructure-operations architecture

> Architecture diagram of the GitHub repository benjaminnhnn/building-a-safe-and-verifiable-ai-agent-framework-for-infrastructure-operations, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/benjaminnhnn/building-a-safe-and-verifiable-ai-agent-framework-for-infrastructure-operations
- Repository: https://github.com/benjaminnhnn/building-a-safe-and-verifiable-ai-agent-framework-for-infrastructure-operations

## Overview

The documented product is an AWS-hosted AIOps platform: monitoring signals become queued incidents, receive deterministic/runbook or RAG-assisted analysis, reach administrators through Telegram, and are rechecked. The tree also contains a safety-gated Moodle incident workflow and synthetic probe. README-described demo-web services are absent from this tree, and sampled excerpts do not expose every worker/API call, so uncertain wiring is omitted.

Main entry points: [`agent_src/core/main.py`](https://github.com/benjaminnhnn/building-a-safe-and-verifiable-ai-agent-framework-for-infrastructure-operations/blob/feature/sprint1-moodle-infrastructure/agent_src/core/main.py) (Agent API), [`agent_src/core/tasks.py`](https://github.com/benjaminnhnn/building-a-safe-and-verifiable-ai-agent-framework-for-infrastructure-operations/blob/feature/sprint1-moodle-infrastructure/agent_src/core/tasks.py) (Incident processing), [`agent_src/core/rag_engine.py`](https://github.com/benjaminnhnn/building-a-safe-and-verifiable-ai-agent-framework-for-infrastructure-operations/blob/feature/sprint1-moodle-infrastructure/agent_src/core/rag_engine.py) (RAG engine), [`platform-config/prometheus.yml`](https://github.com/benjaminnhnn/building-a-safe-and-verifiable-ai-agent-framework-for-infrastructure-operations/blob/feature/sprint1-moodle-infrastructure/platform-config/prometheus.yml) (Prometheus)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_monitoring["Monitoring and Alerting"]
  node_prometheus["Prometheus<br/>[prometheus.yml]"]
  node_blackbox["Blackbox probes<br/>[blackbox.yml]"]
  node_service_monitor["Service monitor<br/>[service_monitor.py]"]
  node_log_watcher["Log watcher<br/>[log_watcher.py]"]
  node_alertmanager["Alertmanager<br/>[alertmanager.yml]"]
end

subgraph group_agent["Incident Agent"]
  node_agent_api["Agent API<br/>[main.py]"]
  node_redis[("Redis queue and cooldowns")]
  node_celery["Celery worker<br/>[celery_app.py]"]
  node_tasks["Incident processing<br/>[tasks.py]"]
end

subgraph group_knowledge["Knowledge and Feedback"]
  node_rag["RAG engine<br/>[rag_engine.py]"]
  node_runbooks["Runbooks"]
  node_chromadb[("ChromaDB memory")]
end

subgraph group_moodle["Moodle Operations"]
  node_moodle_pipeline["Moodle incident workflow<br/>[moodle_pipeline.py]"]
  node_synthetic_probe["Synthetic transaction"]
  node_moodle_endpoint["Moodle probe endpoint"]
end

node_gemini["Gemini"]
node_telegram["Telegram"]
node_admin(("Administrator"))
node_moodle_target["Moodle staging"]

node_blackbox -->|"probe metrics"| node_prometheus
node_prometheus -->|"dispatch alerts"| node_alertmanager
node_service_monitor -->|"posts alerts"| node_agent_api
node_log_watcher -->|"posts alerts"| node_agent_api
node_alertmanager -->|"posts webhooks"| node_agent_api
node_agent_api -->|"deduplicates and enqueues"| node_redis
node_agent_api -->|"dispatches task"| node_celery
node_celery -->|"runs task"| node_tasks
node_tasks -->|"retrieves context"| node_rag
node_rag -->|"ingests runbooks"| node_runbooks
node_rag -->|"queries and stores"| node_chromadb
node_tasks -.->|"requests analysis"| node_gemini
node_tasks -->|"sends report"| node_telegram
node_telegram -->|"notifies"| node_admin
node_admin -->|"submits feedback"| node_agent_api
node_tasks -->|"verifies incident"| node_prometheus
node_moodle_pipeline -.->|"executes gated action"| node_moodle_target
node_moodle_pipeline -.->|"verifies runtime"| node_moodle_target
node_synthetic_probe -->|"runs transaction"| node_moodle_endpoint
node_moodle_endpoint -->|"tests staging state"| node_moodle_target

click node_prometheus "https://github.com/benjaminnhnn/building-a-safe-and-verifiable-ai-agent-framework-for-infrastructure-operations/blob/feature/sprint1-moodle-infrastructure/platform-config/prometheus.yml"
click node_blackbox "https://github.com/benjaminnhnn/building-a-safe-and-verifiable-ai-agent-framework-for-infrastructure-operations/blob/feature/sprint1-moodle-infrastructure/platform-config/blackbox.yml"
click node_service_monitor "https://github.com/benjaminnhnn/building-a-safe-and-verifiable-ai-agent-framework-for-infrastructure-operations/blob/feature/sprint1-moodle-infrastructure/agent_src/monitoring/service_monitor.py"
click node_log_watcher "https://github.com/benjaminnhnn/building-a-safe-and-verifiable-ai-agent-framework-for-infrastructure-operations/blob/feature/sprint1-moodle-infrastructure/agent_src/log_watcher.py"
click node_alertmanager "https://github.com/benjaminnhnn/building-a-safe-and-verifiable-ai-agent-framework-for-infrastructure-operations/blob/feature/sprint1-moodle-infrastructure/ansible/config/alertmanager.yml"
click node_agent_api "https://github.com/benjaminnhnn/building-a-safe-and-verifiable-ai-agent-framework-for-infrastructure-operations/blob/feature/sprint1-moodle-infrastructure/agent_src/core/main.py"
click node_redis "https://github.com/benjaminnhnn/building-a-safe-and-verifiable-ai-agent-framework-for-infrastructure-operations/blob/feature/sprint1-moodle-infrastructure/release/docker-compose.production.yml"
click node_celery "https://github.com/benjaminnhnn/building-a-safe-and-verifiable-ai-agent-framework-for-infrastructure-operations/blob/feature/sprint1-moodle-infrastructure/agent_src/core/celery_app.py"
click node_tasks "https://github.com/benjaminnhnn/building-a-safe-and-verifiable-ai-agent-framework-for-infrastructure-operations/blob/feature/sprint1-moodle-infrastructure/agent_src/core/tasks.py"
click node_rag "https://github.com/benjaminnhnn/building-a-safe-and-verifiable-ai-agent-framework-for-infrastructure-operations/blob/feature/sprint1-moodle-infrastructure/agent_src/core/rag_engine.py"
click node_runbooks "https://github.com/benjaminnhnn/building-a-safe-and-verifiable-ai-agent-framework-for-infrastructure-operations/tree/feature/sprint1-moodle-infrastructure/agent_src/config/knowledge_base"
click node_chromadb "https://github.com/benjaminnhnn/building-a-safe-and-verifiable-ai-agent-framework-for-infrastructure-operations/tree/feature/sprint1-moodle-infrastructure/agent_src/vector_db"
click node_moodle_pipeline "https://github.com/benjaminnhnn/building-a-safe-and-verifiable-ai-agent-framework-for-infrastructure-operations/blob/feature/sprint1-moodle-infrastructure/agent_src/core/moodle_pipeline.py"
click node_synthetic_probe "https://github.com/benjaminnhnn/building-a-safe-and-verifiable-ai-agent-framework-for-infrastructure-operations/blob/feature/sprint1-moodle-infrastructure/automation/moodle-synthetic-transaction.py"
click node_moodle_endpoint "https://github.com/benjaminnhnn/building-a-safe-and-verifiable-ai-agent-framework-for-infrastructure-operations/blob/feature/sprint1-moodle-infrastructure/moodle/synthetic-transaction.php"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_prometheus,node_blackbox,node_service_monitor,node_log_watcher,node_alertmanager toneBlue
class node_agent_api,node_redis,node_celery,node_tasks toneAmber
class node_rag,node_runbooks,node_chromadb toneMint
class node_moodle_pipeline,node_synthetic_probe,node_moodle_endpoint toneRose
class node_gemini,node_telegram,node_admin,node_moodle_target toneIndigo
```

## Components

### Monitoring and Alerting

- [Prometheus](https://github.com/benjaminnhnn/building-a-safe-and-verifiable-ai-agent-framework-for-infrastructure-operations/blob/feature/sprint1-moodle-infrastructure/platform-config/prometheus.yml): `platform-config/prometheus.yml`
- [Blackbox probes](https://github.com/benjaminnhnn/building-a-safe-and-verifiable-ai-agent-framework-for-infrastructure-operations/blob/feature/sprint1-moodle-infrastructure/platform-config/blackbox.yml): `platform-config/blackbox.yml`
- [Service monitor](https://github.com/benjaminnhnn/building-a-safe-and-verifiable-ai-agent-framework-for-infrastructure-operations/blob/feature/sprint1-moodle-infrastructure/agent_src/monitoring/service_monitor.py): `agent_src/monitoring/service_monitor.py`
- [Log watcher](https://github.com/benjaminnhnn/building-a-safe-and-verifiable-ai-agent-framework-for-infrastructure-operations/blob/feature/sprint1-moodle-infrastructure/agent_src/log_watcher.py): `agent_src/log_watcher.py`
- [Alertmanager](https://github.com/benjaminnhnn/building-a-safe-and-verifiable-ai-agent-framework-for-infrastructure-operations/blob/feature/sprint1-moodle-infrastructure/ansible/config/alertmanager.yml): `ansible/config/alertmanager.yml`

### Incident Agent

- [Agent API](https://github.com/benjaminnhnn/building-a-safe-and-verifiable-ai-agent-framework-for-infrastructure-operations/blob/feature/sprint1-moodle-infrastructure/agent_src/core/main.py): `agent_src/core/main.py`
- [Redis queue and cooldowns](https://github.com/benjaminnhnn/building-a-safe-and-verifiable-ai-agent-framework-for-infrastructure-operations/blob/feature/sprint1-moodle-infrastructure/release/docker-compose.production.yml): `release/docker-compose.production.yml`
- [Celery worker](https://github.com/benjaminnhnn/building-a-safe-and-verifiable-ai-agent-framework-for-infrastructure-operations/blob/feature/sprint1-moodle-infrastructure/agent_src/core/celery_app.py): `agent_src/core/celery_app.py`
- [Incident processing](https://github.com/benjaminnhnn/building-a-safe-and-verifiable-ai-agent-framework-for-infrastructure-operations/blob/feature/sprint1-moodle-infrastructure/agent_src/core/tasks.py): `agent_src/core/tasks.py`

### Knowledge and Feedback

- [RAG engine](https://github.com/benjaminnhnn/building-a-safe-and-verifiable-ai-agent-framework-for-infrastructure-operations/blob/feature/sprint1-moodle-infrastructure/agent_src/core/rag_engine.py): `agent_src/core/rag_engine.py`
- [Runbooks](https://github.com/benjaminnhnn/building-a-safe-and-verifiable-ai-agent-framework-for-infrastructure-operations/tree/feature/sprint1-moodle-infrastructure/agent_src/config/knowledge_base): `agent_src/config/knowledge_base`
- [ChromaDB memory](https://github.com/benjaminnhnn/building-a-safe-and-verifiable-ai-agent-framework-for-infrastructure-operations/tree/feature/sprint1-moodle-infrastructure/agent_src/vector_db): `agent_src/vector_db`

### Moodle Operations

- [Moodle incident workflow](https://github.com/benjaminnhnn/building-a-safe-and-verifiable-ai-agent-framework-for-infrastructure-operations/blob/feature/sprint1-moodle-infrastructure/agent_src/core/moodle_pipeline.py): `agent_src/core/moodle_pipeline.py`
- [Synthetic transaction](https://github.com/benjaminnhnn/building-a-safe-and-verifiable-ai-agent-framework-for-infrastructure-operations/blob/feature/sprint1-moodle-infrastructure/automation/moodle-synthetic-transaction.py): `automation/moodle-synthetic-transaction.py`
- [Moodle probe endpoint](https://github.com/benjaminnhnn/building-a-safe-and-verifiable-ai-agent-framework-for-infrastructure-operations/blob/feature/sprint1-moodle-infrastructure/moodle/synthetic-transaction.php): `moodle/synthetic-transaction.php`

### Other components

- **Gemini**
- **Telegram**
- **Administrator**
- **Moodle staging**

## Connections

- Blackbox probes → Prometheus: probe metrics
- Prometheus → Alertmanager: dispatch alerts
- Service monitor → Agent API: posts alerts
- Log watcher → Agent API: posts alerts
- Alertmanager → Agent API: posts webhooks
- Agent API → Redis queue and cooldowns: deduplicates and enqueues
- Agent API → Celery worker: dispatches task
- Celery worker → Incident processing: runs task
- Incident processing → RAG engine: retrieves context
- RAG engine → Runbooks: ingests runbooks
- RAG engine → ChromaDB memory: queries and stores
- Incident processing → Gemini: requests analysis
- Incident processing → Telegram: sends report
- Telegram → Administrator: notifies
- Administrator → Agent API: submits feedback
- Incident processing → Prometheus: verifies incident
- Moodle incident workflow → Moodle staging: executes gated action
- Moodle incident workflow → Moodle staging: verifies runtime
- Synthetic transaction → Moodle probe endpoint: runs transaction
- Moodle probe endpoint → Moodle staging: tests staging state