# bruceowenga/ulinzi-sre-agent architecture

> Architecture diagram of the GitHub repository bruceowenga/ulinzi-sre-agent, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bruceowenga/ulinzi-sre-agent
- Repository: https://github.com/bruceowenga/ulinzi-sre-agent

## Overview

Ulinzi monitors infrastructure metrics, turns detected alerts into shared incident state, classifies incidents with an LLM, and routes remediation by severity. Low/medium incidents use registered runbooks; high/critical incidents get an LLM-generated playbook and pause for operator approval. Both routes report to notification and observability sinks. The graph distinguishes monitoring, incident workflow, model/runbook execution, and reporting. Source excerpts show the main runtime calls; some monitoring and integration capabilities are documented in the README but their implementations were not sampled.

Main entry points: [`agents/remediation.py`](https://github.com/bruceowenga/ulinzi-sre-agent/blob/main/agents/remediation.py) (Automatic remediation), [`main.py`](https://github.com/bruceowenga/ulinzi-sre-agent/blob/main/main.py) (Runtime entry), [`graph.py`](https://github.com/bruceowenga/ulinzi-sre-agent/blob/main/graph.py) (Workflow graph), [`agents/triage.py`](https://github.com/bruceowenga/ulinzi-sre-agent/blob/main/agents/triage.py) (Triage agent)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_monitoring["Monitoring"]
  node_monitor["Monitor agent<br/>[monitor.py]"]
end

subgraph group_workflow["Incident workflow"]
  node_incident["Incident state<br/>[state.py]"]
  node_runtime["Runtime entry<br/>[main.py]"]
  node_checkpoint[("SQLite checkpoint<br/>[main.py]")]
  node_graph["Workflow graph<br/>[graph.py]"]
  node_triage["Triage agent<br/>[triage.py]"]
  node_settings["Configuration<br/>[config.py]"]
  node_prompts["Prompt templates"]
end

subgraph group_response["Response execution"]
  node_llm_client["LLM client<br/>[llm.py]"]
  node_remediation["Automatic remediation<br/>[remediation.py]"]
  node_playbook["Manual playbook<br/>[remediation.py]"]
  node_registry["Runbook registry<br/>[__init__.py]"]
  node_oom["Container recovery<br/>[container_oom.py]"]
  node_disk["Disk cleanup<br/>[disk_pressure.py]"]
  node_logs["Log rotation<br/>[log_flood.py]"]
end

subgraph group_reporting["Reporting"]
  node_reporter["Reporter agent<br/>[reporter.py]"]
end

node_prometheus[("Prometheus")]
node_model_endpoint{{"Model endpoint"}}
node_approval(("Operator approval"))
node_ntfy{{"ntfy"}}
node_loki{{"Loki"}}
node_grafana{{"Grafana"}}

node_runtime -->|"starts"| node_monitor
node_monitor -->|"queries"| node_prometheus
node_monitor -->|"emits incident"| node_runtime
node_monitor -->|"creates"| node_incident
node_runtime -->|"invokes"| node_graph
node_runtime -->|"persists graph state"| node_checkpoint
node_graph -->|"dispatches"| node_triage
node_graph -->|"routes auto"| node_remediation
node_graph -->|"routes manual"| node_playbook
node_graph -->|"dispatches"| node_reporter
node_triage -->|"requests classification"| node_llm_client
node_remediation -->|"requests playbook"| node_llm_client
node_llm_client -->|"calls configured endpoint"| node_model_endpoint
node_triage -->|"reads triage prompt"| node_prompts
node_remediation -->|"reads playbook prompt"| node_prompts
node_triage -->|"reads model settings"| node_settings
node_remediation -->|"reads model settings"| node_settings
node_remediation -->|"looks up runbook"| node_registry
node_registry -->|"registers"| node_oom
node_registry -->|"registers"| node_disk
node_registry -->|"registers"| node_logs
node_playbook -->|"requests decision"| node_approval
node_runtime -->|"collects decision"| node_approval
node_reporter -->|"pushes notification"| node_ntfy
node_reporter -->|"writes incident log"| node_loki
node_reporter -->|"posts annotation"| node_grafana
node_triage -->|"updates classification"| node_incident
node_remediation -->|"updates action"| node_incident
node_reporter -->|"updates report"| node_incident

click node_monitor "https://github.com/bruceowenga/ulinzi-sre-agent/blob/main/agents/monitor.py"
click node_incident "https://github.com/bruceowenga/ulinzi-sre-agent/blob/main/state.py"
click node_runtime "https://github.com/bruceowenga/ulinzi-sre-agent/blob/main/main.py"
click node_checkpoint "https://github.com/bruceowenga/ulinzi-sre-agent/blob/main/main.py"
click node_graph "https://github.com/bruceowenga/ulinzi-sre-agent/blob/main/graph.py"
click node_triage "https://github.com/bruceowenga/ulinzi-sre-agent/blob/main/agents/triage.py"
click node_settings "https://github.com/bruceowenga/ulinzi-sre-agent/blob/main/config.py"
click node_prompts "https://github.com/bruceowenga/ulinzi-sre-agent/tree/main/prompts"
click node_llm_client "https://github.com/bruceowenga/ulinzi-sre-agent/blob/main/agents/llm.py"
click node_remediation "https://github.com/bruceowenga/ulinzi-sre-agent/blob/main/agents/remediation.py"
click node_playbook "https://github.com/bruceowenga/ulinzi-sre-agent/blob/main/agents/remediation.py"
click node_registry "https://github.com/bruceowenga/ulinzi-sre-agent/blob/main/runbooks/__init__.py"
click node_oom "https://github.com/bruceowenga/ulinzi-sre-agent/blob/main/runbooks/container_oom.py"
click node_disk "https://github.com/bruceowenga/ulinzi-sre-agent/blob/main/runbooks/disk_pressure.py"
click node_logs "https://github.com/bruceowenga/ulinzi-sre-agent/blob/main/runbooks/log_flood.py"
click node_reporter "https://github.com/bruceowenga/ulinzi-sre-agent/blob/main/agents/reporter.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_monitor toneBlue
class node_incident,node_runtime,node_checkpoint,node_graph,node_triage,node_settings,node_prompts,node_prometheus toneAmber
class node_llm_client,node_remediation,node_playbook,node_registry,node_oom,node_disk,node_logs toneMint
class node_reporter toneRose
class node_model_endpoint,node_approval,node_ntfy,node_loki,node_grafana toneIndigo
```

## Components

### Monitoring

- [Monitor agent](https://github.com/bruceowenga/ulinzi-sre-agent/blob/main/agents/monitor.py): `agents/monitor.py`

### Incident workflow

- [Incident state](https://github.com/bruceowenga/ulinzi-sre-agent/blob/main/state.py): `state.py`
- [Runtime entry](https://github.com/bruceowenga/ulinzi-sre-agent/blob/main/main.py): `main.py`
- [SQLite checkpoint](https://github.com/bruceowenga/ulinzi-sre-agent/blob/main/main.py): `main.py`
- [Workflow graph](https://github.com/bruceowenga/ulinzi-sre-agent/blob/main/graph.py): `graph.py`
- [Triage agent](https://github.com/bruceowenga/ulinzi-sre-agent/blob/main/agents/triage.py): `agents/triage.py`
- [Configuration](https://github.com/bruceowenga/ulinzi-sre-agent/blob/main/config.py): `config.py`
- [Prompt templates](https://github.com/bruceowenga/ulinzi-sre-agent/tree/main/prompts): `prompts`

### Response execution

- [LLM client](https://github.com/bruceowenga/ulinzi-sre-agent/blob/main/agents/llm.py): `agents/llm.py`
- [Automatic remediation](https://github.com/bruceowenga/ulinzi-sre-agent/blob/main/agents/remediation.py): `agents/remediation.py`
- [Manual playbook](https://github.com/bruceowenga/ulinzi-sre-agent/blob/main/agents/remediation.py): `agents/remediation.py`
- [Runbook registry](https://github.com/bruceowenga/ulinzi-sre-agent/blob/main/runbooks/__init__.py): `runbooks/__init__.py`
- [Container recovery](https://github.com/bruceowenga/ulinzi-sre-agent/blob/main/runbooks/container_oom.py): `runbooks/container_oom.py`
- [Disk cleanup](https://github.com/bruceowenga/ulinzi-sre-agent/blob/main/runbooks/disk_pressure.py): `runbooks/disk_pressure.py`
- [Log rotation](https://github.com/bruceowenga/ulinzi-sre-agent/blob/main/runbooks/log_flood.py): `runbooks/log_flood.py`

### Reporting

- [Reporter agent](https://github.com/bruceowenga/ulinzi-sre-agent/blob/main/agents/reporter.py): `agents/reporter.py`

### Other components

- **Prometheus**
- **Model endpoint**
- **Operator approval**
- **ntfy**
- **Loki**
- **Grafana**

## Connections

- Runtime entry → Monitor agent: starts (evidence: `main.py`)
- Monitor agent → Prometheus: queries (evidence: `agents/monitor.py`)
- Monitor agent → Runtime entry: emits incident (evidence: `main.py`)
- Monitor agent → Incident state: creates (evidence: `agents/monitor.py`)
- Runtime entry → Workflow graph: invokes (evidence: `main.py`)
- Runtime entry → SQLite checkpoint: persists graph state (evidence: `main.py`)
- Workflow graph → Triage agent: dispatches (evidence: `graph.py`)
- Workflow graph → Automatic remediation: routes auto (evidence: `graph.py`)
- Workflow graph → Manual playbook: routes manual (evidence: `graph.py`)
- Workflow graph → Reporter agent: dispatches (evidence: `graph.py`)
- Triage agent → LLM client: requests classification (evidence: `agents/triage.py`)
- Automatic remediation → LLM client: requests playbook (evidence: `agents/remediation.py`)
- LLM client → Model endpoint: calls configured endpoint (evidence: `agents/llm.py`)
- Triage agent → Prompt templates: reads triage prompt (evidence: `agents/triage.py`)
- Automatic remediation → Prompt templates: reads playbook prompt (evidence: `agents/remediation.py`)
- Triage agent → Configuration: reads model settings (evidence: `agents/triage.py`)
- Automatic remediation → Configuration: reads model settings (evidence: `agents/remediation.py`)
- Automatic remediation → Runbook registry: looks up runbook (evidence: `agents/remediation.py`)
- Runbook registry → Container recovery: registers (evidence: `runbooks/__init__.py`)
- Runbook registry → Disk cleanup: registers (evidence: `runbooks/__init__.py`)
- Runbook registry → Log rotation: registers (evidence: `runbooks/__init__.py`)
- Manual playbook → Operator approval: requests decision (evidence: `agents/remediation.py`)
- Runtime entry → Operator approval: collects decision (evidence: `main.py`)
- Reporter agent → ntfy: pushes notification (evidence: `agents/reporter.py`)
- Reporter agent → Loki: writes incident log (evidence: `agents/reporter.py`)
- Reporter agent → Grafana: posts annotation (evidence: `agents/reporter.py`)
- Triage agent → Incident state: updates classification (evidence: `agents/triage.py`)
- Automatic remediation → Incident state: updates action (evidence: `agents/remediation.py`)
- Reporter agent → Incident state: updates report (evidence: `agents/reporter.py`)