# faresalawneh/llm-graduation-project architecture

> Architecture diagram of the GitHub repository faresalawneh/llm-graduation-project, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/faresalawneh/llm-graduation-project
- Repository: https://github.com/faresalawneh/llm-graduation-project

## Overview

INFERSCOPE replays steady or burst BurstGPT traces against vLLM, streams inference metrics through OpenTelemetry, and combines them with GPU telemetry in Prometheus for Grafana dashboards and a JWT-protected REST API. Completed runs can also be recorded in MLflow with metrics, artifacts, and container state. The main flow is documented and supported by sampled replay, exporter, API, and logging code; stack wiring and dashboard details are based on the README and repository paths.

Main entry points: [`monitoring/scripts/aiperf_trace_replay.py`](https://github.com/faresalawneh/llm-graduation-project/blob/main/monitoring/scripts/aiperf_trace_replay.py) (AIPerf replay), [`api/app.py`](https://github.com/faresalawneh/llm-graduation-project/blob/main/api/app.py) (Observability API), [`monitoring/scripts/mlflow_log_run.py`](https://github.com/faresalawneh/llm-graduation-project/blob/main/monitoring/scripts/mlflow_log_run.py) (MLflow logger), [`monitoring/otel/otel-collector-config.yml`](https://github.com/faresalawneh/llm-graduation-project/blob/main/monitoring/otel/otel-collector-config.yml) (OTel collector)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_replay["Trace replay"]
  node_steady_trace["Steady trace<br/>[steady_window.csv]"]
  node_burst_trace["Burst trace<br/>[burst_window.csv]"]
  node_aiperf["AIPerf replay"]
  node_vllm["vLLM server"]
end

subgraph group_telemetry["Telemetry pipeline"]
  node_otel["OTel collector"]
  node_nvml["NVML exporter<br/>[nvml_exporter.py]"]
  node_prom_config["Scrape config<br/>[prometheus.yml]"]
  node_prometheus[("Prometheus")]
end

subgraph group_access["Metrics access"]
  node_grafana["Grafana dashboards"]
  node_api["Observability API<br/>[app.py]"]
  node_auth["JWT and roles<br/>[auth.py]"]
end

subgraph group_tracking["Run tracking"]
  node_run_artifacts["Run artifacts"]
  node_mlflow_logger["MLflow logger<br/>[mlflow_log_run.py]"]
  node_mlflow_store[("MLflow store")]
end

node_operator(("Operator"))
node_api_user(("API user"))

node_steady_trace -->|"replay input"| node_aiperf
node_burst_trace -->|"replay input"| node_aiperf
node_aiperf -->|"HTTP requests"| node_vllm
node_aiperf -->|"streams OTLP"| node_otel
node_aiperf -->|"writes results"| node_run_artifacts
node_vllm -->|"scraped metrics"| node_prometheus
node_otel -->|"scraped metrics"| node_prometheus
node_nvml -->|"scraped gauges"| node_prometheus
node_prom_config -->|"defines scrapes"| node_prometheus
node_operator -->|"starts replay"| node_aiperf
node_prometheus -->|"serves queries"| node_grafana
node_operator -->|"views dashboards"| node_grafana
node_api_user -->|"requests metrics"| node_api
node_api -->|"checks access"| node_auth
node_api -->|"queries metrics"| node_prometheus
node_operator -->|"logs completed run"| node_mlflow_logger
node_mlflow_logger -->|"reads artifacts"| node_run_artifacts
node_mlflow_logger -->|"logs run"| node_mlflow_store
node_aiperf -.->|"optional GPU queries"| node_prometheus

click node_steady_trace "https://github.com/faresalawneh/llm-graduation-project/blob/main/monitoring/scripts/steady_window.csv"
click node_burst_trace "https://github.com/faresalawneh/llm-graduation-project/blob/main/monitoring/scripts/burst_window.csv"
click node_aiperf "https://github.com/faresalawneh/llm-graduation-project/blob/main/monitoring/scripts/aiperf_trace_replay.py"
click node_run_artifacts "https://github.com/faresalawneh/llm-graduation-project/blob/main/monitoring/scripts/aiperf_trace_replay.py"
click node_otel "https://github.com/faresalawneh/llm-graduation-project/blob/main/monitoring/otel/otel-collector-config.yml"
click node_nvml "https://github.com/faresalawneh/llm-graduation-project/blob/main/monitoring/exporters/nvml_exporter.py"
click node_prom_config "https://github.com/faresalawneh/llm-graduation-project/blob/main/monitoring/prometheus/prometheus.yml"
click node_grafana "https://github.com/faresalawneh/llm-graduation-project/tree/main/monitoring/grafana/provisioning/dashboards"
click node_api "https://github.com/faresalawneh/llm-graduation-project/blob/main/api/app.py"
click node_auth "https://github.com/faresalawneh/llm-graduation-project/blob/main/api/auth.py"
click node_mlflow_logger "https://github.com/faresalawneh/llm-graduation-project/blob/main/monitoring/scripts/mlflow_log_run.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_steady_trace,node_burst_trace,node_aiperf,node_vllm,node_api_user toneBlue
class node_otel,node_nvml,node_prom_config,node_prometheus toneAmber
class node_grafana,node_api,node_auth toneMint
class node_run_artifacts,node_mlflow_logger,node_mlflow_store toneRose
class node_operator toneIndigo
```

## Components

### Trace replay

- [Steady trace](https://github.com/faresalawneh/llm-graduation-project/blob/main/monitoring/scripts/steady_window.csv): `monitoring/scripts/steady_window.csv`
- [Burst trace](https://github.com/faresalawneh/llm-graduation-project/blob/main/monitoring/scripts/burst_window.csv): `monitoring/scripts/burst_window.csv`
- [AIPerf replay](https://github.com/faresalawneh/llm-graduation-project/blob/main/monitoring/scripts/aiperf_trace_replay.py): `monitoring/scripts/aiperf_trace_replay.py`
- **vLLM server**

### Telemetry pipeline

- [OTel collector](https://github.com/faresalawneh/llm-graduation-project/blob/main/monitoring/otel/otel-collector-config.yml): `monitoring/otel/otel-collector-config.yml`
- [NVML exporter](https://github.com/faresalawneh/llm-graduation-project/blob/main/monitoring/exporters/nvml_exporter.py): `monitoring/exporters/nvml_exporter.py`
- [Scrape config](https://github.com/faresalawneh/llm-graduation-project/blob/main/monitoring/prometheus/prometheus.yml): `monitoring/prometheus/prometheus.yml`
- **Prometheus**

### Metrics access

- [Grafana dashboards](https://github.com/faresalawneh/llm-graduation-project/tree/main/monitoring/grafana/provisioning/dashboards): `monitoring/grafana/provisioning/dashboards`
- [Observability API](https://github.com/faresalawneh/llm-graduation-project/blob/main/api/app.py): `api/app.py`
- [JWT and roles](https://github.com/faresalawneh/llm-graduation-project/blob/main/api/auth.py): `api/auth.py`

### Run tracking

- [Run artifacts](https://github.com/faresalawneh/llm-graduation-project/blob/main/monitoring/scripts/aiperf_trace_replay.py): `monitoring/scripts/aiperf_trace_replay.py`
- [MLflow logger](https://github.com/faresalawneh/llm-graduation-project/blob/main/monitoring/scripts/mlflow_log_run.py): `monitoring/scripts/mlflow_log_run.py`
- **MLflow store**

### Other components

- **Operator**
- **API user**

## Connections

- Steady trace → AIPerf replay: replay input
- Burst trace → AIPerf replay: replay input
- AIPerf replay → vLLM server: HTTP requests
- AIPerf replay → OTel collector: streams OTLP
- AIPerf replay → Run artifacts: writes results
- vLLM server → Prometheus: scraped metrics
- OTel collector → Prometheus: scraped metrics
- NVML exporter → Prometheus: scraped gauges
- Scrape config → Prometheus: defines scrapes
- Operator → AIPerf replay: starts replay
- Prometheus → Grafana dashboards: serves queries
- Operator → Grafana dashboards: views dashboards
- API user → Observability API: requests metrics
- Observability API → JWT and roles: checks access
- Observability API → Prometheus: queries metrics
- Operator → MLflow logger: logs completed run
- MLflow logger → Run artifacts: reads artifacts
- MLflow logger → MLflow store: logs run
- AIPerf replay → Prometheus: optional GPU queries