# ds-lab/metrion architecture

> Architecture diagram of the GitHub repository ds-lab/metrion, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ds-lab/metrion
- Repository: https://github.com/ds-lab/metrion

## Overview

METRION monitors Linux hardware and process activity, records measurements in a session database, then reconstructs per-process CPU, DRAM, and GPU energy through its attribution CLI. The graph preserves discovery, monitoring sources, the entity/measurement model, persistence, and analysis. Some subsystem links rely on the README or SOURCE INDEX references; internals of unsampled collector and monitoring modules are not inferred.

Main entry points: [`metrion/data_record.py`](https://github.com/ds-lab/metrion/blob/main/metrion/data_record.py) (Measurements and records), [`metrion/orchestrator.py`](https://github.com/ds-lab/metrion/blob/main/metrion/orchestrator.py) (Monitor orchestrator), [`metrion/monitoring/data_source/data_source_factory.py`](https://github.com/ds-lab/metrion/blob/main/metrion/monitoring/data_source/data_source_factory.py) (Data source factory), [`metrion/monitoring/data_source/cpu`](https://github.com/ds-lab/metrion/tree/main/metrion/monitoring/data_source/cpu) (CPU counters and events)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_control["Runtime control"]
  node_orchestrator["Monitor orchestrator<br/>[orchestrator.py]"]
  node_config["Configuration reader<br/>[config_reader.py]"]
end

subgraph group_discovery["Host discovery"]
  node_hardware["Hardware detection"]
  node_scanner["Linux system scanner"]
  node_tree["System tree builder<br/>[tree_builder.py]"]
end

subgraph group_monitoring["Monitoring"]
  node_monitor["Entity monitor<br/>[entity_monitor.py]"]
  node_collector["Data collector<br/>[data_collector.py]"]
  node_source_factory["Data source factory"]
  node_cpu_source["CPU counters and events"]
  node_gpu_source["NVIDIA NVML source<br/>[nvml.py]"]
  node_memory_source["NUMA memory source<br/>[proc_numa.py]"]
  node_network_source["TCP network source<br/>[tcp_monitor.py]"]
end

subgraph group_model["Data model"]
  node_registry["Process entity registry"]
  node_entities["Hardware entity model"]
  node_metrics["Metric definitions<br/>[metric.py]"]
  node_measurements["Measurements and records<br/>[data_record.py]"]
end

subgraph group_results["Storage and analysis"]
  node_storage[("SQLite database storage<br/>[db_data_storage.py]")]
  node_db[("Session database")]
  node_attribution["Energy attribution"]
  node_analysis_cli["Analysis CLI"]
end

node_operator(("Operator"))

node_operator -->|"starts monitoring"| node_orchestrator
node_operator -->|"runs analysis"| node_analysis_cli
node_orchestrator -->|"loads settings"| node_config
node_orchestrator -->|"detects hardware"| node_hardware
node_hardware -->|"builds topology"| node_tree
node_tree -->|"uses system info"| node_scanner
node_orchestrator -->|"starts monitoring"| node_monitor
node_orchestrator -->|"uses collector"| node_collector
node_monitor -.->|"selects sources"| node_source_factory
node_source_factory -.->|"dispatches to"| node_cpu_source
node_source_factory -.->|"dispatches to"| node_gpu_source
node_source_factory -.->|"dispatches to"| node_memory_source
node_source_factory -.->|"dispatches to"| node_network_source
node_cpu_source -->|"updates processes"| node_registry
node_gpu_source -->|"resolves processes"| node_registry
node_network_source -->|"resolves processes"| node_registry
node_cpu_source -->|"emits records"| node_measurements
node_gpu_source -->|"emits records"| node_measurements
node_network_source -->|"emits records"| node_measurements
node_measurements -->|"references entities"| node_entities
node_measurements -->|"references metrics"| node_metrics
node_collector -.->|"persists records"| node_storage
node_storage -->|"writes session data"| node_db
node_analysis_cli -->|"reads session data"| node_db
node_analysis_cli -->|"runs attribution"| node_attribution
node_attribution -->|"uses measurement model"| node_measurements

click node_orchestrator "https://github.com/ds-lab/metrion/blob/main/metrion/orchestrator.py"
click node_config "https://github.com/ds-lab/metrion/blob/main/metrion/config_reader/config_reader.py"
click node_hardware "https://github.com/ds-lab/metrion/blob/main/metrion/discovery/hardware_detection.py"
click node_scanner "https://github.com/ds-lab/metrion/tree/main/metrion/discovery/system_scanner/bare_metal/linux"
click node_tree "https://github.com/ds-lab/metrion/blob/main/metrion/discovery/tree_builder.py"
click node_monitor "https://github.com/ds-lab/metrion/blob/main/metrion/monitoring/entity_monitor.py"
click node_collector "https://github.com/ds-lab/metrion/blob/main/metrion/data_collector.py"
click node_source_factory "https://github.com/ds-lab/metrion/blob/main/metrion/monitoring/data_source/data_source_factory.py"
click node_cpu_source "https://github.com/ds-lab/metrion/tree/main/metrion/monitoring/data_source/cpu"
click node_gpu_source "https://github.com/ds-lab/metrion/blob/main/metrion/monitoring/data_source/gpu/nvml.py"
click node_memory_source "https://github.com/ds-lab/metrion/blob/main/metrion/monitoring/data_source/memory/proc_numa.py"
click node_network_source "https://github.com/ds-lab/metrion/blob/main/metrion/monitoring/data_source/network/tcp_monitor.py"
click node_registry "https://github.com/ds-lab/metrion/blob/main/metrion/entity/logical_entity_registry.py"
click node_entities "https://github.com/ds-lab/metrion/tree/main/metrion/entity"
click node_metrics "https://github.com/ds-lab/metrion/blob/main/metrion/metric.py"
click node_measurements "https://github.com/ds-lab/metrion/blob/main/metrion/data_record.py"
click node_storage "https://github.com/ds-lab/metrion/blob/main/metrion/data_storage/db_data_storage.py"
click node_attribution "https://github.com/ds-lab/metrion/blob/main/metrion/energy_attribution/energy_attribution.py"
click node_analysis_cli "https://github.com/ds-lab/metrion/blob/main/metrion/energy_attribution/energy_attribution_cli.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_orchestrator,node_config toneBlue
class node_hardware,node_scanner,node_tree toneAmber
class node_monitor,node_collector,node_source_factory,node_cpu_source,node_gpu_source,node_memory_source,node_network_source toneMint
class node_registry,node_entities,node_metrics,node_measurements toneRose
class node_storage,node_db,node_attribution,node_analysis_cli,node_operator toneIndigo
```

## Components

### Runtime control

- [Monitor orchestrator](https://github.com/ds-lab/metrion/blob/main/metrion/orchestrator.py): `metrion/orchestrator.py`
- [Configuration reader](https://github.com/ds-lab/metrion/blob/main/metrion/config_reader/config_reader.py): `metrion/config_reader/config_reader.py`

### Host discovery

- [Hardware detection](https://github.com/ds-lab/metrion/blob/main/metrion/discovery/hardware_detection.py): `metrion/discovery/hardware_detection.py`
- [Linux system scanner](https://github.com/ds-lab/metrion/tree/main/metrion/discovery/system_scanner/bare_metal/linux): `metrion/discovery/system_scanner/bare_metal/linux`
- [System tree builder](https://github.com/ds-lab/metrion/blob/main/metrion/discovery/tree_builder.py): `metrion/discovery/tree_builder.py`

### Monitoring

- [Entity monitor](https://github.com/ds-lab/metrion/blob/main/metrion/monitoring/entity_monitor.py): `metrion/monitoring/entity_monitor.py`
- [Data collector](https://github.com/ds-lab/metrion/blob/main/metrion/data_collector.py): `metrion/data_collector.py`
- [Data source factory](https://github.com/ds-lab/metrion/blob/main/metrion/monitoring/data_source/data_source_factory.py): `metrion/monitoring/data_source/data_source_factory.py`
- [CPU counters and events](https://github.com/ds-lab/metrion/tree/main/metrion/monitoring/data_source/cpu): `metrion/monitoring/data_source/cpu`
- [NVIDIA NVML source](https://github.com/ds-lab/metrion/blob/main/metrion/monitoring/data_source/gpu/nvml.py): `metrion/monitoring/data_source/gpu/nvml.py`
- [NUMA memory source](https://github.com/ds-lab/metrion/blob/main/metrion/monitoring/data_source/memory/proc_numa.py): `metrion/monitoring/data_source/memory/proc_numa.py`
- [TCP network source](https://github.com/ds-lab/metrion/blob/main/metrion/monitoring/data_source/network/tcp_monitor.py): `metrion/monitoring/data_source/network/tcp_monitor.py`

### Data model

- [Process entity registry](https://github.com/ds-lab/metrion/blob/main/metrion/entity/logical_entity_registry.py): `metrion/entity/logical_entity_registry.py`
- [Hardware entity model](https://github.com/ds-lab/metrion/tree/main/metrion/entity): `metrion/entity`
- [Metric definitions](https://github.com/ds-lab/metrion/blob/main/metrion/metric.py): `metrion/metric.py`
- [Measurements and records](https://github.com/ds-lab/metrion/blob/main/metrion/data_record.py): `metrion/data_record.py`

### Storage and analysis

- [SQLite database storage](https://github.com/ds-lab/metrion/blob/main/metrion/data_storage/db_data_storage.py): `metrion/data_storage/db_data_storage.py`
- **Session database**
- [Energy attribution](https://github.com/ds-lab/metrion/blob/main/metrion/energy_attribution/energy_attribution.py): `metrion/energy_attribution/energy_attribution.py`
- [Analysis CLI](https://github.com/ds-lab/metrion/blob/main/metrion/energy_attribution/energy_attribution_cli.py): `metrion/energy_attribution/energy_attribution_cli.py`

### Other components

- **Operator**

## Connections

- Operator → Monitor orchestrator: starts monitoring
- Operator → Analysis CLI: runs analysis
- Monitor orchestrator → Configuration reader: loads settings (evidence: `metrion/orchestrator.py`)
- Monitor orchestrator → Hardware detection: detects hardware (evidence: `metrion/orchestrator.py`)
- Hardware detection → System tree builder: builds topology (evidence: `metrion/discovery/hardware_detection.py`)
- System tree builder → Linux system scanner: uses system info (evidence: `metrion/discovery/tree_builder.py`)
- Monitor orchestrator → Entity monitor: starts monitoring (evidence: `metrion/orchestrator.py`)
- Monitor orchestrator → Data collector: uses collector (evidence: `metrion/orchestrator.py`)
- Entity monitor → Data source factory: selects sources
- Data source factory → CPU counters and events: dispatches to
- Data source factory → NVIDIA NVML source: dispatches to
- Data source factory → NUMA memory source: dispatches to
- Data source factory → TCP network source: dispatches to
- CPU counters and events → Process entity registry: updates processes (evidence: `metrion/monitoring/data_source/cpu/sched_switch.py`)
- NVIDIA NVML source → Process entity registry: resolves processes (evidence: `metrion/monitoring/data_source/gpu/nvml.py`)
- TCP network source → Process entity registry: resolves processes (evidence: `metrion/monitoring/data_source/network/tcp_monitor.py`)
- CPU counters and events → Measurements and records: emits records (evidence: `metrion/monitoring/data_source/cpu/sched_switch.py`)
- NVIDIA NVML source → Measurements and records: emits records (evidence: `metrion/monitoring/data_source/gpu/nvml.py`)
- TCP network source → Measurements and records: emits records (evidence: `metrion/monitoring/data_source/network/tcp_monitor.py`)
- Measurements and records → Hardware entity model: references entities (evidence: `metrion/data_record.py`)
- Measurements and records → Metric definitions: references metrics (evidence: `metrion/monitoring/data_source/data_source_base.py`)
- Data collector → SQLite database storage: persists records
- SQLite database storage → Session database: writes session data
- Analysis CLI → Session database: reads session data
- Analysis CLI → Energy attribution: runs attribution (evidence: `metrion/energy_attribution/energy_attribution_cli.py`)
- Energy attribution → Measurements and records: uses measurement model (evidence: `metrion/energy_attribution/energy_attribution.py`)
