# danielcristho/croft architecture

> Architecture diagram of the GitHub repository danielcristho/croft, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/danielcristho/croft
- Repository: https://github.com/danielcristho/croft

## Overview

Croft is an experimental Kubernetes autoscaler driven by per-container eBPF and kernel signals. Its main workflow is: the node collector samples processes and kernel counters, records dataset rows and exposes metrics, then `croft scale` uses calibrated syscall rates to adjust a Deployment. The CLI also collects and analyzes datasets; the Python package provides dataset and Prometheus analysis, while the research harness runs and compares workloads. Several subsystem links are supported by README descriptions rather than sampled call sites, so those links cite the README; unsampled implementation details are not asserted.

Main entry points: [`collector/collector/src/cli.rs`](https://github.com/danielcristho/croft/blob/main/collector/collector/src/cli.rs) (Croft CLI), [`collector/collector/src/command/run.rs`](https://github.com/danielcristho/croft/blob/main/collector/collector/src/command/run.rs) (Collector runtime), [`collector/collector/src/pipeline/dataset.rs`](https://github.com/danielcristho/croft/blob/main/collector/collector/src/pipeline/dataset.rs) (Dataset pipeline), [`labs/research/experiment/run.py`](https://github.com/danielcristho/croft/blob/main/labs/research/experiment/run.py) (Experiment runner)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_collection["Node collection"]
  node_cli["Croft CLI<br/>[cli.rs]"]
  node_ebpf["eBPF probes"]
  node_sensing["Kernel sensing"]
  node_inspection["Process inspection"]
  node_collector["Collector runtime<br/>[run.rs]"]
  node_metrics["Prometheus metrics<br/>[metrics.rs]"]
  node_framework["Framework detection<br/>[framework.rs]"]
  node_resources["Resource inspection<br/>[resources.rs]"]
end

subgraph group_scaling["Autoscaling"]
  node_scaler["Scaling decisions<br/>[scaler.rs]"]
end

subgraph group_analysis["Dataset analysis"]
  node_dataset[("Dataset pipeline<br/>[dataset.rs]")]
  node_segments[("Compressed segments<br/>[segments.rs]")]
  node_collect_cmd["Cluster collect<br/>[collect.rs]"]
  node_stats["Dataset statistics<br/>[stats.rs]"]
  node_profile["Resource profiles<br/>[profile.rs]"]
  node_top["Live top view<br/>[top.rs]"]
end

subgraph group_research["Research harness"]
  node_experiments["Experiment runner<br/>[run.py]"]
  node_web_runs["Web load runs<br/>[web.py]"]
  node_profiles["Load profiles<br/>[profiles.py]"]
  node_app_catalog["Test app catalog<br/>[apps.py]"]
  node_comparison["Run comparison<br/>[compare.py]"]
  node_prometheus_queries["Prometheus queries<br/>[prometheus.py]"]
end

subgraph group_python["Python analysis"]
  node_py_dataset["Python dataset reader<br/>[dataset.py]"]
  node_py_metrics["Python metrics<br/>[metrics.py]"]
end

node_operator(("Operator"))
node_metrics_service["Prometheus"]
node_deployment["Kubernetes Deployment"]

node_operator -->|"invokes commands"| node_cli
node_cli -.->|"starts collection"| node_collector
node_ebpf -->|"provides kernel events"| node_sensing
node_sensing -->|"supplies signals"| node_collector
node_inspection -->|"adds process data"| node_collector
node_collector -->|"writes rows"| node_dataset
node_dataset -->|"rotates and compresses"| node_segments
node_collector -->|"publishes metrics"| node_metrics
node_metrics -->|"serves Prometheus"| node_metrics_service
node_cli -->|"runs scale"| node_scaler
node_metrics_service -->|"provides syscall rates"| node_scaler
node_scaler -->|"sets replicas"| node_deployment
node_cli -->|"runs collect"| node_collect_cmd
node_collect_cmd -->|"pulls node data"| node_dataset
node_cli -->|"runs stats"| node_stats
node_stats -->|"reads rows"| node_dataset
node_cli -->|"runs profile"| node_profile
node_profile -->|"reads rows"| node_dataset
node_cli -->|"runs top"| node_top
node_top -->|"renders resource data"| node_resources
node_collector -->|"detects frameworks"| node_framework
node_collector -->|"inspects usage"| node_resources
node_py_dataset -->|"reads dataset format"| node_dataset
node_py_metrics -->|"queries metrics"| node_metrics_service
node_operator -.->|"runs experiments"| node_experiments
node_experiments -->|"dispatches web runs"| node_web_runs
node_experiments -->|"dispatches comparisons"| node_comparison
node_experiments -->|"loads app catalog"| node_app_catalog
node_web_runs -->|"uses load profiles"| node_profiles
node_web_runs -->|"runs configured apps"| node_app_catalog
node_comparison -->|"compares metrics"| node_prometheus_queries
node_comparison -->|"uses app definitions"| node_app_catalog

click node_cli "https://github.com/danielcristho/croft/blob/main/collector/collector/src/cli.rs"
click node_ebpf "https://github.com/danielcristho/croft/tree/main/collector/collector-ebpf/src"
click node_sensing "https://github.com/danielcristho/croft/tree/main/collector/collector/src/sensing"
click node_inspection "https://github.com/danielcristho/croft/tree/main/collector/collector/src/inspect"
click node_collector "https://github.com/danielcristho/croft/blob/main/collector/collector/src/command/run.rs"
click node_dataset "https://github.com/danielcristho/croft/blob/main/collector/collector/src/pipeline/dataset.rs"
click node_segments "https://github.com/danielcristho/croft/blob/main/collector/collector/src/pipeline/segments.rs"
click node_metrics "https://github.com/danielcristho/croft/blob/main/collector/collector/src/pipeline/metrics.rs"
click node_scaler "https://github.com/danielcristho/croft/blob/main/collector/collector/src/decision/scaler.rs"
click node_collect_cmd "https://github.com/danielcristho/croft/blob/main/collector/collector/src/command/collect.rs"
click node_stats "https://github.com/danielcristho/croft/blob/main/collector/collector/src/command/stats.rs"
click node_profile "https://github.com/danielcristho/croft/blob/main/collector/collector/src/command/profile.rs"
click node_top "https://github.com/danielcristho/croft/blob/main/collector/collector/src/command/top.rs"
click node_framework "https://github.com/danielcristho/croft/blob/main/collector/collector/src/inspect/framework.rs"
click node_resources "https://github.com/danielcristho/croft/blob/main/collector/collector/src/inspect/resources.rs"
click node_py_dataset "https://github.com/danielcristho/croft/blob/main/python/src/croft/dataset.py"
click node_py_metrics "https://github.com/danielcristho/croft/blob/main/python/src/croft/metrics.py"
click node_experiments "https://github.com/danielcristho/croft/blob/main/labs/research/experiment/run.py"
click node_web_runs "https://github.com/danielcristho/croft/blob/main/labs/research/experiment/experiment/web.py"
click node_profiles "https://github.com/danielcristho/croft/blob/main/labs/research/experiment/experiment/profiles.py"
click node_app_catalog "https://github.com/danielcristho/croft/blob/main/labs/research/experiment/experiment/apps.py"
click node_comparison "https://github.com/danielcristho/croft/blob/main/labs/research/experiment/experiment/compare.py"
click node_prometheus_queries "https://github.com/danielcristho/croft/blob/main/labs/research/experiment/experiment/prometheus.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_ebpf,node_sensing,node_inspection,node_collector,node_metrics,node_framework,node_resources toneBlue
class node_scaler toneAmber
class node_dataset,node_segments,node_collect_cmd,node_stats,node_profile,node_top toneMint
class node_experiments,node_web_runs,node_profiles,node_app_catalog,node_comparison,node_prometheus_queries toneRose
class node_py_dataset,node_py_metrics,node_operator,node_metrics_service,node_deployment toneIndigo
```

## Components

### Node collection

- [Croft CLI](https://github.com/danielcristho/croft/blob/main/collector/collector/src/cli.rs): `collector/collector/src/cli.rs`
- [eBPF probes](https://github.com/danielcristho/croft/tree/main/collector/collector-ebpf/src): `collector/collector-ebpf/src`
- [Kernel sensing](https://github.com/danielcristho/croft/tree/main/collector/collector/src/sensing): `collector/collector/src/sensing`
- [Process inspection](https://github.com/danielcristho/croft/tree/main/collector/collector/src/inspect): `collector/collector/src/inspect`
- [Collector runtime](https://github.com/danielcristho/croft/blob/main/collector/collector/src/command/run.rs): `collector/collector/src/command/run.rs`
- [Prometheus metrics](https://github.com/danielcristho/croft/blob/main/collector/collector/src/pipeline/metrics.rs): `collector/collector/src/pipeline/metrics.rs`
- [Framework detection](https://github.com/danielcristho/croft/blob/main/collector/collector/src/inspect/framework.rs): `collector/collector/src/inspect/framework.rs`
- [Resource inspection](https://github.com/danielcristho/croft/blob/main/collector/collector/src/inspect/resources.rs): `collector/collector/src/inspect/resources.rs`

### Autoscaling

- [Scaling decisions](https://github.com/danielcristho/croft/blob/main/collector/collector/src/decision/scaler.rs): `collector/collector/src/decision/scaler.rs`

### Dataset analysis

- [Dataset pipeline](https://github.com/danielcristho/croft/blob/main/collector/collector/src/pipeline/dataset.rs): `collector/collector/src/pipeline/dataset.rs`
- [Compressed segments](https://github.com/danielcristho/croft/blob/main/collector/collector/src/pipeline/segments.rs): `collector/collector/src/pipeline/segments.rs`
- [Cluster collect](https://github.com/danielcristho/croft/blob/main/collector/collector/src/command/collect.rs): `collector/collector/src/command/collect.rs`
- [Dataset statistics](https://github.com/danielcristho/croft/blob/main/collector/collector/src/command/stats.rs): `collector/collector/src/command/stats.rs`
- [Resource profiles](https://github.com/danielcristho/croft/blob/main/collector/collector/src/command/profile.rs): `collector/collector/src/command/profile.rs`
- [Live top view](https://github.com/danielcristho/croft/blob/main/collector/collector/src/command/top.rs): `collector/collector/src/command/top.rs`

### Research harness

- [Experiment runner](https://github.com/danielcristho/croft/blob/main/labs/research/experiment/run.py): `labs/research/experiment/run.py`
- [Web load runs](https://github.com/danielcristho/croft/blob/main/labs/research/experiment/experiment/web.py): `labs/research/experiment/experiment/web.py`
- [Load profiles](https://github.com/danielcristho/croft/blob/main/labs/research/experiment/experiment/profiles.py): `labs/research/experiment/experiment/profiles.py`
- [Test app catalog](https://github.com/danielcristho/croft/blob/main/labs/research/experiment/experiment/apps.py): `labs/research/experiment/experiment/apps.py`
- [Run comparison](https://github.com/danielcristho/croft/blob/main/labs/research/experiment/experiment/compare.py): `labs/research/experiment/experiment/compare.py`
- [Prometheus queries](https://github.com/danielcristho/croft/blob/main/labs/research/experiment/experiment/prometheus.py): `labs/research/experiment/experiment/prometheus.py`

### Python analysis

- [Python dataset reader](https://github.com/danielcristho/croft/blob/main/python/src/croft/dataset.py): `python/src/croft/dataset.py`
- [Python metrics](https://github.com/danielcristho/croft/blob/main/python/src/croft/metrics.py): `python/src/croft/metrics.py`

### Other components

- **Operator**
- **Prometheus**
- **Kubernetes Deployment**

## Connections

- Operator → Croft CLI: invokes commands (evidence: `README.md`)
- Croft CLI → Collector runtime: starts collection
- eBPF probes → Kernel sensing: provides kernel events (evidence: `README.md`)
- Kernel sensing → Collector runtime: supplies signals (evidence: `README.md`)
- Process inspection → Collector runtime: adds process data (evidence: `README.md`)
- Collector runtime → Dataset pipeline: writes rows (evidence: `README.md`)
- Dataset pipeline → Compressed segments: rotates and compresses (evidence: `README.md`)
- Collector runtime → Prometheus metrics: publishes metrics (evidence: `README.md`)
- Prometheus metrics → Prometheus: serves Prometheus (evidence: `README.md`)
- Croft CLI → Scaling decisions: runs scale
- Prometheus → Scaling decisions: provides syscall rates (evidence: `README.md`)
- Scaling decisions → Kubernetes Deployment: sets replicas (evidence: `README.md`)
- Croft CLI → Cluster collect: runs collect
- Cluster collect → Dataset pipeline: pulls node data (evidence: `README.md`)
- Croft CLI → Dataset statistics: runs stats
- Dataset statistics → Dataset pipeline: reads rows (evidence: `README.md`)
- Croft CLI → Resource profiles: runs profile
- Resource profiles → Dataset pipeline: reads rows (evidence: `collector/collector/src/command/profile.rs`)
- Croft CLI → Live top view: runs top
- Live top view → Resource inspection: renders resource data
- Collector runtime → Framework detection: detects frameworks (evidence: `README.md`)
- Collector runtime → Resource inspection: inspects usage (evidence: `README.md`)
- Python dataset reader → Dataset pipeline: reads dataset format (evidence: `README.md`)
- Python metrics → Prometheus: queries metrics (evidence: `python/src/croft/metrics.py`)
- Operator → Experiment runner: runs experiments
- Experiment runner → Web load runs: dispatches web runs (evidence: `labs/research/experiment/run.py`)
- Experiment runner → Run comparison: dispatches comparisons (evidence: `labs/research/experiment/run.py`)
- Experiment runner → Test app catalog: loads app catalog (evidence: `labs/research/experiment/run.py`)
- Web load runs → Load profiles: uses load profiles (evidence: `labs/research/experiment/experiment/web.py`)
- Web load runs → Test app catalog: runs configured apps (evidence: `labs/research/experiment/experiment/web.py`)
- Run comparison → Prometheus queries: compares metrics (evidence: `labs/research/experiment/experiment/compare.py`)
- Run comparison → Test app catalog: uses app definitions (evidence: `labs/research/experiment/experiment/compare.py`)
