# dch2r/signalfloor architecture

> Architecture diagram of the GitHub repository dch2r/signalfloor, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dch2r/signalfloor
- Repository: https://github.com/dch2r/signalfloor

## Overview

signalfloor helps teams distinguish real evaluation regressions from noise, assess judge reliability, and measure run-to-run flakiness. The CLI compares case-paired scores, calibrates judge labels, and analyzes repeated runs; the server and dashboard add stored runs, comparisons, jobs, and OpenTelemetry traces. A reference agent supports retrieval-backed evaluation. The map emphasizes documented capabilities, but some server, dashboard, and agent connections are inferred only at the product level because their wiring was not sampled.

Main entry points: [`packages/signalfloor/src/signalfloor/cli.py`](https://github.com/dch2r/signalfloor/blob/main/packages/signalfloor/src/signalfloor/cli.py) (CLI commands), [`packages/signalfloor/src/signalfloor/compare.py`](https://github.com/dch2r/signalfloor/blob/main/packages/signalfloor/src/signalfloor/compare.py) (Run comparison), [`packages/server/src/signalfloor_server/app.py`](https://github.com/dch2r/signalfloor/blob/main/packages/server/src/signalfloor_server/app.py) (HTTP application), [`packages/signalfloor/src/signalfloor/calibration.py`](https://github.com/dch2r/signalfloor/blob/main/packages/signalfloor/src/signalfloor/calibration.py) (Judge calibration)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_analysis["Statistical analysis"]
  node_cli["CLI commands<br/>[cli.py]"]
  node_compare["Run comparison<br/>[compare.py]"]
  node_betting["Betting bounds<br/>[_betting.py]"]
  node_pairing["Case pairing<br/>[_pairing.py]"]
  node_resample["Resampling<br/>[_resample.py]"]
  node_calibration["Judge calibration<br/>[calibration.py]"]
  node_stability["Flakiness analysis<br/>[stability.py]"]
  node_results["Result formatting<br/>[results.py]"]
end

subgraph group_agent["Reference agent"]
  node_agent["Reference agent<br/>[agent.py]"]
  node_retrieval["Passage retrieval<br/>[retrieval.py]"]
  node_embedding["Embedding client<br/>[embeddings.py]"]
end

subgraph group_server["Run data service"]
  node_server["HTTP application<br/>[app.py]"]
  node_run_api["Run endpoints<br/>[runs.py]"]
  node_comparison_api["Stored comparisons<br/>[comparisons.py]"]
  node_job_api["Job endpoints<br/>[job_routes.py]"]
  node_jobs["Job processing<br/>[jobs.py]"]
  node_trace_api["Trace ingestion<br/>[traces.py]"]
  node_db_models[("Run and span records<br/>[models.py]")]
end

subgraph group_dashboard["Dashboard"]
  node_dashboard["Web dashboard<br/>[page.tsx]"]
  node_dashboard_api["Server API client<br/>[api.ts]"]
  node_api_types["API types<br/>[api-types.ts]"]
end

node_user(("Eval practitioner"))
node_otel(("OTel exporters"))

node_user -->|"runs checks"| node_cli
node_cli -->|"dispatches"| node_compare
node_cli -->|"dispatches"| node_calibration
node_cli -->|"dispatches"| node_stability
node_cli -->|"formats results"| node_results
node_compare -->|"pairs cases"| node_pairing
node_compare -->|"computes bounds"| node_betting
node_compare -->|"bootstraps interval"| node_resample
node_calibration -->|"aligns labels"| node_pairing
node_calibration -->|"resamples agreement"| node_resample
node_stability -->|"aligns cases"| node_pairing
node_user -.->|"evaluates questions"| node_agent
node_agent -->|"retrieves passages"| node_retrieval
node_retrieval -->|"embeds content"| node_embedding
node_user -->|"reviews runs"| node_dashboard
node_dashboard -.->|"loads views"| node_dashboard_api
node_dashboard_api -->|"uses schemas"| node_api_types
node_dashboard_api -->|"requests API"| node_server
node_server -->|"registers routes"| node_run_api
node_server -->|"registers routes"| node_trace_api
node_server -->|"registers routes"| node_job_api
node_run_api -->|"queries records"| node_db_models
node_run_api -->|"computes comparisons"| node_comparison_api
node_job_api -->|"enqueues work"| node_jobs
node_jobs -->|"runs comparisons"| node_comparison_api
node_trace_api -->|"stores spans"| node_db_models
node_otel -->|"exports traces"| node_trace_api

click node_cli "https://github.com/dch2r/signalfloor/blob/main/packages/signalfloor/src/signalfloor/cli.py"
click node_compare "https://github.com/dch2r/signalfloor/blob/main/packages/signalfloor/src/signalfloor/compare.py"
click node_betting "https://github.com/dch2r/signalfloor/blob/main/packages/signalfloor/src/signalfloor/_betting.py"
click node_pairing "https://github.com/dch2r/signalfloor/blob/main/packages/signalfloor/src/signalfloor/_pairing.py"
click node_resample "https://github.com/dch2r/signalfloor/blob/main/packages/signalfloor/src/signalfloor/_resample.py"
click node_calibration "https://github.com/dch2r/signalfloor/blob/main/packages/signalfloor/src/signalfloor/calibration.py"
click node_stability "https://github.com/dch2r/signalfloor/blob/main/packages/signalfloor/src/signalfloor/stability.py"
click node_results "https://github.com/dch2r/signalfloor/blob/main/packages/signalfloor/src/signalfloor/results.py"
click node_agent "https://github.com/dch2r/signalfloor/blob/main/packages/agent/src/signalfloor_agent/agent.py"
click node_retrieval "https://github.com/dch2r/signalfloor/blob/main/packages/agent/src/signalfloor_agent/retrieval.py"
click node_embedding "https://github.com/dch2r/signalfloor/blob/main/packages/agent/src/signalfloor_agent/embeddings.py"
click node_server "https://github.com/dch2r/signalfloor/blob/main/packages/server/src/signalfloor_server/app.py"
click node_run_api "https://github.com/dch2r/signalfloor/blob/main/packages/server/src/signalfloor_server/runs.py"
click node_comparison_api "https://github.com/dch2r/signalfloor/blob/main/packages/server/src/signalfloor_server/comparisons.py"
click node_job_api "https://github.com/dch2r/signalfloor/blob/main/packages/server/src/signalfloor_server/job_routes.py"
click node_jobs "https://github.com/dch2r/signalfloor/blob/main/packages/server/src/signalfloor_server/jobs.py"
click node_trace_api "https://github.com/dch2r/signalfloor/blob/main/packages/server/src/signalfloor_server/traces.py"
click node_db_models "https://github.com/dch2r/signalfloor/blob/main/packages/server/src/signalfloor_server/models.py"
click node_dashboard "https://github.com/dch2r/signalfloor/blob/main/dashboard/src/app/page.tsx"
click node_dashboard_api "https://github.com/dch2r/signalfloor/blob/main/dashboard/src/lib/api.ts"
click node_api_types "https://github.com/dch2r/signalfloor/blob/main/dashboard/src/lib/api-types.ts"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_compare,node_betting,node_pairing,node_resample,node_calibration,node_stability,node_results toneBlue
class node_agent,node_retrieval,node_embedding toneAmber
class node_server,node_run_api,node_comparison_api,node_job_api,node_jobs,node_trace_api,node_db_models toneMint
class node_dashboard,node_dashboard_api,node_api_types toneRose
class node_user,node_otel toneIndigo
```

## Components

### Statistical analysis

- [CLI commands](https://github.com/dch2r/signalfloor/blob/main/packages/signalfloor/src/signalfloor/cli.py): `packages/signalfloor/src/signalfloor/cli.py`
- [Run comparison](https://github.com/dch2r/signalfloor/blob/main/packages/signalfloor/src/signalfloor/compare.py): `packages/signalfloor/src/signalfloor/compare.py`
- [Betting bounds](https://github.com/dch2r/signalfloor/blob/main/packages/signalfloor/src/signalfloor/_betting.py): `packages/signalfloor/src/signalfloor/_betting.py`
- [Case pairing](https://github.com/dch2r/signalfloor/blob/main/packages/signalfloor/src/signalfloor/_pairing.py): `packages/signalfloor/src/signalfloor/_pairing.py`
- [Resampling](https://github.com/dch2r/signalfloor/blob/main/packages/signalfloor/src/signalfloor/_resample.py): `packages/signalfloor/src/signalfloor/_resample.py`
- [Judge calibration](https://github.com/dch2r/signalfloor/blob/main/packages/signalfloor/src/signalfloor/calibration.py): `packages/signalfloor/src/signalfloor/calibration.py`
- [Flakiness analysis](https://github.com/dch2r/signalfloor/blob/main/packages/signalfloor/src/signalfloor/stability.py): `packages/signalfloor/src/signalfloor/stability.py`
- [Result formatting](https://github.com/dch2r/signalfloor/blob/main/packages/signalfloor/src/signalfloor/results.py): `packages/signalfloor/src/signalfloor/results.py`

### Reference agent

- [Reference agent](https://github.com/dch2r/signalfloor/blob/main/packages/agent/src/signalfloor_agent/agent.py): `packages/agent/src/signalfloor_agent/agent.py`
- [Passage retrieval](https://github.com/dch2r/signalfloor/blob/main/packages/agent/src/signalfloor_agent/retrieval.py): `packages/agent/src/signalfloor_agent/retrieval.py`
- [Embedding client](https://github.com/dch2r/signalfloor/blob/main/packages/agent/src/signalfloor_agent/embeddings.py): `packages/agent/src/signalfloor_agent/embeddings.py`

### Run data service

- [HTTP application](https://github.com/dch2r/signalfloor/blob/main/packages/server/src/signalfloor_server/app.py): `packages/server/src/signalfloor_server/app.py`
- [Run endpoints](https://github.com/dch2r/signalfloor/blob/main/packages/server/src/signalfloor_server/runs.py): `packages/server/src/signalfloor_server/runs.py`
- [Stored comparisons](https://github.com/dch2r/signalfloor/blob/main/packages/server/src/signalfloor_server/comparisons.py): `packages/server/src/signalfloor_server/comparisons.py`
- [Job endpoints](https://github.com/dch2r/signalfloor/blob/main/packages/server/src/signalfloor_server/job_routes.py): `packages/server/src/signalfloor_server/job_routes.py`
- [Job processing](https://github.com/dch2r/signalfloor/blob/main/packages/server/src/signalfloor_server/jobs.py): `packages/server/src/signalfloor_server/jobs.py`
- [Trace ingestion](https://github.com/dch2r/signalfloor/blob/main/packages/server/src/signalfloor_server/traces.py): `packages/server/src/signalfloor_server/traces.py`
- [Run and span records](https://github.com/dch2r/signalfloor/blob/main/packages/server/src/signalfloor_server/models.py): `packages/server/src/signalfloor_server/models.py`

### Dashboard

- [Web dashboard](https://github.com/dch2r/signalfloor/blob/main/dashboard/src/app/page.tsx): `dashboard/src/app/page.tsx`
- [Server API client](https://github.com/dch2r/signalfloor/blob/main/dashboard/src/lib/api.ts): `dashboard/src/lib/api.ts`
- [API types](https://github.com/dch2r/signalfloor/blob/main/dashboard/src/lib/api-types.ts): `dashboard/src/lib/api-types.ts`

### Other components

- **Eval practitioner**
- **OTel exporters**

## Connections

- Eval practitioner → CLI commands: runs checks (evidence: `README.md`)
- CLI commands → Run comparison: dispatches (evidence: `packages/signalfloor/src/signalfloor/cli.py`)
- CLI commands → Judge calibration: dispatches (evidence: `packages/signalfloor/src/signalfloor/cli.py`)
- CLI commands → Flakiness analysis: dispatches (evidence: `packages/signalfloor/src/signalfloor/cli.py`)
- CLI commands → Result formatting: formats results (evidence: `packages/signalfloor/src/signalfloor/cli.py`)
- Run comparison → Case pairing: pairs cases (evidence: `packages/signalfloor/src/signalfloor/compare.py`)
- Run comparison → Betting bounds: computes bounds (evidence: `packages/signalfloor/src/signalfloor/compare.py`)
- Run comparison → Resampling: bootstraps interval (evidence: `packages/signalfloor/src/signalfloor/compare.py`)
- Judge calibration → Case pairing: aligns labels (evidence: `packages/signalfloor/src/signalfloor/calibration.py`)
- Judge calibration → Resampling: resamples agreement (evidence: `packages/signalfloor/src/signalfloor/calibration.py`)
- Flakiness analysis → Case pairing: aligns cases (evidence: `packages/signalfloor/src/signalfloor/stability.py`)
- Eval practitioner → Reference agent: evaluates questions (evidence: `README.md`)
- Reference agent → Passage retrieval: retrieves passages (evidence: `packages/agent/src/signalfloor_agent/agent.py`)
- Passage retrieval → Embedding client: embeds content (evidence: `packages/agent/src/signalfloor_agent/retrieval.py`)
- Eval practitioner → Web dashboard: reviews runs (evidence: `README.md`)
- Web dashboard → Server API client: loads views
- Server API client → API types: uses schemas (evidence: `dashboard/src/lib/api.ts`)
- Server API client → HTTP application: requests API (evidence: `dashboard/src/lib/api.ts`)
- HTTP application → Run endpoints: registers routes (evidence: `packages/server/src/signalfloor_server/app.py`)
- HTTP application → Trace ingestion: registers routes (evidence: `packages/server/src/signalfloor_server/app.py`)
- HTTP application → Job endpoints: registers routes (evidence: `packages/server/src/signalfloor_server/app.py`)
- Run endpoints → Run and span records: queries records (evidence: `packages/server/src/signalfloor_server/runs.py`)
- Run endpoints → Stored comparisons: computes comparisons (evidence: `packages/server/src/signalfloor_server/runs.py`)
- Job endpoints → Job processing: enqueues work (evidence: `packages/server/src/signalfloor_server/job_routes.py`)
- Job processing → Stored comparisons: runs comparisons (evidence: `packages/server/src/signalfloor_server/jobs.py`)
- Trace ingestion → Run and span records: stores spans (evidence: `packages/server/src/signalfloor_server/traces.py`)
- OTel exporters → Trace ingestion: exports traces (evidence: `packages/server/src/signalfloor_server/traces.py`)