# replit/datacat architecture

> Architecture diagram of the GitHub repository replit/datacat, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/replit/datacat
- Repository: https://github.com/replit/datacat

## Overview

DataCat is a self-hosted observability product. Its main workflow is sending telemetry from agents or SDKs to the API, which decodes and stores it; users then explore metrics, logs, traces, hosts and dashboards in the web UI and manage monitors. The graph preserves those documented product areas and highlights authentication and storage as internal layers. Some query, monitor and SDK details are not excerpted, so connections to those components are omitted or marked unverified rather than inferred from names alone.

Main entry points: [`artifacts/api-server/src/intake/routes.ts`](https://github.com/replit/datacat/blob/develop/artifacts/api-server/src/intake/routes.ts) (Telemetry endpoints), [`artifacts/web/src/App.tsx`](https://github.com/replit/datacat/blob/develop/artifacts/web/src/App.tsx) (Web application), [`artifacts/api-server/src/intake/store.ts`](https://github.com/replit/datacat/blob/develop/artifacts/api-server/src/intake/store.ts) (Telemetry writes), [`artifacts/api-server/src/middlewares/authMiddleware.ts`](https://github.com/replit/datacat/blob/develop/artifacts/api-server/src/middlewares/authMiddleware.ts) (Request authentication)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_senders["Telemetry senders"]
  node_agent["Host agent<br/>[main.go]"]
  node_node_sdk["Node SDK<br/>[index.js]"]
  node_node_export["Node span exporter<br/>[span-exporter.js]"]
  node_python_sdk["Python SDK<br/>[__init__.py]"]
  node_python_export["Python span exporter<br/>[_span_exporter.py]"]
end

subgraph group_intake["API and intake"]
  node_api_app["API application<br/>[app.ts]"]
  node_intake_routes["Telemetry endpoints<br/>[routes.ts]"]
  node_decode["Payload decoding<br/>[decode.ts]"]
  node_proto["Protocol decoding<br/>[proto.ts]"]
  node_otlp["OTLP handling<br/>[otlp.ts]"]
  node_agent_traces["Agent trace handling<br/>[agentTraces.ts]"]
end

subgraph group_storage["Storage and queries"]
  node_intake_store["Telemetry writes<br/>[store.ts]"]
  node_storage_choice["Storage selection<br/>[storage.ts]"]
  node_clickhouse["ClickHouse access<br/>[ch.ts]"]
  node_apm_stats["APM rollups<br/>[apmStats.ts]"]
  node_query_api["Query services"]
  node_postgres[("PostgreSQL")]
end

subgraph group_product["Web product"]
  node_web_app["Web application<br/>[App.tsx]"]
  node_dashboards["Dashboards<br/>[Dashboards.tsx]"]
  node_metrics["Metrics explorer<br/>[Metrics.tsx]"]
  node_logs["Log explorer<br/>[Logs.tsx]"]
  node_apm_ui["APM explorer<br/>[Apm.tsx]"]
  node_infrastructure["Host maps<br/>[Infrastructure.tsx]"]
  node_monitors_ui["Monitor management<br/>[Monitors.tsx]"]
  node_monitor_jobs["Monitor evaluation<br/>[monitors.ts]"]
end

subgraph group_identity["Identity and operations"]
  node_auth_middleware["Request authentication<br/>[authMiddleware.ts]"]
  node_auth["Sessions and passwords<br/>[auth.ts]"]
  node_org["Organizations and keys<br/>[org.ts]"]
end

node_user(("Observability user"))
node_external{{"Webhook and Slack"}}

node_user -->|"uses"| node_web_app
node_web_app -->|"routes to"| node_dashboards
node_web_app -->|"routes to"| node_metrics
node_web_app -->|"routes to"| node_logs
node_web_app -->|"routes to"| node_apm_ui
node_web_app -->|"routes to"| node_infrastructure
node_web_app -->|"routes to"| node_monitors_ui
node_agent -.->|"sends telemetry"| node_intake_routes
node_node_sdk -->|"configures"| node_node_export
node_node_sdk -.->|"exports OTLP"| node_intake_routes
node_python_sdk -->|"uses"| node_python_export
node_python_sdk -.->|"sends telemetry"| node_intake_routes
node_api_app -->|"mounts"| node_intake_routes
node_intake_routes -->|"decodes payloads"| node_decode
node_intake_routes -->|"decodes protobuf"| node_proto
node_intake_routes -->|"decodes OTLP"| node_otlp
node_intake_routes -->|"decodes agent traces"| node_agent_traces
node_intake_routes -->|"writes telemetry"| node_intake_store
node_intake_routes -->|"resolves API key"| node_org
node_intake_store -->|"inserts telemetry"| node_clickhouse
node_intake_store -->|"records span stats"| node_apm_stats
node_intake_store -.->|"writes when selected"| node_postgres
node_intake_store -.->|"selects backend"| node_storage_choice
node_query_api -.->|"queries optionally"| node_clickhouse
node_query_api -.->|"queries by default"| node_postgres
node_api_app -->|"protects API"| node_auth_middleware
node_auth_middleware -->|"checks sessions"| node_auth
node_auth_middleware -->|"resolves membership"| node_org
node_web_app -.->|"requests data"| node_query_api
node_monitor_jobs -.->|"notifies"| node_external
node_user -.->|"configures monitors"| node_monitors_ui

click node_agent "https://github.com/replit/datacat/blob/develop/agent/main.go"
click node_node_sdk "https://github.com/replit/datacat/blob/develop/sdks/node/index.js"
click node_node_export "https://github.com/replit/datacat/blob/develop/sdks/node/lib/span-exporter.js"
click node_python_sdk "https://github.com/replit/datacat/blob/develop/sdks/python/src/datacat/__init__.py"
click node_python_export "https://github.com/replit/datacat/blob/develop/sdks/python/src/datacat/_span_exporter.py"
click node_api_app "https://github.com/replit/datacat/blob/develop/artifacts/api-server/src/app.ts"
click node_intake_routes "https://github.com/replit/datacat/blob/develop/artifacts/api-server/src/intake/routes.ts"
click node_decode "https://github.com/replit/datacat/blob/develop/artifacts/api-server/src/intake/decode.ts"
click node_proto "https://github.com/replit/datacat/blob/develop/artifacts/api-server/src/intake/proto.ts"
click node_otlp "https://github.com/replit/datacat/blob/develop/artifacts/api-server/src/intake/otlp.ts"
click node_agent_traces "https://github.com/replit/datacat/blob/develop/artifacts/api-server/src/intake/agentTraces.ts"
click node_intake_store "https://github.com/replit/datacat/blob/develop/artifacts/api-server/src/intake/store.ts"
click node_storage_choice "https://github.com/replit/datacat/blob/develop/artifacts/api-server/src/lib/storage.ts"
click node_clickhouse "https://github.com/replit/datacat/blob/develop/artifacts/api-server/src/lib/ch.ts"
click node_apm_stats "https://github.com/replit/datacat/blob/develop/artifacts/api-server/src/query/apmStats.ts"
click node_query_api "https://github.com/replit/datacat/tree/develop/artifacts/api-server/src/query"
click node_web_app "https://github.com/replit/datacat/blob/develop/artifacts/web/src/App.tsx"
click node_dashboards "https://github.com/replit/datacat/blob/develop/artifacts/web/src/pages/Dashboards.tsx"
click node_metrics "https://github.com/replit/datacat/blob/develop/artifacts/web/src/pages/Metrics.tsx"
click node_logs "https://github.com/replit/datacat/blob/develop/artifacts/web/src/pages/Logs.tsx"
click node_apm_ui "https://github.com/replit/datacat/blob/develop/artifacts/web/src/pages/Apm.tsx"
click node_infrastructure "https://github.com/replit/datacat/blob/develop/artifacts/web/src/pages/Infrastructure.tsx"
click node_monitors_ui "https://github.com/replit/datacat/blob/develop/artifacts/web/src/pages/Monitors.tsx"
click node_monitor_jobs "https://github.com/replit/datacat/blob/develop/artifacts/api-server/src/jobs/monitors.ts"
click node_auth_middleware "https://github.com/replit/datacat/blob/develop/artifacts/api-server/src/middlewares/authMiddleware.ts"
click node_auth "https://github.com/replit/datacat/blob/develop/artifacts/api-server/src/lib/auth.ts"
click node_org "https://github.com/replit/datacat/blob/develop/artifacts/api-server/src/lib/org.ts"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_agent,node_node_sdk,node_node_export,node_python_sdk,node_python_export,node_user toneBlue
class node_api_app,node_intake_routes,node_decode,node_proto,node_otlp,node_agent_traces toneAmber
class node_intake_store,node_storage_choice,node_clickhouse,node_apm_stats,node_query_api,node_postgres,node_external toneMint
class node_web_app,node_dashboards,node_metrics,node_logs,node_apm_ui,node_infrastructure,node_monitors_ui,node_monitor_jobs toneRose
class node_auth_middleware,node_auth,node_org toneIndigo
```

## Components

### Telemetry senders

- [Host agent](https://github.com/replit/datacat/blob/develop/agent/main.go): `agent/main.go`
- [Node SDK](https://github.com/replit/datacat/blob/develop/sdks/node/index.js): `sdks/node/index.js`
- [Node span exporter](https://github.com/replit/datacat/blob/develop/sdks/node/lib/span-exporter.js): `sdks/node/lib/span-exporter.js`
- [Python SDK](https://github.com/replit/datacat/blob/develop/sdks/python/src/datacat/__init__.py): `sdks/python/src/datacat/__init__.py`
- [Python span exporter](https://github.com/replit/datacat/blob/develop/sdks/python/src/datacat/_span_exporter.py): `sdks/python/src/datacat/_span_exporter.py`

### API and intake

- [API application](https://github.com/replit/datacat/blob/develop/artifacts/api-server/src/app.ts): `artifacts/api-server/src/app.ts`
- [Telemetry endpoints](https://github.com/replit/datacat/blob/develop/artifacts/api-server/src/intake/routes.ts): `artifacts/api-server/src/intake/routes.ts`
- [Payload decoding](https://github.com/replit/datacat/blob/develop/artifacts/api-server/src/intake/decode.ts): `artifacts/api-server/src/intake/decode.ts`
- [Protocol decoding](https://github.com/replit/datacat/blob/develop/artifacts/api-server/src/intake/proto.ts): `artifacts/api-server/src/intake/proto.ts`
- [OTLP handling](https://github.com/replit/datacat/blob/develop/artifacts/api-server/src/intake/otlp.ts): `artifacts/api-server/src/intake/otlp.ts`
- [Agent trace handling](https://github.com/replit/datacat/blob/develop/artifacts/api-server/src/intake/agentTraces.ts): `artifacts/api-server/src/intake/agentTraces.ts`

### Storage and queries

- [Telemetry writes](https://github.com/replit/datacat/blob/develop/artifacts/api-server/src/intake/store.ts): `artifacts/api-server/src/intake/store.ts`
- [Storage selection](https://github.com/replit/datacat/blob/develop/artifacts/api-server/src/lib/storage.ts): `artifacts/api-server/src/lib/storage.ts`
- [ClickHouse access](https://github.com/replit/datacat/blob/develop/artifacts/api-server/src/lib/ch.ts): `artifacts/api-server/src/lib/ch.ts`
- [APM rollups](https://github.com/replit/datacat/blob/develop/artifacts/api-server/src/query/apmStats.ts): `artifacts/api-server/src/query/apmStats.ts`
- [Query services](https://github.com/replit/datacat/tree/develop/artifacts/api-server/src/query): `artifacts/api-server/src/query`
- **PostgreSQL**

### Web product

- [Web application](https://github.com/replit/datacat/blob/develop/artifacts/web/src/App.tsx): `artifacts/web/src/App.tsx`
- [Dashboards](https://github.com/replit/datacat/blob/develop/artifacts/web/src/pages/Dashboards.tsx): `artifacts/web/src/pages/Dashboards.tsx`
- [Metrics explorer](https://github.com/replit/datacat/blob/develop/artifacts/web/src/pages/Metrics.tsx): `artifacts/web/src/pages/Metrics.tsx`
- [Log explorer](https://github.com/replit/datacat/blob/develop/artifacts/web/src/pages/Logs.tsx): `artifacts/web/src/pages/Logs.tsx`
- [APM explorer](https://github.com/replit/datacat/blob/develop/artifacts/web/src/pages/Apm.tsx): `artifacts/web/src/pages/Apm.tsx`
- [Host maps](https://github.com/replit/datacat/blob/develop/artifacts/web/src/pages/Infrastructure.tsx): `artifacts/web/src/pages/Infrastructure.tsx`
- [Monitor management](https://github.com/replit/datacat/blob/develop/artifacts/web/src/pages/Monitors.tsx): `artifacts/web/src/pages/Monitors.tsx`
- [Monitor evaluation](https://github.com/replit/datacat/blob/develop/artifacts/api-server/src/jobs/monitors.ts): `artifacts/api-server/src/jobs/monitors.ts`

### Identity and operations

- [Request authentication](https://github.com/replit/datacat/blob/develop/artifacts/api-server/src/middlewares/authMiddleware.ts): `artifacts/api-server/src/middlewares/authMiddleware.ts`
- [Sessions and passwords](https://github.com/replit/datacat/blob/develop/artifacts/api-server/src/lib/auth.ts): `artifacts/api-server/src/lib/auth.ts`
- [Organizations and keys](https://github.com/replit/datacat/blob/develop/artifacts/api-server/src/lib/org.ts): `artifacts/api-server/src/lib/org.ts`

### Other components

- **Observability user**
- **Webhook and Slack**

## Connections

- Observability user → Web application: uses (evidence: `artifacts/web/src/App.tsx`)
- Web application → Dashboards: routes to (evidence: `artifacts/web/src/App.tsx`)
- Web application → Metrics explorer: routes to (evidence: `artifacts/web/src/App.tsx`)
- Web application → Log explorer: routes to (evidence: `artifacts/web/src/App.tsx`)
- Web application → APM explorer: routes to (evidence: `artifacts/web/src/App.tsx`)
- Web application → Host maps: routes to (evidence: `artifacts/web/src/App.tsx`)
- Web application → Monitor management: routes to (evidence: `artifacts/web/src/App.tsx`)
- Host agent → Telemetry endpoints: sends telemetry
- Node SDK → Node span exporter: configures (evidence: `sdks/node/index.js`)
- Node SDK → Telemetry endpoints: exports OTLP
- Python SDK → Python span exporter: uses (evidence: `sdks/python/src/datacat/__init__.py`)
- Python SDK → Telemetry endpoints: sends telemetry
- API application → Telemetry endpoints: mounts (evidence: `artifacts/api-server/src/app.ts`)
- Telemetry endpoints → Payload decoding: decodes payloads (evidence: `artifacts/api-server/src/intake/routes.ts`)
- Telemetry endpoints → Protocol decoding: decodes protobuf (evidence: `artifacts/api-server/src/intake/routes.ts`)
- Telemetry endpoints → OTLP handling: decodes OTLP (evidence: `artifacts/api-server/src/intake/routes.ts`)
- Telemetry endpoints → Agent trace handling: decodes agent traces (evidence: `artifacts/api-server/src/intake/routes.ts`)
- Telemetry endpoints → Telemetry writes: writes telemetry (evidence: `artifacts/api-server/src/intake/routes.ts`)
- Telemetry endpoints → Organizations and keys: resolves API key (evidence: `artifacts/api-server/src/intake/routes.ts`)
- Telemetry writes → ClickHouse access: inserts telemetry (evidence: `artifacts/api-server/src/intake/store.ts`)
- Telemetry writes → APM rollups: records span stats (evidence: `artifacts/api-server/src/intake/store.ts`)
- Telemetry writes → PostgreSQL: writes when selected
- Telemetry writes → Storage selection: selects backend (evidence: `artifacts/api-server/src/intake/store.ts`)
- Query services → ClickHouse access: queries optionally (evidence: `artifacts/api-server/src/query/searchCh.ts`)
- Query services → PostgreSQL: queries by default
- API application → Request authentication: protects API (evidence: `artifacts/api-server/src/app.ts`)
- Request authentication → Sessions and passwords: checks sessions (evidence: `artifacts/api-server/src/middlewares/authMiddleware.ts`)
- Request authentication → Organizations and keys: resolves membership (evidence: `artifacts/api-server/src/middlewares/authMiddleware.ts`)
- Web application → Query services: requests data
- Monitor evaluation → Webhook and Slack: notifies
- Observability user → Monitor management: configures monitors
