How blazeup-ai/observal works
1. What this repo is
- Observal is a self-hosted platform for AI coding agents, split into three user-facing surfaces: the registry/server, the desktop/IDE CLI, and the web UI.
- The architecture is intentionally “dual-plane”: a registry plane for agent definitions and a telemetry/observability plane for sessions, traces, scores, and reports.
2. Backend server is the system hub
- The main backend lives in observal-server/ and is a Python 3.11+ FastAPI application with Strawberry GraphQL layered on top.
- Route structure is split by domain: agents, components, auth, ingest, sessions, telemetry, eval, review, insights, admin, sandbox, MCP, and more.
- Cross-cutting middleware covers request IDs, content-type handling, rate limiting, sanitization, and SSRF/security guards.
3. Persistent storage is split by workload
- PostgreSQL stores durable product data: users, orgs, agents, components, versions, auth config, reviews, alerts, prompts, sandboxes, and enterprise settings.
- ClickHouse stores high-volume telemetry: traces, sessions, spans, token usage, dashboards, and insight/eval analytics.
- Redis is used for cache, locks, and async job coordination; the repo includes arq-style worker processing.
4. Telemetry ingestion pipeline
- The ingest boundary is exposed by backend routes and is fed by the CLI/IDE hooks and shim integrations.
- Incoming session data is parsed by dedicated session parsers (claude_code, kiro, generic ingest classification) and then normalized into session/trace models.
- Telemetry flows into ClickHouse for querying by dashboard, trace detail, errors, and insights pages.
5. Registry and versioning subsystem
- The registry models an agent as a bundle of config: MCP servers, skills, hooks, prompts, sandboxes, model choices, and IDE-specific metadata.
- Agents and components are versioned separately; the server has resolvers/builders/generators for producing installable configs and lock files.
- A review workflow exists for submissions, drafts, and approvals, which is surfaced in admin UI and enforced in server routes/services.
6. Analytics, eval, and insights are a separate processing layer
- The services/eval/ package implements the scoring pipeline: sanitization, canary/adversarial checks, structural scoring, aggregation, and watchdog/control logic.
- services/insights/ plus ee/observal_insights/ generate higher-level reports from sessions, including deduplication, facets, enrichment, narrative summaries, and HTML export.
- The eval engine can call external LLMs, including AWS Bedrock or OpenAI-compatible endpoints, to score or summarize sessions.
7. CLI is the client-side orchestration layer
- observal_cli/ is a Python/Typer/Rich command-line client used for auth, pulling registry artifacts, reconciling local configs, scanning, doctor/support, sandboxing, and telemetry flushing.
- It also installs and manages IDE hooks, proxy/shim behavior, and session push logic, acting as the bridge between local developer tools and the server.
- This is the main edge boundary where agent activity is captured before it reaches the backend.
8. IDE integrations and local capture
- The CLI contains IDE-specific specs and hooks for Claude Code, Kiro, and others, plus a shim/proxy path for tools that cannot emit native hooks.
- Local capture also includes telemetry buffering and redaction so sensitive payloads can be minimized before upload.
- This makes the CLI a control point for both installation/config generation and runtime data collection.
9. Web app is the operator and analyst surface
- web/ is a Next.js 16 + React 19 application used for registry browsing, agent/component editing, dashboards, insights, review queues, admin pages, and trace inspection.
- It consumes backend APIs via a typed API layer and GraphQL client; route groups separate auth, registry, user, and admin areas.
- The frontend is mostly a presentation/query layer, not a domain engine.
10. Enterprise extensions are isolated
- The ee/ tree adds enterprise-only backend modules: SSO/SAML, SCIM, audit logging, enterprise guards, and additional insights support.
- The separation is architectural as well as licensing-based: base OSS server routes are augmented by enterprise middleware/routes/services where available.
- This is the main boundary for identity lifecycle and enterprise admin features.
11. Deployment topology
- The repo includes a full containerized stack via Docker Compose and dedicated Dockerfiles for API and web.
- Supporting infra includes Nginx, Prometheus, Grafana dashboards, ClickHouse config, and AWS Terraform for self-hosting on ECS/VPC/ALB/Postgres/Redis/S3.
- There is also a one-line install/server-package path that bootstraps a prebuilt stack without cloning the repo.
12. Operational and safety concerns are first-class
- Support bundles, redaction, security-event logging, payload sanitization, and SSRF protection are built into the backend/CLI flow.
- Tests are broad and include backend API, telemetry ingestion, eval pipelines, CLI commands, enterprise auth, and end-to-end frontend flows, reflecting the system’s multiple boundaries.
- For an architecture diagram, the key arrows are: IDE/CLI → backend ingest → ClickHouse/Postgres → web dashboards/eval/insights, with enterprise identity and async workers attached to the backend core.
Main entry points: main.py API server · ingest.py Telemetry routes · clickhouse.py ClickHouse · README.md AWS Terraform
Components 22
Backend core
Python server, data stores, and processing services.
- API server · Main backend entrypoint exposing REST routes, GraphQL, middleware, and security controls.
observal-server/main.py - Registry routes · Agent, component, skill, prompt, MCP, versioning, and review endpoints.
observal-server/api/routes/agent.py - Telemetry routes · Ingest, sessions, telemetry, dashboards, insights, eval, and feedback endpoints.
observal-server/api/routes/ingest.py - Registry services · Agent/component config generation, lockfiles, versioning, snapshots, and registry cache logic.
observal-server/services/agent_builder.py - Ingest pipeline · Session parsing, classification, redaction, and normalization before persistence or analytics.
observal-server/services/session_ingest.py - Eval engine · Sanitization, canary/adversarial checks, structural scoring, aggregation, and watchdog control.
observal-server/services/eval/eval_engine.py - Insights engine · Batch deduplication, enrichment, narrative synthesis, and report assembly.
observal-server/services/insights/batch.py - Postgres · Durable product data for users, orgs, agents, components, reviews, prompts, sandboxes, and config.
observal-server/database.py - ClickHouse · High-volume telemetry and analytics store for sessions, traces, spans, token usage, and dashboards.
observal-server/services/clickhouse.py - Redis · Cache, locks, and async coordination backing background processing.
observal-server/services/redis.py - Worker · Background job runner for queued processing and delayed maintenance tasks.
observal-server/worker.py
Clients
CLI capture layer and web operator surface.
- CLI client · Local control plane for auth, pull, reconcile, scan, sandbox, support, and telemetry flush.
observal_cli/main.py - IDE hooks · IDE-specific hooks, session push logic, and shim/proxy behavior for local capture.
observal_cli/hooks/session_push.py - Telemetry buffer · Local buffering, redaction, and upload staging before telemetry reaches the backend.
observal_cli/telemetry_buffer.py - Web app · Operator-facing frontend for registry browsing, editing, dashboards, traces, review, and admin workflows.
web/src/app/layout.tsx - Registry UI · Agent and component browsing, editing, install dialogs, pull commands, and submission/review interactions.
web/src/components/registry/agent-edit-form.tsx - Analytics UI · Dashboards, traces, insights, eval views, and admin reporting surfaces.
web/src/components/dashboard/dashboard-card.tsx
Enterprise
Enterprise-only extensions for identity and analytics.
- Enterprise identity · Enterprise-only SSO, SAML, SCIM, audit logging, and guard middleware layered onto the base server.
ee/observal_server/routes/sso_saml.py - EE insights · Enterprise extensions for anonymization, enrichment, cross-user analysis, and HTML exports.
ee/observal_insights/generator.py
Infra & deploy
Containerization, observability, and AWS deployment assets.
- Docker stack · Containerized API, web, Nginx, ClickHouse, and supporting service definitions for self-hosting.
docker/docker-compose.yml - Grafana dashboards · Prebuilt dashboards and datasource provisioning for operational and product telemetry.
grafana/dashboards/session-overview.json - AWS Terraform · Terraform modules for ECS, ALB, VPC, Postgres, Redis, ClickHouse, S3, DNS, and secrets.
infra/terraform/aws/README.md
Connections 30
- IDE hooks → Telemetry buffer: buffers · Local capture is redacted and staged before upload.
- CLI client → IDE hooks: installs · CLI manages IDE hooks and shim/proxy capture paths.
- CLI client → API server: auth/pull · CLI talks to the backend for login, registry sync, and control operations.
- CLI client → Telemetry routes: pushes · CLI flushes session telemetry and support bundles into ingest endpoints.
- API server → Registry routes: serves · Backend exposes registry and versioning APIs.
- API server → Telemetry routes: serves · Backend exposes ingest, session, telemetry, insights, and eval APIs.
- Registry routes → Registry services: uses · Registry endpoints rely on builders, resolvers, and lockfile generators.
- Registry routes → Postgres: reads/writes · Durable registry data is persisted in Postgres.
- Telemetry routes → Ingest pipeline: normalizes · Incoming telemetry is parsed and converted to internal session models.
- Telemetry routes → Eval engine: scores · Evaluation endpoints feed scoring and aggregation pipelines.
- Telemetry routes → Insights engine: summarizes · Insights endpoints drive report generation and enrichment.
- Ingest pipeline → ClickHouse: writes · Normalized telemetry is stored for analytics and traces.
- Insights engine → ClickHouse: queries · Insights are derived from telemetry and session data in ClickHouse.
- Eval engine → ClickHouse: queries · Eval scoring reads telemetry and session artifacts from ClickHouse.
- Registry services → Postgres: persists · Registry state, versions, and submissions live in Postgres.
- API server → Redis: uses · API uses Redis for cache, locks, and coordination.
- API server → Worker: queues · Backend hands off async jobs to worker processing.
- API server → Enterprise identity: extends · Enterprise routes and guards augment the base server.
- Insights engine → EE insights: extends · Enterprise insights adds anonymization, cross-user analysis, and exports.
- Web app → API server: consumes · Frontend queries backend APIs and GraphQL for data and mutations.
- Registry UI → Registry routes: renders · Registry UI is driven by registry and versioning endpoints.
- Analytics UI → Telemetry routes: renders · Analytics UI is driven by telemetry, dashboards, insights, and eval endpoints.
- Docker stack → API server: runs · Compose stack deploys the backend service.
- Docker stack → Web app: runs · Compose stack deploys the web application.
- Docker stack → ClickHouse: includes · Stack includes ClickHouse for telemetry analytics.
- Grafana dashboards → ClickHouse: visualizes · Dashboards read analytics data from ClickHouse.
- AWS Terraform → Docker stack: mirrors · Terraform models the same self-hosted topology in AWS.
- AWS Terraform → Postgres: provisions · Infra defines managed Postgres for durable data.
- AWS Terraform → Redis: provisions · Infra defines Redis for cache and coordination.
- AWS Terraform → ClickHouse: provisions · Infra defines ClickHouse for telemetry analytics.