# katanemo/plano architecture

> Architecture diagram of the GitHub repository katanemo/plano, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-07-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/katanemo/plano
- Repository: https://github.com/katanemo/plano

## Overview

##### Rust dataplane core
The primary production system lives under `crates/`. The main executable is `crates/brightstaff`, which appears to be the server/runtime that accepts traffic, applies routing/filtering/state, and emits observability data. Supporting crates split reusable protocol logic and policy logic:
- `crates/common` for shared config, routing, HTTP, tokenization, PII, rate limiting, tracing, and provider abstractions.
- `crates/hermesllm` for provider adapters and request/response transforms.
- `crates/llm_gateway` for model-routing gateway behavior.
- `crates/prompt_gateway` for prompt/filter/tool orchestration behavior.

##### Request flow through the proxy
The runtime is organized around HTTP request handling and streaming responses. Incoming traffic is routed through `router/`, then into `handlers/` for specific features such as agents, LLM handling, function calling, model selection, and debug endpoints. This suggests a single dataplane that can act as:
- an agent orchestrator,
- a model gateway,
- or a filter/guardrail pipeline,
depending on configuration.

##### Agent orchestration subsystem
The `handlers/agents` and `router/orchestrator*` code indicates first-class multi-agent routing. Agent requests are selected and dispatched based on descriptions or routing logic, with orchestrator/pipeline/selector components separating:
- intent or agent selection,
- orchestration of one or more downstream agents,
- and JSON-RPC-style or OpenAI-compatible request/response handling.
This is the boundary where Plano coordinates multiple agent services without app code needing explicit routing logic.

##### LLM routing gateway
`crates/llm_gateway` and the corresponding handlers under `brightstaff/src/handlers/llm` indicate a dedicated model-routing path. The repo supports routing by explicit model name, aliases, and preference-based selection. This is distinct from agent orchestration: here the proxy decides *which model/provider to use* for a request, while preserving a unified API surface for clients.

##### Prompt gateway and filter-chain behavior
`crates/prompt_gateway` plus the `demos/filter_chains` material show a separate concern: applying guardrails, context building, memory hooks, tool invocation, and request/response filtering around prompts. The filter chain is not just a simple middleware; it is a structured pipeline with HTTP context, stream context, filter context, and tool/context helpers. This is the repository’s safety/moderation/memory boundary.

##### Signals, tracing, and observability
Observability is a first-class subsystem in Rust under `brightstaff/src/signals`, `metrics`, and `tracing`, plus shared trace utilities in `common/src/traces`. The codebase treats “signals” as domain-specific analytical events beyond raw logs/traces, with analyzers for interaction, execution, and environment conditions. There is also PostHog and OpenTelemetry export plumbing, so requests can be observed end-to-end without instrumenting each agent manually.

##### State and session persistence
Session or conversation state is abstracted in `brightstaff/src/state` and `session_cache`, with in-memory and Redis/PostgreSQL backends. This is important because several features in the repo rely on preserving conversational or response state across turns, especially for orchestrated or responses-style APIs. State management is a separate data boundary from routing and from provider communication.

##### Python CLI and operational tooling
The `cli/planoai` package is the operator-facing control plane. It handles init/config generation, template sync, native binary launching, Docker-based workflows, tracing helpers, and observability/reporting commands. It also bundles template YAMLs for common patterns. In architecture terms, the CLI prepares and launches the runtime rather than serving end-user traffic itself.

##### Configuration, templates, and deployment surfaces
Runtime behavior is driven by YAML configs in `config/` and many demo-specific configs. There are clear deployment artifacts for Docker Compose, Envoy templates, Prometheus/Grafana dashboards, and supervised local environments. This indicates the dataplane is meant to be deployed as an infrastructure component, not embedded in application code. The `config/envoy.template.yaml` and related files suggest Envoy is part of the external proxy/deployment model.

##### Docs site and product site
There are two Next.js apps under `apps/`: `apps/www` for the main documentation/product site and `apps/katanemo-www` for the marketing site. The docs app also includes a Sanity studio route, blog content, SEO endpoints, and a lot of explainer UI around routing, observability, and concepts. These are not part of the runtime dataplane, but they document and manage the product’s architecture and examples.

##### Examples, demos, and verification
`demos/` is large and intentionally mirrors the major architectural modes: agent orchestration, filter chains, LLM routing, provider integrations, and getting started examples. `tests/` includes integration, end-to-end, REST/Hurl, and parity tests that validate the proxy behavior against model routing and prompt gateway scenarios. This test structure reinforces the repo’s boundaries by exercising each subsystem as an externally visible service.

##### Overall architectural picture
The repo’s central idea is a configurable AI middleware layer that sits between client applications and downstream LLMs/agents. The Rust runtime handles request interception, routing, filtering, state, and observability; `hermesllm` adapts provider APIs; the Python CLI and YAML configs define deployment/runtime behavior; and the web/docs/demo assets demonstrate how to compose these pieces into agentic applications.

Main entry points: [`crates/brightstaff/src/main.rs`](https://github.com/katanemo/plano/blob/main/crates/brightstaff/src/main.rs) (Brightstaff), [`crates/brightstaff/src/handlers`](https://github.com/katanemo/plano/tree/main/crates/brightstaff/src/handlers) (Handlers), [`cli/planoai/main.py`](https://github.com/katanemo/plano/blob/main/cli/planoai/main.py) (Plano CLI), [`crates/common/src`](https://github.com/katanemo/plano/tree/main/crates/common/src) (Common core)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_control_plane["Control plane"]
  node_cli_main["Plano CLI<br/>python operator<br/>[main.py]"]
  node_cli_templates["CLI templates<br/>yaml presets"]
  node_config_bundle["Config bundle<br/>deployment config"]
end

subgraph group_runtime["Rust runtime"]
  node_brightstaff_main["Brightstaff<br/>rust server<br/>[main.rs]"]
  node_router_core["Router core<br/>request routing"]
  node_handlers_core["Handlers"]
  node_agents_orchestration["Agent orchestration<br/>multi-agent dispatch"]
  node_llm_handler["LLM handler<br/>model gateway"]
  node_state_store[("State store<br/>session persistence")]
  node_session_cache[("Session cache<br/>turn cache")]
  node_signals_obs(("Signals<br/>domain telemetry"))
  node_tracing_obs(("Tracing<br/>otel/posthog"))
  node_common_lib["Common core<br/>shared rust lib"]
  node_hermesllm["HermesLLM<br/>provider adapter"]
  node_llm_gateway["LLM gateway<br/>routing lib"]
  node_prompt_gateway["Prompt gateway<br/>filter pipeline"]
end

subgraph group_experience["Docs and demos"]
  node_docs_site["Docs site<br/>nextjs docs"]
  node_marketing_site["Marketing site<br/>nextjs site"]
  node_demos_bundle["Demos<br/>example suite"]
  node_docs_skills["Skills<br/>guides"]
end

subgraph group_verification["Verification"]
  node_tests_suite["Tests<br/>validation suite"]
end

node_cli_main -->|"generates"| node_cli_templates
node_cli_main -->|"writes"| node_config_bundle
node_cli_main -->|"launches"| node_brightstaff_main
node_brightstaff_main -->|"routes through"| node_router_core
node_router_core -->|"dispatches"| node_handlers_core
node_handlers_core -->|"includes"| node_agents_orchestration
node_handlers_core -->|"includes"| node_llm_handler
node_handlers_core -->|"includes"| node_prompt_gateway
node_agents_orchestration -->|"uses"| node_common_lib
node_llm_handler -->|"adapts via"| node_hermesllm
node_llm_handler -->|"routes via"| node_llm_gateway
node_prompt_gateway -->|"uses"| node_common_lib
node_brightstaff_main -->|"persists"| node_state_store
node_brightstaff_main -->|"caches"| node_session_cache
node_brightstaff_main -->|"emits"| node_signals_obs
node_brightstaff_main -->|"exports"| node_tracing_obs
node_brightstaff_main -->|"shares"| node_common_lib
node_hermesllm -->|"shares"| node_common_lib
node_docs_site -.->|"documents"| node_demos_bundle
node_docs_site -.->|"explains"| node_config_bundle
node_demos_bundle -->|"exercises"| node_brightstaff_main
node_tests_suite -->|"validates"| node_brightstaff_main
node_tests_suite -->|"validates"| node_prompt_gateway
node_tests_suite -->|"validates"| node_llm_gateway
node_docs_skills -.->|"guides"| node_cli_main
node_docs_skills -.->|"guides"| node_config_bundle

click node_cli_main "https://github.com/katanemo/plano/blob/main/cli/planoai/main.py"
click node_cli_templates "https://github.com/katanemo/plano/tree/main/cli/planoai/templates"
click node_config_bundle "https://github.com/katanemo/plano/tree/main/config"
click node_brightstaff_main "https://github.com/katanemo/plano/blob/main/crates/brightstaff/src/main.rs"
click node_router_core "https://github.com/katanemo/plano/tree/main/crates/brightstaff/src/router"
click node_handlers_core "https://github.com/katanemo/plano/tree/main/crates/brightstaff/src/handlers"
click node_agents_orchestration "https://github.com/katanemo/plano/tree/main/crates/brightstaff/src/handlers/agents"
click node_llm_handler "https://github.com/katanemo/plano/tree/main/crates/brightstaff/src/handlers/llm"
click node_state_store "https://github.com/katanemo/plano/tree/main/crates/brightstaff/src/state"
click node_session_cache "https://github.com/katanemo/plano/tree/main/crates/brightstaff/src/session_cache"
click node_signals_obs "https://github.com/katanemo/plano/tree/main/crates/brightstaff/src/signals"
click node_tracing_obs "https://github.com/katanemo/plano/tree/main/crates/brightstaff/src/tracing"
click node_common_lib "https://github.com/katanemo/plano/tree/main/crates/common/src"
click node_hermesllm "https://github.com/katanemo/plano/tree/main/crates/hermesllm/src"
click node_llm_gateway "https://github.com/katanemo/plano/tree/main/crates/llm_gateway/src"
click node_prompt_gateway "https://github.com/katanemo/plano/tree/main/crates/prompt_gateway/src"
click node_docs_site "https://github.com/katanemo/plano/tree/main/apps/www/src"
click node_marketing_site "https://github.com/katanemo/plano/tree/main/apps/katanemo-www/src"
click node_demos_bundle "https://github.com/katanemo/plano/tree/main/demos"
click node_tests_suite "https://github.com/katanemo/plano/tree/main/tests"
click node_docs_skills "https://github.com/katanemo/plano/tree/main/skills"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli_main,node_cli_templates,node_config_bundle toneBlue
class node_brightstaff_main,node_router_core,node_handlers_core,node_agents_orchestration,node_llm_handler,node_state_store,node_session_cache,node_signals_obs,node_tracing_obs,node_common_lib,node_hermesllm,node_llm_gateway,node_prompt_gateway toneAmber
class node_docs_site,node_marketing_site,node_demos_bundle,node_docs_skills toneMint
class node_tests_suite toneRose
```

## Components

### Control plane

Operator tooling, configuration, and deployment surfaces.

- [Plano CLI](https://github.com/katanemo/plano/blob/main/cli/planoai/main.py): python operator · Python command-line tool for init, config generation, launch, tracing, and observability workflows. · `cli/planoai/main.py`
- [CLI templates](https://github.com/katanemo/plano/tree/main/cli/planoai/templates): yaml presets · Opinionated runtime templates used by the operator to generate ready-made Plano configs. · `cli/planoai/templates`
- [Config bundle](https://github.com/katanemo/plano/tree/main/config): deployment config · YAML schema, environment, and local deployment templates that shape runtime behavior. · `config`

### Rust runtime

Dataplane server and shared protocol logic.

- [Brightstaff](https://github.com/katanemo/plano/blob/main/crates/brightstaff/src/main.rs): rust server · Primary Rust dataplane executable that receives requests and dispatches them through routing, handlers, state, and observability. · `crates/brightstaff/src/main.rs`
- [Router core](https://github.com/katanemo/plano/tree/main/crates/brightstaff/src/router): request routing · HTTP and orchestrator routing layer that selects request paths and model/orchestrator flows. · `crates/brightstaff/src/router`
- [Handlers](https://github.com/katanemo/plano/tree/main/crates/brightstaff/src/handlers): request handlers · Feature handlers for agents, LLMs, model selection, function calling, debug, and response flows. · `crates/brightstaff/src/handlers`
- [Agent orchestration](https://github.com/katanemo/plano/tree/main/crates/brightstaff/src/handlers/agents): multi-agent dispatch · Agent selection and orchestration pipeline, including selector, pipeline, and JSON-RPC handling. · `crates/brightstaff/src/handlers/agents`
- [LLM handler](https://github.com/katanemo/plano/tree/main/crates/brightstaff/src/handlers/llm): model gateway · LLM request handling and model selection path for unified client-facing model APIs. · `crates/brightstaff/src/handlers/llm`
- [State store](https://github.com/katanemo/plano/tree/main/crates/brightstaff/src/state): session persistence · Conversation and response state persistence with memory and Redis/PostgreSQL backends. · `crates/brightstaff/src/state`
- [Session cache](https://github.com/katanemo/plano/tree/main/crates/brightstaff/src/session_cache): turn cache · In-process and Redis-backed cache for conversational/session continuity. · `crates/brightstaff/src/session_cache`
- [Signals](https://github.com/katanemo/plano/tree/main/crates/brightstaff/src/signals): domain telemetry · Domain-specific analyzers and event schemas for interaction, execution, and environment signals. · `crates/brightstaff/src/signals`
- [Tracing](https://github.com/katanemo/plano/tree/main/crates/brightstaff/src/tracing): otel/posthog · Trace initialization and exporters that connect runtime activity to observability backends. · `crates/brightstaff/src/tracing`
- [Common core](https://github.com/katanemo/plano/tree/main/crates/common/src): shared rust lib · Shared config, HTTP, routing, rate limiting, tokenization, PII, traces, and provider abstractions. · `crates/common/src`
- [HermesLLM](https://github.com/katanemo/plano/tree/main/crates/hermesllm/src): provider adapter · Provider normalization layer for OpenAI, Anthropic, Bedrock, responses APIs, and streaming transforms. · `crates/hermesllm/src`
- [LLM gateway](https://github.com/katanemo/plano/tree/main/crates/llm_gateway/src): routing lib · Model-routing gateway logic and filter/stream contexts used by the runtime and demos. · `crates/llm_gateway/src`
- [Prompt gateway](https://github.com/katanemo/plano/tree/main/crates/prompt_gateway/src): filter pipeline · Structured prompt/filter pipeline with HTTP context, stream context, tools, and filter orchestration. · `crates/prompt_gateway/src`

### Docs and demos

Documentation site, marketing site, and executable examples.

- [Docs site](https://github.com/katanemo/plano/tree/main/apps/www/src): nextjs docs · Primary documentation and product site with concepts, guides, studio, and API content. · `apps/www/src`
- [Marketing site](https://github.com/katanemo/plano/tree/main/apps/katanemo-www/src): nextjs site · Secondary web app for the public-facing product/brand site. · `apps/katanemo-www/src`
- [Demos](https://github.com/katanemo/plano/tree/main/demos): example suite · Executable examples covering agent orchestration, filter chains, LLM routing, and integrations. · `demos`
- [Skills](https://github.com/katanemo/plano/tree/main/skills): guides · Operational and pattern-oriented skill docs that encode usage, deployment, and configuration recipes. · `skills`

### Verification

Tests and parity checks that exercise the proxy behaviors.

- [Tests](https://github.com/katanemo/plano/tree/main/tests): validation suite · End-to-end, Hurl, REST, and parity tests that validate gateway behavior externally. · `tests`

## Connections

- Plano CLI → CLI templates: generates · CLI uses templates to scaffold common Plano configurations.
- Plano CLI → Config bundle: writes · CLI creates and validates runtime configuration and deployment files.
- Plano CLI → Brightstaff: launches · Operator starts the Rust runtime locally or in containers.
- Brightstaff → Router core: routes through · Server entrypoint forwards requests into the router layer.
- Router core → Handlers: dispatches · Router selects the appropriate feature handler for each request.
- Handlers → Agent orchestration: includes · Agent-specific paths are handled by the orchestration subsystem.
- Handlers → LLM handler: includes · Model-centric paths are handled by the LLM gateway handler.
- Handlers → Prompt gateway: includes · Prompt/filter flows are processed through the prompt gateway pipeline.
- Agent orchestration → Common core: uses · Agent flows share routing, HTTP, and provider primitives.
- LLM handler → HermesLLM: adapts via · LLM requests and responses are normalized through provider adapters.
- LLM handler → LLM gateway: routes via · Model selection and alias handling are delegated to the gateway library.
- Prompt gateway → Common core: uses · Filter-chain components reuse shared context and utility code.
- Brightstaff → State store: persists · Runtime stores and reloads conversation or response state.
- Brightstaff → Session cache: caches · Server keeps turn/session data hot for conversational continuity.
- Brightstaff → Signals: emits · Runtime emits domain signals for analysis and telemetry.
- Brightstaff → Tracing: exports · Server initializes tracing and sends spans to observability backends.
- Brightstaff → Common core: shares · Server depends on shared configuration, routing, HTTP, and trace utilities.
- HermesLLM → Common core: shares · Provider adapters reuse shared API and trace helpers.
- Docs site → Demos: documents · Docs site references example architectures and runnable workflows.
- Docs site → Config bundle: explains · Documentation is centered on configuration, deployment, and runtime behavior.
- Demos → Brightstaff: exercises · Demos drive the runtime through its proxy and orchestration modes.
- Tests → Brightstaff: validates · Integration and end-to-end tests exercise the Rust dataplane externally.
- Tests → Prompt gateway: validates · Dedicated tests cover prompt gateway and filter-chain behavior.
- Tests → LLM gateway: validates · Gateway-specific tests verify model routing and selection.
- Skills → Plano CLI: guides · Skill docs encode how to use and operate the CLI.
- Skills → Config bundle: guides · Skill docs explain configuration and deployment patterns.
