# topherchris420/james_library architecture

> Architecture diagram of the GitHub repository topherchris420/james_library, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-04-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/topherchris420/james_library
- Repository: https://github.com/topherchris420/james_library

## Overview

1. **Overall shape**
   - This is a **polyglot assistant platform** centered on “R.A.I.N. Lab” / “James”: a local-first research and automation system with a Rust core, Python orchestration scripts, a TypeScript web UI, firmware for devices, and extensive docs/CI around release and ops.

2. **Primary runtime boundary**
   - The **Rust workspace in `src/` and `crates/`** is the main product engine: it owns agent execution, tool invocation, providers, channel adapters, memory, security, observability, and the gateway/API surface.
   - The **Python layer** (`rain_lab.py`, `james_library/`, many top-level helper scripts) wraps launch, setup, lab workflows, backup/health checks, and higher-level orchestration.
   - The **web app in `web/`** is a separate frontend client that talks to the Rust gateway over API/SSE/WebSocket.

3. **Agent and conversation core**
   - `src/agent/` contains the central agent loop, prompt/history handling, manifest loading, memory loading, tool-call parsing, tool filtering, and execution.
   - This is the decision point where incoming user/channel messages become model prompts, tool calls, and structured responses.
   - `src/commands/`, `src/cli/`, and `src/cli_input.rs` provide local command-line entry points and maintenance operations.

4. **Model/provider layer**
   - `src/providers/` abstracts LLM backends: Anthropic, OpenAI, Gemini, Bedrock, Ollama, OpenRouter, Azure OpenAI, and other compatible/custom providers.
   - `src/auth/` handles provider-specific auth flows, including OAuth/token management.
   - `src/tools/model_routing_config.rs`, `src/tools/model_switch.rs`, and provider traits indicate runtime routing between models and fallbacks rather than a single hardcoded backend.

5. **Channels and external comms**
   - `src/channels/` is the multi-channel ingress/egress layer for Slack, Telegram, Matrix, Discord, WhatsApp, IRC, email, webhooks, and more.
   - Channel-specific state, history, sanitization, session handling, and transcription/TTS hooks live here.
   - The architectural pattern is: channel event in → normalized message → agent/runtime processing → response back through the same or another target channel.

6. **Gateway and client-facing interface**
   - `src/gateway/` exposes the runtime to the web UI and integrations via **HTTP API, WebSocket, SSE, static file serving, pairing, and plugin APIs**.
   - The gateway is the boundary between the internal assistant runtime and external clients like the `web/` app and potentially paired devices.
   - `src/gateway/hardware_context.rs` and `src/nodes/` suggest the gateway also brokers node/device context, not just text chat.

7. **Memory and retrieval**
   - `src/memory/` is a pluggable memory subsystem with backends including **SQLite, Postgres, Qdrant, Mem0, and no-op** implementations.
   - Supporting modules handle chunking, embeddings, vector search, knowledge graph, consolidation, response cache, hygiene, and snapshotting.
   - The flow is typically: agent/tool writes facts or artifacts into memory; later prompts and tools recall from that store to ground answers.

8. **Tools as the action surface**
   - `src/tools/` is a large capability catalog: file operations, shell, browser/web fetch/search, LSP, MCP, cron, Google Workspace, Microsoft 365, Jira, LinkedIn, Notion, weather, screenshot, workspace, and more.
   - Tools are mediated by schemas, discovery, filtering, approvals, and verifiable-intent components, so the system can decide what is safe to execute and what needs gating.
   - This makes the assistant not only conversational but operational.

9. **Security and safety boundaries**
   - `src/security/` includes sandboxing and containment mechanisms such as Docker, Bubblewrap, Firejail, Landlock, OTP/pairing, secret handling, leak detection, policy checks, and prompt/security guards.
   - `src/approval/`, `src/verifiable_intent/`, and parts of `src/sop/` enforce human or policy approval for risky actions.
   - Security is therefore not a single layer; it is woven through tool execution, pairing, channel input, and sandbox selection.

10. **Hardware and peripheral integration**
   - `src/hardware/`, `src/peripherals/`, `src/p2p/`, `src/nodes/`, and `src/tunnel/` show support for physical/edge deployments: serial devices, GPIO, Raspberry Pi, Arduino/Nucleo/Pico/UNO-Q bridges, flashing, and transport abstractions.
   - The `firmware/` and `crates/robot-kit` directories reinforce that some deployments include embedded device control and sensor/actuator I/O.
   - This is a real architectural branch, not an example folder: the runtime can operate as a lab assistant with hardware-backed capabilities.

11. **Observability, cost, and lifecycle**
   - `src/observability/`, `src/cost/`, and `src/health/` provide logging, tracing, metrics, runtime trace capture, and cost tracking.
   - `src/cron/`, `src/heartbeat/`, and `src/daemon/` support background scheduled tasks and long-running services.
   - This indicates the platform is designed as an always-on assistant system with operational telemetry, not a single-shot CLI tool.

12. **Plugin and extension model**
   - `src/plugins/` plus `example-plugin/` and `plugins/logic_prover/` show a plugin architecture, including WASM-based tool/channel extensions.
   - This lets the core runtime load capabilities without baking every integration into the main binary.
   - `src/skillforge/` and `src/skills/` provide a higher-level skill packaging/evaluation layer on top of that extension model.

13. **Python support ecosystem**
   - The Python files at repo root and under `james_library/`, `lab/`, `graph_r1_engine/`, `godot_client/` support lab workflows, research orchestration, memory/ingestion helpers, and companion applications.
   - These are not the primary runtime core, but they are important glue for launchers, experiments, backups, integrations, and automation around the assistant.

14. **Frontends, docs, and release/CI system**
   - `web/` is the interactive UI; `godot_client/` is a separate visualization/client track; `docs/` is unusually large and acts as a first-class part of the product.
   - `.github/workflows/`, `scripts/`, `tests/`, `fuzz/`, and the Docker/Nix/Cargo setup form a heavy release and quality pipeline covering Rust, Python, docs, security, and multi-platform packaging.
   - For an architecture diagram, treat CI/CD, docs generation, and packaging as supporting systems around the runtime rather than part of the request-processing path.

Main entry points: [`src/agent/mod.rs`](https://github.com/topherchris420/james_library/blob/main/src/agent/mod.rs) (Agent core), [`src/tools/mod.rs`](https://github.com/topherchris420/james_library/blob/main/src/tools/mod.rs) (Tool system), [`src/main.rs`](https://github.com/topherchris420/james_library/blob/main/src/main.rs) (Main binary), [`src/gateway/mod.rs`](https://github.com/topherchris420/james_library/blob/main/src/gateway/mod.rs) (Gateway)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_g_rust["Rust runtime"]
  node_n_main(("Main binary<br/>rust entrypoint<br/>[main.rs]"))
  node_n_agent["Agent core<br/>reasoning loop<br/>[mod.rs]"]
  node_n_providers["Providers<br/>llm adapters<br/>[mod.rs]"]
  node_n_channels["Channels<br/>ingress/egress<br/>[mod.rs]"]
  node_n_gateway["Gateway<br/>api surface<br/>[mod.rs]"]
  node_n_tools["Tool system<br/>action surface<br/>[mod.rs]"]
  node_n_memory[("Memory<br/>retrieval store<br/>[mod.rs]")]
  node_n_security{{"Security<br/>safety boundary<br/>[mod.rs]"}}
  node_n_intent{{"Verifiable intent<br/>approval guard<br/>[mod.rs]"}}
  node_n_plugins["Plugins<br/>extension host<br/>[mod.rs]"]
  node_n_obs["Observability<br/>telemetry<br/>[mod.rs]"]
  node_n_cron["Scheduler<br/>background jobs<br/>[mod.rs]"]
  node_n_daemon(("Daemon<br/>service loop<br/>[mod.rs]"))
  node_n_cli["CLI<br/>local control<br/>[mod.rs]"]
end

subgraph group_g_python["Python orchestration"]
  node_n_python["Rain Lab<br/>orchestration<br/>[rain_lab.py]"]
end

subgraph group_g_web["Web client"]
  node_n_web["Web app<br/>ts frontend<br/>[App.tsx]"]
end

subgraph group_g_devices["Devices and firmware"]
  node_n_robot["Robot kit<br/>device runtime<br/>[lib.rs]"]
  node_n_esp32(("ESP32 firmware<br/>embedded target<br/>[main.rs]"))
end

subgraph group_g_ops["Docs and delivery"]
  node_n_docs["Docs home<br/>[README.md]"]
  node_n_ci["CI workflow<br/>delivery pipeline<br/>[ci.yml]"]
end

node_n_main -->|"runs"| node_n_agent
node_n_main -->|"serves"| node_n_channels
node_n_main -->|"exposes"| node_n_gateway
node_n_agent -->|"calls"| node_n_providers
node_n_agent -->|"invokes"| node_n_tools
node_n_agent -->|"reads/writes"| node_n_memory
node_n_agent -->|"checks"| node_n_security
node_n_agent -->|"gates"| node_n_intent
node_n_channels -->|"delivers to"| node_n_agent
node_n_channels -->|"bridges"| node_n_gateway
node_n_gateway -->|"streams to"| node_n_web
node_n_gateway -->|"loads"| node_n_plugins
node_n_tools -->|"sandboxed by"| node_n_security
node_n_tools -->|"approved by"| node_n_intent
node_n_tools -->|"stores/recalls"| node_n_memory
node_n_plugins -->|"extends"| node_n_tools
node_n_obs -->|"observes"| node_n_agent
node_n_cron -->|"scheduled by"| node_n_daemon
node_n_python -->|"orchestrates"| node_n_main
node_n_python -.->|"supports"| node_n_docs
node_n_esp32 -->|"pairs with"| node_n_robot
node_n_docs -.->|"governs"| node_n_ci
node_n_ci -.->|"builds/tests"| node_n_main
node_n_web -->|"consumes"| node_n_gateway

click node_n_main "https://github.com/topherchris420/james_library/blob/main/src/main.rs"
click node_n_agent "https://github.com/topherchris420/james_library/blob/main/src/agent/mod.rs"
click node_n_providers "https://github.com/topherchris420/james_library/blob/main/src/providers/mod.rs"
click node_n_channels "https://github.com/topherchris420/james_library/blob/main/src/channels/mod.rs"
click node_n_gateway "https://github.com/topherchris420/james_library/blob/main/src/gateway/mod.rs"
click node_n_tools "https://github.com/topherchris420/james_library/blob/main/src/tools/mod.rs"
click node_n_memory "https://github.com/topherchris420/james_library/blob/main/src/memory/mod.rs"
click node_n_security "https://github.com/topherchris420/james_library/blob/main/src/security/mod.rs"
click node_n_intent "https://github.com/topherchris420/james_library/blob/main/src/verifiable_intent/mod.rs"
click node_n_plugins "https://github.com/topherchris420/james_library/blob/main/src/plugins/mod.rs"
click node_n_obs "https://github.com/topherchris420/james_library/blob/main/src/observability/mod.rs"
click node_n_cron "https://github.com/topherchris420/james_library/blob/main/src/cron/mod.rs"
click node_n_daemon "https://github.com/topherchris420/james_library/blob/main/src/daemon/mod.rs"
click node_n_cli "https://github.com/topherchris420/james_library/blob/main/src/cli/mod.rs"
click node_n_python "https://github.com/topherchris420/james_library/blob/main/james_library/launcher/rain_lab.py"
click node_n_web "https://github.com/topherchris420/james_library/blob/main/web/src/App.tsx"
click node_n_robot "https://github.com/topherchris420/james_library/blob/main/crates/robot-kit/src/lib.rs"
click node_n_esp32 "https://github.com/topherchris420/james_library/blob/main/firmware/esp32/src/main.rs"
click node_n_docs "https://github.com/topherchris420/james_library/blob/main/docs/README.md"
click node_n_ci "https://github.com/topherchris420/james_library/blob/main/.github/workflows/ci.yml"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_n_main,node_n_agent,node_n_providers,node_n_channels,node_n_gateway,node_n_tools,node_n_memory,node_n_security,node_n_intent,node_n_plugins,node_n_obs,node_n_cron,node_n_daemon,node_n_cli toneBlue
class node_n_python toneAmber
class node_n_web toneMint
class node_n_robot,node_n_esp32 toneRose
class node_n_docs,node_n_ci toneIndigo
```

## Components

### Rust runtime

Core assistant engine and boundaries in the Rust workspace.

- [Main binary](https://github.com/topherchris420/james_library/blob/main/src/main.rs): rust entrypoint · Top-level executable that wires the runtime together. · `src/main.rs`
- [Agent core](https://github.com/topherchris420/james_library/blob/main/src/agent/mod.rs): reasoning loop · Conversation loop, prompt assembly, history, manifests, and tool-call execution. · `src/agent/mod.rs`
- [Providers](https://github.com/topherchris420/james_library/blob/main/src/providers/mod.rs): llm adapters · LLM backend adapters, routing, compatibility layers, and provider auth boundaries. · `src/providers/mod.rs`
- [Channels](https://github.com/topherchris420/james_library/blob/main/src/channels/mod.rs): ingress/egress · Multi-channel message adapters for chat, social, email, and webhook transport. · `src/channels/mod.rs`
- [Gateway](https://github.com/topherchris420/james_library/blob/main/src/gateway/mod.rs): api surface · HTTP, WebSocket, SSE, static serving, pairing, and plugin-facing endpoints. · `src/gateway/mod.rs`
- [Tool system](https://github.com/topherchris420/james_library/blob/main/src/tools/mod.rs): action surface · Capability catalog plus discovery, schemas, approvals, and execution mediation. · `src/tools/mod.rs`
- [Memory](https://github.com/topherchris420/james_library/blob/main/src/memory/mod.rs): retrieval store · Pluggable persistence for facts, embeddings, vector search, consolidation, and cache. · `src/memory/mod.rs`
- [Security](https://github.com/topherchris420/james_library/blob/main/src/security/mod.rs): safety boundary · Sandboxing, policy checks, secrets handling, pairing, and leak detection. · `src/security/mod.rs`
- [Verifiable intent](https://github.com/topherchris420/james_library/blob/main/src/verifiable_intent/mod.rs): approval guard · Cryptographic issuance and verification for high-trust or risky actions. · `src/verifiable_intent/mod.rs`
- [Plugins](https://github.com/topherchris420/james_library/blob/main/src/plugins/mod.rs): extension host · WASM plugin host for channel and tool extensions. · `src/plugins/mod.rs`
- [Observability](https://github.com/topherchris420/james_library/blob/main/src/observability/mod.rs): telemetry · Logs, traces, metrics, and runtime trace capture. · `src/observability/mod.rs`
- [Scheduler](https://github.com/topherchris420/james_library/blob/main/src/cron/mod.rs): background jobs · Scheduled tasks, cron storage, and recurring execution support. · `src/cron/mod.rs`
- [Daemon](https://github.com/topherchris420/james_library/blob/main/src/daemon/mod.rs): service loop · Long-running background service and keepalive orchestration. · `src/daemon/mod.rs`
- [CLI](https://github.com/topherchris420/james_library/blob/main/src/cli/mod.rs): local control · Command-line entry points and maintenance operations. · `src/cli/mod.rs`

### Python orchestration

Launchers, lab workflows, and glue scripts around the core runtime.

- [Rain Lab](https://github.com/topherchris420/james_library/blob/main/james_library/launcher/rain_lab.py): orchestration · Python launcher and workflow layer for setup, runs, and lab automation. · `james_library/launcher/rain_lab.py`

### Web client

TypeScript frontend that talks to the gateway APIs and streams.

- [Web app](https://github.com/topherchris420/james_library/blob/main/web/src/App.tsx): ts frontend · TypeScript client UI for chat, config, memory, tools, and ops views. · `web/src/App.tsx`

### Devices and firmware

Embedded targets and robot/device control paths.

- [Robot kit](https://github.com/topherchris420/james_library/blob/main/crates/robot-kit/src/lib.rs): device runtime · Rust robot control library for sensing, driving, and safety. · `crates/robot-kit/src/lib.rs`
- [ESP32 firmware](https://github.com/topherchris420/james_library/blob/main/firmware/esp32/src/main.rs): embedded target · Firmware entrypoint for an embedded device runtime. · `firmware/esp32/src/main.rs`

### Docs and delivery

Documentation, CI, release, and packaging support systems.

- [Docs home](https://github.com/topherchris420/james_library/blob/main/docs/README.md): docs · Primary documentation hub for setup, reference, security, and operations. · `docs/README.md`
- [CI workflow](https://github.com/topherchris420/james_library/blob/main/.github/workflows/ci.yml): delivery pipeline · Representative GitHub Actions workflow for build, test, security, and release automation. · `.github/workflows/ci.yml`

## Connections

- Main binary → Agent core: runs · Executable enters the core agent loop.
- Main binary → Channels: serves · Main runtime binds channel ingress and egress.
- Main binary → Gateway: exposes · Main runtime starts the client-facing gateway.
- Agent core → Providers: calls · Agent delegates model requests to provider adapters.
- Agent core → Tool system: invokes · Agent executes structured tool actions.
- Agent core → Memory: reads/writes · Agent grounds responses in persistent memory.
- Agent core → Security: checks · Agent path is filtered by safety and approval controls.
- Agent core → Verifiable intent: gates · Risky actions can require verifiable intent.
- Channels → Agent core: delivers to · Normalized channel events are handed to the agent.
- Channels → Gateway: bridges · Some channels and web flows are coordinated through the gateway.
- Gateway → Web app: streams to · Gateway serves the web client over API/SSE/WebSocket.
- Gateway → Plugins: loads · Gateway exposes plugin-facing APIs.
- Tool system → Security: sandboxed by · Tool execution is constrained by security policy and sandboxing.
- Tool system → Verifiable intent: approved by · Selected tools/actions require verifiable intent or human approval.
- Tool system → Memory: stores/recalls · Memory tools connect the action surface to persistence.
- Plugins → Tool system: extends · Plugins can add tools to the capability catalog.
- Observability → Agent core: observes · Telemetry covers core runtime behavior.
- Scheduler → Daemon: scheduled by · Background scheduler runs inside or alongside daemon mode.
- Rain Lab → Main binary: orchestrates · Python launcher/scripts prepare and start the Rust runtime.
- Rain Lab → Docs home: supports · Python workflows are documented and guided by repo docs.
- ESP32 firmware → Robot kit: pairs with · Embedded firmware complements the robot/device library.
- Docs home → CI workflow: governs · Docs and release process are enforced through CI workflows.
- CI workflow → Main binary: builds/tests · CI validates the main Rust runtime.
- Web app → Gateway: consumes · Frontend talks to the gateway APIs and streams.
