# vectorize-io/hindsight-cookbook architecture

> Architecture diagram of the GitHub repository vectorize-io/hindsight-cookbook, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vectorize-io/hindsight-cookbook
- Repository: https://github.com/vectorize-io/hindsight-cookbook

## Overview

1. **Repository shape**
   This is not a single product codebase; it is a **cookbook of standalone demos** centered on the Hindsight memory service. Each folder under `applications/` is an independent example app, while `notebooks/` contains smaller tutorial-style integrations.

2. **Primary architectural boundary: Hindsight as external memory**
   The core dependency across the repo is the **Hindsight API/service** running separately from these examples. Most apps send conversation events, observations, or domain facts to Hindsight for **retain / recall / reflect**-style memory, then read that memory back to influence future responses.

3. **Notebook layer: direct, minimal integrations**
   The Jupyter notebooks are the simplest architecture: a notebook cell drives the whole flow, calling Hindsight and an LLM inline. These are good for diagramming the **request → memory retrieval → model response** loop without extra app layers.

4. **Simple Python agent demos**
   Examples like `agno-memory`, `pydantic-ai-memory`, `smolagents-memory`, `crewai-memory`, `openai-fitness-coach`, and others are **single-process Python agents**. Their structure is usually:
   - one agent script
   - a small `requirements.txt`
   - a memory helper/seed script
   - optional tool wrappers  
   The important boundary is between the agent runtime and the external Hindsight memory store.

5. **Next.js-based chat apps**
   Apps such as `chat-memory` and `chat-memory-cloud` use a typical **Next.js frontend + API route** structure:
   - `app/components/*` handles chat UI
   - `app/api/chat/route.ts` orchestrates requests
   - `lib/hindsight.ts` and `lib/llm.ts` / `lib/groq.ts` isolate model and memory calls  
   The backend logic is intentionally thin; the main flow is UI → API route → LLM + Hindsight → streamed/returned answer.

6. **Multi-platform bot architecture**
   `chat-sdk-multi-platform` extends the same pattern to **Discord + webhook-based platforms**. Instead of a browser chat UI, the ingress points are:
   - Discord gateway route
   - platform webhook route  
   Shared bot/memory logic lives in `lib/bot.ts` and `lib/hindsight.ts`, so the architecture is “multiple event sources, one shared conversation/memory core.”

7. **Full-stack Python backend + TypeScript frontend demos**
   `cable-co`, `claims-iq`, and `deliveryman-demo` use a more explicit separation:
   - a **Python backend** under `backend/app`
   - a **TypeScript/React frontend** under `frontend`
   - WebSocket hooks for live state updates  
   The backend contains service layers such as `agent_service.py` and `memory_service.py`, while the frontend renders operational panels, controls, and live telemetry. This is the most diagram-worthy “UI ↔ backend orchestration ↔ memory” pattern in the repo.

8. **Operational event stream as a first-class UI concern**
   In the live demo apps, the frontend is not just a chat window. Components like control bars, knowledge panels, mental-model panels, activity logs, metrics, and pipeline visualizations indicate a **streaming operational dashboard**. The backend pushes state changes over WebSocket, and the UI reflects the agent’s intermediate reasoning, not only final answers.

9. **Domain-specific pipelines and external data sources**
   `stancetracker` is a richer pipeline app: it combines scheduled scraping, stance extraction, RAG, vectorization, and a database layer. Its architecture includes:
   - API routes for sessions, scheduler, and stance management
   - `lib/` modules for scraping, extraction, scheduling, and vector/search integration
   - SQL schema and DB access under `lib/db`  
   This example is best drawn as an ingestion pipeline feeding a search/memory system, not as a simple chat app.

10. **Content sync / ingestion app**
    `sanity-blog-memory` is a synchronization pipeline rather than an interactive agent. It pulls content from Sanity CMS, transforms it, and syncs it into Hindsight via `src/sync.ts`, `hindsight-client.ts`, and `sanity-client.ts`. The main boundary is **CMS source → sync job → Hindsight memory**.

11. **Benchmarking and experiments as separate lanes**
    `deliveryman-demo` includes benchmark services, configs, tests, and a separate `frontend-benchmark` game-style UI built with Phaser. It also has an `experiments/` folder for retraining and retrieval tests. This means the repo contains both **product demos** and **evaluation harnesses**, which should be diagrammed as separate from runtime user paths.

12. **Cross-cutting local storage and session state**
    Frontends use local stores (`sessionStore`, `claimsStore`, `gameStore`) and hooks (`useWebSocket`, `useToast`) for ephemeral UI state. This local state is intentionally separate from long-term memory, which is handled externally by Hindsight. That distinction is important: **UI state is local and transient; memory is shared and persistent**.

13. **Infrastructure and tooling**
    Most apps are self-contained and bring their own runtime:
    - Python apps: `requirements.txt`, `run.sh`, sometimes FastAPI-style app structure
    - Node/Next apps: `package.json`, `tsconfig`, `next.config.*`
    - React/Vite apps: `frontend/` with Vite config
    - Docker and `.env.example` files where external services are required  
    The repo also has a GitHub Actions workflow to test notebooks, showing that notebooks are treated as part of the supported surface area.

14. **How to draw the architecture at repo level**
    The clearest high-level diagram is:
    - **User interfaces**: notebooks, web UIs, chat UIs, bot endpoints
    - **App orchestration layer**: Python services or Next.js API routes
    - **Agent/model layer**: tool use, retrieval, reasoning, pipelines
    - **Shared memory layer**: Hindsight service
    - **External data/services**: LLM providers, Discord/webhooks, CMS, scraping targets, databases  
    Each example plugs these pieces together differently, but the recurring design is always **event-driven orchestration with Hindsight as the durable memory boundary**.

Main entry points: [`applications/cable-co/backend/app/main.py`](https://github.com/vectorize-io/hindsight-cookbook/blob/main/applications/cable-co/backend/app/main.py) (Backend app), [`applications/chat-sdk-multi-platform/lib/bot.ts`](https://github.com/vectorize-io/hindsight-cookbook/blob/main/applications/chat-sdk-multi-platform/lib/bot.ts) (Bot core), [`applications/stancetracker/lib/stance-pipeline.ts`](https://github.com/vectorize-io/hindsight-cookbook/blob/main/applications/stancetracker/lib/stance-pipeline.ts) (Stance pipeline), [`applications/sanity-blog-memory/src/sync.ts`](https://github.com/vectorize-io/hindsight-cookbook/blob/main/applications/sanity-blog-memory/src/sync.ts) (Content sync)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interactive["Interactive demos"]
  node_notebook["Notebook flow<br/>tutorial loop"]
  node_chatui["Chat UI<br/>next.js ui<br/>[Chat.tsx]"]
  node_chatapi["Chat route<br/>api orchestration<br/>[route.ts]"]
  node_botcore["Bot core<br/>shared bot logic<br/>[bot.ts]"]
  node_discordingress["Discord gateway<br/>platform route<br/>[route.ts]"]
  node_webhookingress["Webhook route<br/>platform route<br/>[route.ts]"]
end

subgraph group_agents["Agent demos"]
  node_pythonagent["Python agent<br/>single-process agent"]
  node_coach["Fitness coach<br/>agent workflow<br/>[openai_coach.py]"]
  node_seed["Seed memory<br/>bootstrap script<br/>[seed_memory.py]"]
end

subgraph group_fullstack["Full-stack apps"]
  node_backend["Backend app<br/>python service<br/>[main.py]"]
  node_frontend["Frontend app<br/>react ui<br/>[App.tsx]"]
  node_ws(("WebSocket sync<br/>live state channel<br/>[manager.py]"))
  node_benchui["Benchmark UI<br/>game ui<br/>[PhaserGame.tsx]"]
end

subgraph group_pipelines["Pipelines"]
  node_survey["Stance pipeline<br/>ingestion pipeline<br/>[stance-pipeline.ts]"]
  node_scraper["Scraper agent<br/>fetch/extract<br/>[scraper-agent.ts]"]
  node_sync["Content sync<br/>cms sync job<br/>[sync.ts]"]
end

subgraph group_infra["Shared services"]
  node_hindsight[("Hindsight<br/>memory service")]
  node_llm(("LLM API<br/>model provider"))
  node_db[("Database<br/>sql/vector store")]
  node_discord(("Discord/Webhooks<br/>event sources"))
  node_cms(("Sanity CMS<br/>content source"))
  node_scrape(("Web sources<br/>ingest targets"))
  node_service["Memory service<br/>go service<br/>[main.go]"]
end

node_notebook -->|"store / recall"| node_hindsight
node_notebook -->|"prompt / answer"| node_llm
node_chatui -->|"send message"| node_chatapi
node_chatapi -->|"retrieve memory"| node_hindsight
node_chatapi -->|"generate reply"| node_llm
node_botcore -->|"persist memory"| node_hindsight
node_discordingress -->|"discord event"| node_botcore
node_webhookingress -->|"platform event"| node_botcore
node_botcore -->|"respond"| node_llm
node_pythonagent -->|"agent memory"| node_hindsight
node_pythonagent -->|"reason / act"| node_llm
node_coach -->|"fitness memory"| node_hindsight
node_coach -->|"coach response"| node_llm
node_seed -->|"preload"| node_hindsight
node_backend -->|"agent memory"| node_hindsight
node_backend -->|"orchestrate agent"| node_llm
node_frontend -->|"live updates"| node_ws
node_ws -->|"stream state"| node_backend
node_backend -.->|"runtime example"| node_service
node_benchui -->|"benchmark events"| node_ws
node_survey -->|"ingest sources"| node_scraper
node_scraper -->|"store facts"| node_hindsight
node_survey -->|"persist / query"| node_db
node_survey -->|"extract stance"| node_llm
node_sync -->|"pull content"| node_cms
node_sync -->|"sync memory"| node_hindsight
node_sync -->|"transform"| node_llm
node_hindsight -.->|"retrieval context"| node_llm

click node_notebook "https://github.com/vectorize-io/hindsight-cookbook/blob/main/notebooks/01-quickstart.ipynb"
click node_chatui "https://github.com/vectorize-io/hindsight-cookbook/blob/main/applications/chat-memory/app/components/Chat.tsx"
click node_chatapi "https://github.com/vectorize-io/hindsight-cookbook/blob/main/applications/chat-memory/app/api/chat/route.ts"
click node_botcore "https://github.com/vectorize-io/hindsight-cookbook/blob/main/applications/chat-sdk-multi-platform/lib/bot.ts"
click node_discordingress "https://github.com/vectorize-io/hindsight-cookbook/blob/main/applications/chat-sdk-multi-platform/app/api/discord/gateway/route.ts"
click node_webhookingress "https://github.com/vectorize-io/hindsight-cookbook/blob/main/applications/chat-sdk-multi-platform/app/api/webhooks/[platform]/route.ts"
click node_pythonagent "https://github.com/vectorize-io/hindsight-cookbook/blob/main/applications/agno-memory/personal_assistant.py"
click node_coach "https://github.com/vectorize-io/hindsight-cookbook/blob/main/applications/openai-fitness-coach/openai_coach.py"
click node_seed "https://github.com/vectorize-io/hindsight-cookbook/blob/main/applications/claude-code-memory/seed_memory.py"
click node_backend "https://github.com/vectorize-io/hindsight-cookbook/blob/main/applications/cable-co/backend/app/main.py"
click node_frontend "https://github.com/vectorize-io/hindsight-cookbook/blob/main/applications/cable-co/frontend/src/App.tsx"
click node_ws "https://github.com/vectorize-io/hindsight-cookbook/blob/main/applications/deliveryman-demo/backend/app/websocket/manager.py"
click node_benchui "https://github.com/vectorize-io/hindsight-cookbook/blob/main/applications/deliveryman-demo/frontend-benchmark/src/game/PhaserGame.tsx"
click node_survey "https://github.com/vectorize-io/hindsight-cookbook/blob/main/applications/stancetracker/lib/stance-pipeline.ts"
click node_scraper "https://github.com/vectorize-io/hindsight-cookbook/blob/main/applications/stancetracker/lib/scraper-agent.ts"
click node_sync "https://github.com/vectorize-io/hindsight-cookbook/blob/main/applications/sanity-blog-memory/src/sync.ts"
click node_service "https://github.com/vectorize-io/hindsight-cookbook/blob/main/applications/go-memory-service/main.go"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_notebook,node_chatui,node_chatapi,node_botcore,node_discordingress,node_webhookingress toneBlue
class node_pythonagent,node_coach,node_seed toneAmber
class node_backend,node_frontend,node_ws,node_benchui toneMint
class node_survey,node_scraper,node_sync toneRose
class node_hindsight,node_llm,node_db,node_discord,node_cms,node_scrape,node_service toneIndigo
```

## Components

### Interactive demos

Notebook and chat-style examples with direct request-response memory loops.

- [Notebook flow](https://github.com/vectorize-io/hindsight-cookbook/blob/main/notebooks/01-quickstart.ipynb): tutorial loop · Inline notebook-driven memory and model calls. · `notebooks/01-quickstart.ipynb`
- [Chat UI](https://github.com/vectorize-io/hindsight-cookbook/blob/main/applications/chat-memory/app/components/Chat.tsx): next.js ui · Browser chat surface for the simple memory chat demos. · `applications/chat-memory/app/components/Chat.tsx`
- [Chat route](https://github.com/vectorize-io/hindsight-cookbook/blob/main/applications/chat-memory/app/api/chat/route.ts): api orchestration · Thin Next.js route that coordinates chat, retrieval, and response generation. · `applications/chat-memory/app/api/chat/route.ts`
- [Bot core](https://github.com/vectorize-io/hindsight-cookbook/blob/main/applications/chat-sdk-multi-platform/lib/bot.ts): shared bot logic · Shared conversation and memory logic for platform bots. · `applications/chat-sdk-multi-platform/lib/bot.ts`
- [Discord gateway](https://github.com/vectorize-io/hindsight-cookbook/blob/main/applications/chat-sdk-multi-platform/app/api/discord/gateway/route.ts): platform route · Discord gateway entry point into the shared bot core. · `applications/chat-sdk-multi-platform/app/api/discord/gateway/route.ts`
- [Webhook route](https://github.com/vectorize-io/hindsight-cookbook/blob/main/applications/chat-sdk-multi-platform/app/api/webhooks/[platform]/route.ts): platform route · Webhook-based ingress for additional platforms. · `applications/chat-sdk-multi-platform/app/api/webhooks/[platform]/route.ts`

### Agent demos

Single-process Python agent examples that wrap Hindsight memory with an LLM.

- [Python agent](https://github.com/vectorize-io/hindsight-cookbook/blob/main/applications/agno-memory/personal_assistant.py): single-process agent · Representative Python agent demo with direct memory helper integration. · `applications/agno-memory/personal_assistant.py`
- [Fitness coach](https://github.com/vectorize-io/hindsight-cookbook/blob/main/applications/openai-fitness-coach/openai_coach.py): agent workflow · OpenAI-based coaching demo that stores and recalls personal memory. · `applications/openai-fitness-coach/openai_coach.py`
- [Seed memory](https://github.com/vectorize-io/hindsight-cookbook/blob/main/applications/claude-code-memory/seed_memory.py): bootstrap script · Memory seeding and reset-style scripts used by agent demos. · `applications/claude-code-memory/seed_memory.py`

### Full-stack apps

Python backend plus TypeScript frontend demos with live operational state.

- [Backend app](https://github.com/vectorize-io/hindsight-cookbook/blob/main/applications/cable-co/backend/app/main.py): python service · Representative Python backend that orchestrates agent state and memory. · `applications/cable-co/backend/app/main.py`
- [Frontend app](https://github.com/vectorize-io/hindsight-cookbook/blob/main/applications/cable-co/frontend/src/App.tsx): react ui · Representative live dashboard frontend for agent demos. · `applications/cable-co/frontend/src/App.tsx`
- [WebSocket sync](https://github.com/vectorize-io/hindsight-cookbook/blob/main/applications/deliveryman-demo/backend/app/websocket/manager.py): live state channel · Realtime event stream between backend and dashboard UI. · `applications/deliveryman-demo/backend/app/websocket/manager.py`
- [Benchmark UI](https://github.com/vectorize-io/hindsight-cookbook/blob/main/applications/deliveryman-demo/frontend-benchmark/src/game/PhaserGame.tsx): game ui · Separate game-style benchmark surface used for evaluation and experiments. · `applications/deliveryman-demo/frontend-benchmark/src/game/PhaserGame.tsx`

### Pipelines

Ingestion, sync, search, and domain-specific processing flows.

- [Stance pipeline](https://github.com/vectorize-io/hindsight-cookbook/blob/main/applications/stancetracker/lib/stance-pipeline.ts): ingestion pipeline · Domain-specific scraping, extraction, RAG, and vectorization pipeline. · `applications/stancetracker/lib/stance-pipeline.ts`
- [Scraper agent](https://github.com/vectorize-io/hindsight-cookbook/blob/main/applications/stancetracker/lib/scraper-agent.ts): fetch/extract · Fetches and parses external sources for stance tracking. · `applications/stancetracker/lib/scraper-agent.ts`
- [Content sync](https://github.com/vectorize-io/hindsight-cookbook/blob/main/applications/sanity-blog-memory/src/sync.ts): cms sync job · Transforms CMS content and syncs it into Hindsight. · `applications/sanity-blog-memory/src/sync.ts`

### Shared services

External memory, model, platform, and data services used across demos.

- **Hindsight**: memory service · External durable memory store used for retain / recall / reflect flows.
- **LLM API**: model provider · External model provider(s) invoked by the demos.
- **Database**: sql/vector store · Persistence layer used by richer pipeline demos.
- **Discord/Webhooks**: event sources · Inbound platform events for the multi-platform bot example.
- **Sanity CMS**: content source · External content source synced into Hindsight.
- **Web sources**: ingest targets · Sites and feeds scraped by pipeline-style demos.
- [Memory service](https://github.com/vectorize-io/hindsight-cookbook/blob/main/applications/go-memory-service/main.go): go service · Standalone Go memory-service runtime used as an implementation example. · `applications/go-memory-service/main.go`

## Connections

- Notebook flow → Hindsight: store / recall · Notebook cells call Hindsight directly in a minimal loop.
- Notebook flow → LLM API: prompt / answer · The notebook drives the model inline with retrieved memory.
- Chat UI → Chat route: send message · Browser chat UI submits user turns to the API route.
- Chat route → Hindsight: retrieve memory · The route looks up relevant memories before generating a reply.
- Chat route → LLM API: generate reply · The route calls the model provider for the final answer.
- Bot core → Hindsight: persist memory · Shared bot logic records conversation state in Hindsight.
- Discord gateway → Bot core: discord event · Discord gateway forwards incoming messages into shared bot logic.
- Webhook route → Bot core: platform event · Webhook routes normalize events before handing off to the bot core.
- Bot core → LLM API: respond · The bot core calls an LLM to compose platform replies.
- Python agent → Hindsight: agent memory · Single-process Python agents use Hindsight as the durable memory boundary.
- Python agent → LLM API: reason / act · The agent runtime queries the model for responses and actions.
- Fitness coach → Hindsight: fitness memory · The coaching demo stores user history and preferences externally.
- Fitness coach → LLM API: coach response · The demo uses the model to produce personalized guidance.
- Seed memory → Hindsight: preload · Seed scripts initialize example memories before the demos run.
- Backend app → Hindsight: agent memory · Python backends manage memory through a dedicated service layer.
- Backend app → LLM API: orchestrate agent · The backend service coordinates model calls and tool use.
- Frontend app → WebSocket sync: live updates · The dashboard consumes streaming state from the backend.
- WebSocket sync → Backend app: stream state · Backend pushes operational events and progress updates to the UI.
- Backend app → Memory service: runtime example · The repo also includes a standalone Go memory-service implementation.
- Benchmark UI → WebSocket sync: benchmark events · The benchmark game UI reads the same realtime event stream pattern.
- Stance pipeline → Scraper agent: ingest sources · The pipeline starts with scraping and source discovery.
- Scraper agent → Hindsight: store facts · Extracted stance data is written into memory-backed storage.
- Stance pipeline → Database: persist / query · The pipeline uses a database for sessions, search, and processed records.
- Stance pipeline → LLM API: extract stance · LLM-assisted extraction is part of the stance pipeline.
- Content sync → Sanity CMS: pull content · The sync job reads source content from Sanity.
- Content sync → Hindsight: sync memory · Transformed content is pushed into Hindsight as durable memory.
- Content sync → LLM API: transform · Content normalization and enrichment may call model services.
- Hindsight → LLM API: retrieval context · Apps use recalled memories as context for model prompts.