# devanathandna/vegapunk-multi-tab-persistent-memory-conversational-ai architecture

> Architecture diagram of the GitHub repository devanathandna/vegapunk-multi-tab-persistent-memory-conversational-ai, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/devanathandna/vegapunk-multi-tab-persistent-memory-conversational-ai
- Repository: https://github.com/devanathandna/vegapunk-multi-tab-persistent-memory-conversational-ai

## Overview

1. **Overall shape:** VEGAPUNK is a **client-server, local-first AI chat system**. The browser UI is a React/Vite frontend, and all AI orchestration, memory persistence, and retrieval happen in the Python backend. The main external dependency is the **Google Gemini API**; the rest of the system relies on local stores.

2. **Frontend boundary:** `src/` contains a TypeScript React app with a small page layer (`Home`, `VegaChat`, `NotFound`) and chat-focused components (`MainChat`, `SubChat`, `ChatInput`, `ChatMessage`, sidebars). The UI is mostly presentation and chat interaction; it does not own retrieval or memory logic. The many `src/components/ui/*` files are a reusable shadcn/ui layer for dialogs, inputs, tabs, sheets, etc.

3. **Backend API boundary:** `backend/app.py` is the HTTP entry point and request router. It exposes the chat service to the frontend and delegates actual reasoning to the agent layer. The backend is a Flask REST API, so the diagram should show a clear network boundary between React UI and Flask service.

4. **Orchestration core:** `backend/agent.py` is the central runtime coordinator. It handles message intake, query classification, tool/LLM decision-making, retrieval fan-out, answer synthesis, and background memory updates. This is the most important internal subsystem to place at the center of the backend.

5. **Hybrid memory architecture:** The system fuses three distinct memory stores:
   - `backend/database.py` for **episodic memory** in SQLite,
   - `backend/vector_store.py` for **semantic memory** via embeddings/vector search,
   - `backend/graph_store.py` for **structural memory** using a knowledge graph.
   These are independent retrieval paths, not a single monolithic database.

6. **Episodic layer:** SQLite stores the explicit conversation timeline: sessions, messages, timestamps, and other raw conversational records. This is the “what was said, and when” layer, used when recency and exact history matter more than conceptual similarity.

7. **Semantic layer:** `vector_store.py` represents the fuzzy recall path. Messages are embedded and indexed for similarity search, allowing the agent to retrieve related past content even when the wording differs. In the architecture diagram, this is the concept-level memory branch.

8. **Structural layer:** `graph_store.py` plus `entity_extractor.py` form the Graph RAG branch. The backend extracts entities and relationships from messages, then persists them into a weighted knowledge graph. This supports multi-hop reasoning and relationship-based recall that vector search alone would miss.

9. **Fusion and ranking:** `backend/fusion_layer.py` combines candidates from the three memory branches into a final relevance score using semantic, structural, and recency components. This layer is the ranking boundary between retrieval and generation; it determines what context the LLM actually sees.

10. **Request lifecycle:** A chat turn flows from the UI into Flask, then into the agent, which classifies the query and may trigger **parallel retrieval** across SQLite, vector search, and graph search. The fused top results are then sent to Gemini for final synthesis. Afterward, background entity extraction updates the vector store and graph so the next turn can use the newly learned context.

11. **Main chat vs sub chat:** The frontend distinguishes a persistent **Main Chat** from **Sub Chat**. Main Chat contributes to long-term memory, while Sub Chat is an isolated sandbox for exploratory or debugging conversations that should not contaminate the shared memory stores. That is an important architectural boundary for understanding data persistence.

12. **Operational concerns:** The backend is designed around **multiple Gemini API keys** for classification, synthesis, and background extraction, reducing rate-limit pressure on the main response path. The repo also includes test scripts (`test_api.py`, `test_graph_rag.py`, `test_gemini.py`, etc.) and documentation diagrams under `docs/`, which are useful for validating and visualizing the architecture but are not runtime subsystems themselves.

Main entry points: [`backend/agent.py`](https://github.com/devanathandna/vegapunk-multi-tab-persistent-memory-conversational-ai/blob/main/backend/agent.py) (Agent core), [`src/components/MainChat.tsx`](https://github.com/devanathandna/vegapunk-multi-tab-persistent-memory-conversational-ai/blob/main/src/components/MainChat.tsx) (Chat UI), [`backend/fusion_layer.py`](https://github.com/devanathandna/vegapunk-multi-tab-persistent-memory-conversational-ai/blob/main/backend/fusion_layer.py) (Fusion layer), [`backend/entity_extractor.py`](https://github.com/devanathandna/vegapunk-multi-tab-persistent-memory-conversational-ai/blob/main/backend/entity_extractor.py) (Entity extractor)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_frontend["Frontend"]
  node_src_main["React app<br/>Vite client<br/>[main.tsx]"]
  node_src_pages["Pages<br/>UI routes<br/>[VegaChat.tsx]"]
  node_src_chat["Chat UI<br/>components<br/>[MainChat.tsx]"]
  node_src_sidebar["Sidebars<br/>navigation<br/>[Sidebar.tsx]"]
  node_main_chat(("Main chat<br/>persistent thread<br/>[MainChat.tsx]"))
  node_sub_chat(("Sub chat<br/>sandbox thread<br/>[SubChat.tsx]"))
end

subgraph group_backend["Backend"]
  node_backend_app["Flask API<br/>HTTP entry<br/>[app.py]"]
  node_backend_agent["Agent core<br/>orchestrator<br/>[agent.py]"]
  node_backend_fusion["Fusion layer<br/>ranker<br/>[fusion_layer.py]"]
  node_backend_extractor["Entity extractor<br/>graph prep"]
end

subgraph group_memory["Memory"]
  node_backend_database[("SQLite store<br/>episodic memory<br/>[database.py]")]
  node_backend_vector[("Vector store<br/>semantic memory<br/>[vector_store.py]")]
  node_backend_graph[("Graph store<br/>structural memory<br/>[graph_store.py]")]
end

node_gemini_api{{"Gemini API<br/>LLM service"}}

node_src_main -->|"renders"| node_src_pages
node_src_pages -->|"hosts"| node_src_chat
node_src_pages -->|"includes"| node_src_sidebar
node_src_chat -->|"drives"| node_main_chat
node_src_chat -->|"switches"| node_sub_chat
node_src_chat -->|"chat request"| node_backend_app
node_backend_app -->|"delegates"| node_backend_agent
node_backend_agent -->|"ranks"| node_backend_fusion
node_backend_agent -->|"recalls"| node_backend_database
node_backend_agent -->|"recalls"| node_backend_vector
node_backend_agent -->|"recalls"| node_backend_graph
node_backend_agent -->|"synthesizes"| node_gemini_api
node_backend_agent -->|"updates"| node_backend_extractor
node_backend_extractor -->|"extracts via"| node_gemini_api
node_backend_extractor -->|"indexes"| node_backend_vector
node_backend_extractor -->|"writes"| node_backend_graph
node_backend_fusion -->|"scores"| node_backend_database
node_backend_fusion -->|"scores"| node_backend_vector
node_backend_fusion -->|"scores"| node_backend_graph
node_main_chat -->|"persists"| node_backend_database
node_sub_chat -.->|"isolated from"| node_backend_database

click node_src_main "https://github.com/devanathandna/vegapunk-multi-tab-persistent-memory-conversational-ai/blob/main/src/main.tsx"
click node_src_pages "https://github.com/devanathandna/vegapunk-multi-tab-persistent-memory-conversational-ai/blob/main/src/pages/VegaChat.tsx"
click node_src_chat "https://github.com/devanathandna/vegapunk-multi-tab-persistent-memory-conversational-ai/blob/main/src/components/MainChat.tsx"
click node_src_sidebar "https://github.com/devanathandna/vegapunk-multi-tab-persistent-memory-conversational-ai/blob/main/src/components/Sidebar.tsx"
click node_backend_app "https://github.com/devanathandna/vegapunk-multi-tab-persistent-memory-conversational-ai/blob/main/backend/app.py"
click node_backend_agent "https://github.com/devanathandna/vegapunk-multi-tab-persistent-memory-conversational-ai/blob/main/backend/agent.py"
click node_backend_fusion "https://github.com/devanathandna/vegapunk-multi-tab-persistent-memory-conversational-ai/blob/main/backend/fusion_layer.py"
click node_backend_extractor "https://github.com/devanathandna/vegapunk-multi-tab-persistent-memory-conversational-ai/blob/main/backend/entity_extractor.py"
click node_backend_database "https://github.com/devanathandna/vegapunk-multi-tab-persistent-memory-conversational-ai/blob/main/backend/database.py"
click node_backend_vector "https://github.com/devanathandna/vegapunk-multi-tab-persistent-memory-conversational-ai/blob/main/backend/vector_store.py"
click node_backend_graph "https://github.com/devanathandna/vegapunk-multi-tab-persistent-memory-conversational-ai/blob/main/backend/graph_store.py"
click node_main_chat "https://github.com/devanathandna/vegapunk-multi-tab-persistent-memory-conversational-ai/blob/main/src/components/MainChat.tsx"
click node_sub_chat "https://github.com/devanathandna/vegapunk-multi-tab-persistent-memory-conversational-ai/blob/main/src/components/SubChat.tsx"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_src_main,node_src_pages,node_src_chat,node_src_sidebar,node_main_chat,node_sub_chat toneBlue
class node_backend_app,node_backend_agent,node_backend_fusion,node_backend_extractor toneAmber
class node_backend_database,node_backend_vector,node_backend_graph toneMint
class node_gemini_api toneNeutral
```

## Components

### Frontend

React/Vite chat UI and interaction layer

- [React app](https://github.com/devanathandna/vegapunk-multi-tab-persistent-memory-conversational-ai/blob/main/src/main.tsx): Vite client · Browser application shell and routing · `src/main.tsx`
- [Pages](https://github.com/devanathandna/vegapunk-multi-tab-persistent-memory-conversational-ai/blob/main/src/pages/VegaChat.tsx): UI routes · Top-level views for home, chat, and fallback · `src/pages/VegaChat.tsx`
- [Chat UI](https://github.com/devanathandna/vegapunk-multi-tab-persistent-memory-conversational-ai/blob/main/src/components/MainChat.tsx): components · Main chat, sub chat, input, and message presentation · `src/components/MainChat.tsx`
- [Sidebars](https://github.com/devanathandna/vegapunk-multi-tab-persistent-memory-conversational-ai/blob/main/src/components/Sidebar.tsx): navigation · Conversation navigation and tab-like chat switching · `src/components/Sidebar.tsx`
- [Main chat](https://github.com/devanathandna/vegapunk-multi-tab-persistent-memory-conversational-ai/blob/main/src/components/MainChat.tsx): persistent thread · Primary conversation flow that contributes to long-term memory · `src/components/MainChat.tsx`
- [Sub chat](https://github.com/devanathandna/vegapunk-multi-tab-persistent-memory-conversational-ai/blob/main/src/components/SubChat.tsx): sandbox thread · Isolated conversation branch that should not update shared memory · `src/components/SubChat.tsx`

### Backend

Flask API, orchestration, and memory subsystems

- [Flask API](https://github.com/devanathandna/vegapunk-multi-tab-persistent-memory-conversational-ai/blob/main/backend/app.py): HTTP entry · Request router and chat service boundary · `backend/app.py`
- [Agent core](https://github.com/devanathandna/vegapunk-multi-tab-persistent-memory-conversational-ai/blob/main/backend/agent.py): orchestrator · Classifies queries, coordinates retrieval, and synthesizes responses · `backend/agent.py`
- [Fusion layer](https://github.com/devanathandna/vegapunk-multi-tab-persistent-memory-conversational-ai/blob/main/backend/fusion_layer.py): ranker · Combines retrieval candidates into final relevance scores · `backend/fusion_layer.py`
- [Entity extractor](https://github.com/devanathandna/vegapunk-multi-tab-persistent-memory-conversational-ai/blob/main/backend/entity_extractor.py): graph prep · Extracts entities and relations from chat turns · `backend/entity_extractor.py`

### Memory

Persistent episodic, semantic, and structural storage

- [SQLite store](https://github.com/devanathandna/vegapunk-multi-tab-persistent-memory-conversational-ai/blob/main/backend/database.py): episodic memory · Conversation sessions, messages, and timestamps · `backend/database.py`
- [Vector store](https://github.com/devanathandna/vegapunk-multi-tab-persistent-memory-conversational-ai/blob/main/backend/vector_store.py): semantic memory · Embedding-based similarity retrieval over past content · `backend/vector_store.py`
- [Graph store](https://github.com/devanathandna/vegapunk-multi-tab-persistent-memory-conversational-ai/blob/main/backend/graph_store.py): structural memory · Knowledge graph for entities, relations, and multi-hop recall · `backend/graph_store.py`

### Other components

- **Gemini API**: LLM service · External model used for classification, synthesis, and extraction

## Connections

- React app → Pages: renders · App shell loads route-level views
- Pages → Chat UI: hosts · Chat page composes the main conversation surface
- Pages → Sidebars: includes · Pages provide conversation navigation
- Chat UI → Main chat: drives · Primary UI path for persistent conversations
- Chat UI → Sub chat: switches · Secondary UI path for isolated exploration
- Chat UI → Flask API: chat request · User turns are sent to the backend API
- Flask API → Agent core: delegates · HTTP layer forwards reasoning work to the agent core
- Agent core → Fusion layer: ranks · Agent uses fusion to choose the best context
- Agent core → SQLite store: recalls · Fetches episodic conversation history
- Agent core → Vector store: recalls · Fetches semantic matches from embeddings
- Agent core → Graph store: recalls · Fetches structural facts and relations
- Agent core → Gemini API: synthesizes · LLM generates the final response
- Agent core → Entity extractor: updates · Background path prepares memory enrichment
- Entity extractor → Gemini API: extracts via · Model-assisted entity and relation extraction
- Entity extractor → Vector store: indexes · Stores embedded content for fuzzy recall
- Entity extractor → Graph store: writes · Persists entities and weighted relationships
- Fusion layer → SQLite store: scores · Uses recency and exact-history signals
- Fusion layer → Vector store: scores · Uses semantic similarity signals
- Fusion layer → Graph store: scores · Uses structural and multi-hop signals
- Main chat → SQLite store: persists · Main chat contributes to long-term episodic memory
- Sub chat → SQLite store: isolated from · Sub chat should not contaminate shared persistent memory