# mshtawythug/second-brain architecture

> Architecture diagram of the GitHub repository mshtawythug/second-brain, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mshtawythug/second-brain
- Repository: https://github.com/mshtawythug/second-brain

## Overview

Second Brain is a local knowledge base combining document ingestion, PostgreSQL/pgvector storage, hybrid retrieval, an optional entity graph, and note-vault workflows. Users capture or import material through the CLI, then search, recall, or ask questions through the CLI, MCP server, or web UI; proactive digest, review, and linking features extend that loop. The principal pipeline and several dependencies are evidenced by excerpts and the source index; README-backed branches use documented capabilities, while unsampled implementation details are not expanded.

Main entry points: [`src/brain/ingest/__init__.py`](https://github.com/mshtawythug/second-brain/blob/master/src/brain/ingest/__init__.py) (Ingest Pipeline), [`src/brain/search_pipeline.py`](https://github.com/mshtawythug/second-brain/blob/master/src/brain/search_pipeline.py) (Search Pipeline), [`src/brain/ui/ingest_service.py`](https://github.com/mshtawythug/second-brain/blob/master/src/brain/ui/ingest_service.py) (Ingest Jobs), [`src/brain/cli.py`](https://github.com/mshtawythug/second-brain/blob/master/src/brain/cli.py) (Brain CLI)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_capture["Capture and Ingest"]
  node_ingest_service["Ingest Jobs<br/>[ingest_service.py]"]
  node_ingest_cli["CLI Ingest<br/>[cli_ingest.py]"]
  node_extract["Extractor Dispatch<br/>[dispatch.py]"]
  node_ingest_core["Ingest Pipeline<br/>[__init__.py]"]
end

subgraph group_retrieval["Search and Answers"]
  node_search_pipeline["Search Pipeline<br/>[search_pipeline.py]"]
  node_search["Hybrid Search<br/>[search.py]"]
  node_recall["Recall and Ask<br/>[recall.py]"]
end

subgraph group_knowledge["Knowledge Features"]
  node_enrich["LLM Enrichment<br/>[enrichment.py]"]
  node_sensitivity["Sensitivity Policy<br/>[sensitivity.py]"]
  node_graph["GraphRAG Sync<br/>[sync.py]"]
  node_vault["Markdown Vault<br/>[sync.py]"]
  node_proactive["Proactive Synthesis<br/>[brief.py]"]
end

subgraph group_interfaces["User Interfaces"]
  node_cli["Brain CLI<br/>[cli.py]"]
  node_mcp["MCP Server<br/>[mcp_server.py]"]
  node_web["Local Web UI<br/>[server.py]"]
  node_web_ingest["Ingest Routes<br/>[routes_ingest.py]"]
end

subgraph group_platform["Storage and Models"]
  node_embed["Embedding Backends<br/>[embeddings.py]"]
  node_db[("Postgres Store<br/>[db.py]")]
  node_config["Runtime Configuration<br/>[config.py]"]
end

node_person(("User or Agent"))
node_ollama["Ollama Service"]

node_person -->|"runs commands"| node_cli
node_person -->|"uses UI"| node_web
node_person -->|"queries knowledge"| node_mcp
node_cli -->|"ingests sources"| node_ingest_cli
node_ingest_cli -->|"dispatches extraction"| node_extract
node_extract -->|"provides extracted docs"| node_ingest_core
node_ingest_core -->|"embeds content"| node_embed
node_ingest_core -->|"enriches documents"| node_enrich
node_ingest_core -->|"guards content"| node_sensitivity
node_ingest_core -->|"writes documents"| node_db
node_ingest_core -->|"reconciles graph"| node_graph
node_web -.->|"serves routes"| node_web_ingest
node_web_ingest -->|"starts jobs"| node_ingest_service
node_ingest_service -->|"extracts uploads"| node_extract
node_ingest_service -->|"ingests documents"| node_ingest_core
node_cli -->|"searches corpus"| node_search_pipeline
node_search_pipeline -->|"runs retrieval"| node_search
node_search -->|"retrieves records"| node_db
node_search_pipeline -->|"supports recall and ask"| node_recall
node_mcp -.->|"searches knowledge"| node_search_pipeline
node_web -->|"searches corpus"| node_search_pipeline
node_person -->|"requests passages"| node_recall
node_ingest_core -->|"exports notes"| node_vault
node_person -->|"runs synthesis"| node_proactive
node_proactive -.->|"publishes pages"| node_vault
node_person -->|"uses GraphRAG"| node_graph
node_embed -->|"requests embeddings"| node_ollama
node_enrich -->|"requests enrichment"| node_ollama
node_config -->|"selects backend"| node_embed
node_config -->|"configures connection"| node_db

click node_cli "https://github.com/mshtawythug/second-brain/blob/master/src/brain/cli.py"
click node_mcp "https://github.com/mshtawythug/second-brain/blob/master/src/brain/mcp_server.py"
click node_web "https://github.com/mshtawythug/second-brain/blob/master/src/brain/ui/server.py"
click node_web_ingest "https://github.com/mshtawythug/second-brain/blob/master/src/brain/ui/routes_ingest.py"
click node_ingest_service "https://github.com/mshtawythug/second-brain/blob/master/src/brain/ui/ingest_service.py"
click node_ingest_cli "https://github.com/mshtawythug/second-brain/blob/master/src/brain/cli_ingest.py"
click node_extract "https://github.com/mshtawythug/second-brain/blob/master/src/brain/ingest/dispatch.py"
click node_ingest_core "https://github.com/mshtawythug/second-brain/blob/master/src/brain/ingest/__init__.py"
click node_embed "https://github.com/mshtawythug/second-brain/blob/master/src/brain/embeddings.py"
click node_enrich "https://github.com/mshtawythug/second-brain/blob/master/src/brain/enrichment.py"
click node_sensitivity "https://github.com/mshtawythug/second-brain/blob/master/src/brain/sensitivity.py"
click node_db "https://github.com/mshtawythug/second-brain/blob/master/src/brain/db.py"
click node_search_pipeline "https://github.com/mshtawythug/second-brain/blob/master/src/brain/search_pipeline.py"
click node_search "https://github.com/mshtawythug/second-brain/blob/master/src/brain/search.py"
click node_recall "https://github.com/mshtawythug/second-brain/blob/master/src/brain/recall.py"
click node_graph "https://github.com/mshtawythug/second-brain/blob/master/src/brain/graph_rag/sync.py"
click node_vault "https://github.com/mshtawythug/second-brain/blob/master/src/brain/vault/sync.py"
click node_proactive "https://github.com/mshtawythug/second-brain/blob/master/src/brain/brief.py"
click node_config "https://github.com/mshtawythug/second-brain/blob/master/src/brain/config.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_ingest_service,node_ingest_cli,node_extract,node_ingest_core,node_person toneBlue
class node_search_pipeline,node_search,node_recall toneAmber
class node_enrich,node_sensitivity,node_graph,node_vault,node_proactive toneMint
class node_cli,node_mcp,node_web,node_web_ingest toneRose
class node_embed,node_db,node_config,node_ollama toneIndigo
```

## Components

### Capture and Ingest

- [Ingest Jobs](https://github.com/mshtawythug/second-brain/blob/master/src/brain/ui/ingest_service.py): `src/brain/ui/ingest_service.py`
- [CLI Ingest](https://github.com/mshtawythug/second-brain/blob/master/src/brain/cli_ingest.py): `src/brain/cli_ingest.py`
- [Extractor Dispatch](https://github.com/mshtawythug/second-brain/blob/master/src/brain/ingest/dispatch.py): `src/brain/ingest/dispatch.py`
- [Ingest Pipeline](https://github.com/mshtawythug/second-brain/blob/master/src/brain/ingest/__init__.py): `src/brain/ingest/__init__.py`

### Search and Answers

- [Search Pipeline](https://github.com/mshtawythug/second-brain/blob/master/src/brain/search_pipeline.py): `src/brain/search_pipeline.py`
- [Hybrid Search](https://github.com/mshtawythug/second-brain/blob/master/src/brain/search.py): `src/brain/search.py`
- [Recall and Ask](https://github.com/mshtawythug/second-brain/blob/master/src/brain/recall.py): `src/brain/recall.py`

### Knowledge Features

- [LLM Enrichment](https://github.com/mshtawythug/second-brain/blob/master/src/brain/enrichment.py): `src/brain/enrichment.py`
- [Sensitivity Policy](https://github.com/mshtawythug/second-brain/blob/master/src/brain/sensitivity.py): `src/brain/sensitivity.py`
- [GraphRAG Sync](https://github.com/mshtawythug/second-brain/blob/master/src/brain/graph_rag/sync.py): `src/brain/graph_rag/sync.py`
- [Markdown Vault](https://github.com/mshtawythug/second-brain/blob/master/src/brain/vault/sync.py): `src/brain/vault/sync.py`
- [Proactive Synthesis](https://github.com/mshtawythug/second-brain/blob/master/src/brain/brief.py): `src/brain/brief.py`

### User Interfaces

- [Brain CLI](https://github.com/mshtawythug/second-brain/blob/master/src/brain/cli.py): `src/brain/cli.py`
- [MCP Server](https://github.com/mshtawythug/second-brain/blob/master/src/brain/mcp_server.py): `src/brain/mcp_server.py`
- [Local Web UI](https://github.com/mshtawythug/second-brain/blob/master/src/brain/ui/server.py): `src/brain/ui/server.py`
- [Ingest Routes](https://github.com/mshtawythug/second-brain/blob/master/src/brain/ui/routes_ingest.py): `src/brain/ui/routes_ingest.py`

### Storage and Models

- [Embedding Backends](https://github.com/mshtawythug/second-brain/blob/master/src/brain/embeddings.py): `src/brain/embeddings.py`
- [Postgres Store](https://github.com/mshtawythug/second-brain/blob/master/src/brain/db.py): `src/brain/db.py`
- [Runtime Configuration](https://github.com/mshtawythug/second-brain/blob/master/src/brain/config.py): `src/brain/config.py`

### Other components

- **User or Agent**
- **Ollama Service**

## Connections

- User or Agent → Brain CLI: runs commands
- User or Agent → Local Web UI: uses UI
- User or Agent → MCP Server: queries knowledge
- Brain CLI → CLI Ingest: ingests sources (evidence: `src/brain/cli_ingest.py`)
- CLI Ingest → Extractor Dispatch: dispatches extraction (evidence: `src/brain/cli_ingest.py`)
- Extractor Dispatch → Ingest Pipeline: provides extracted docs (evidence: `src/brain/ingest/__init__.py`)
- Ingest Pipeline → Embedding Backends: embeds content (evidence: `src/brain/ingest/__init__.py`)
- Ingest Pipeline → LLM Enrichment: enriches documents (evidence: `src/brain/enrichment.py`)
- Ingest Pipeline → Sensitivity Policy: guards content (evidence: `src/brain/ingest/__init__.py`)
- Ingest Pipeline → Postgres Store: writes documents (evidence: `src/brain/ingest/__init__.py`)
- Ingest Pipeline → GraphRAG Sync: reconciles graph (evidence: `src/brain/ingest/__init__.py`)
- Local Web UI → Ingest Routes: serves routes
- Ingest Routes → Ingest Jobs: starts jobs (evidence: `src/brain/ui/routes_ingest.py`)
- Ingest Jobs → Extractor Dispatch: extracts uploads (evidence: `src/brain/ui/ingest_service.py`)
- Ingest Jobs → Ingest Pipeline: ingests documents (evidence: `src/brain/ui/ingest_service.py`)
- Brain CLI → Search Pipeline: searches corpus (evidence: `src/brain/search_pipeline.py`)
- Search Pipeline → Hybrid Search: runs retrieval (evidence: `src/brain/search_pipeline.py`)
- Hybrid Search → Postgres Store: retrieves records
- Search Pipeline → Recall and Ask: supports recall and ask (evidence: `src/brain/search_pipeline.py`)
- MCP Server → Search Pipeline: searches knowledge
- Local Web UI → Search Pipeline: searches corpus
- User or Agent → Recall and Ask: requests passages
- Ingest Pipeline → Markdown Vault: exports notes (evidence: `src/brain/ingest/__init__.py`)
- User or Agent → Proactive Synthesis: runs synthesis
- Proactive Synthesis → Markdown Vault: publishes pages
- User or Agent → GraphRAG Sync: uses GraphRAG
- Embedding Backends → Ollama Service: requests embeddings (evidence: `src/brain/embeddings.py`)
- LLM Enrichment → Ollama Service: requests enrichment (evidence: `src/brain/enrichment.py`)
- Runtime Configuration → Embedding Backends: selects backend (evidence: `src/brain/embeddings.py`)
- Runtime Configuration → Postgres Store: configures connection (evidence: `src/brain/db.py`)
