How mshtawythug/second-brain works
Second Brain is a local knowledge base combining document ingestion, PostgreSQL/pgvector storage, hybrid retrieval, an optional entity graph, and note-vault workflows. Users capture or import material through the CLI, then search, recall, or ask questions through the CLI, MCP server, or web UI; proactive digest, review, and linking features extend that loop. The principal pipeline and several dependencies are evidenced by excerpts and the source index; README-backed branches use documented capabilities, while unsampled implementation details are not expanded.
Main entry points: __init__.py Ingest Pipeline · search_pipeline.py Search Pipeline · ingest_service.py Ingest Jobs · cli.py Brain CLI
Components 21
Capture and Ingest
- Ingest Jobs
src/brain/ui/ingest_service.py - CLI Ingest
src/brain/cli_ingest.py - Extractor Dispatch
src/brain/ingest/dispatch.py - Ingest Pipeline
src/brain/ingest/__init__.py
Search and Answers
- Search Pipeline
src/brain/search_pipeline.py - Hybrid Search
src/brain/search.py - Recall and Ask
src/brain/recall.py
Knowledge Features
- LLM Enrichment
src/brain/enrichment.py - Sensitivity Policy
src/brain/sensitivity.py - GraphRAG Sync
src/brain/graph_rag/sync.py - Markdown Vault
src/brain/vault/sync.py - Proactive Synthesis
src/brain/brief.py
User Interfaces
- Brain CLI
src/brain/cli.py - MCP Server
src/brain/mcp_server.py - Local Web UI
src/brain/ui/server.py - Ingest Routes
src/brain/ui/routes_ingest.py
Storage and Models
- Embedding Backends
src/brain/embeddings.py - Postgres Store
src/brain/db.py - Runtime Configuration
src/brain/config.py
Other components
- User or Agent
- Ollama Service
Connections 30
- User or Agent → Brain CLI: runs commands
- User or Agent → Local Web UI: uses UI
- User or Agent → MCP Server: queries knowledge
- Brain CLI → CLI Ingest: ingests sources
src/brain/cli_ingest.py - CLI Ingest → Extractor Dispatch: dispatches extraction
src/brain/cli_ingest.py - Extractor Dispatch → Ingest Pipeline: provides extracted docs
src/brain/ingest/__init__.py - Ingest Pipeline → Embedding Backends: embeds content
src/brain/ingest/__init__.py - Ingest Pipeline → LLM Enrichment: enriches documents
src/brain/enrichment.py - Ingest Pipeline → Sensitivity Policy: guards content
src/brain/ingest/__init__.py - Ingest Pipeline → Postgres Store: writes documents
src/brain/ingest/__init__.py - Ingest Pipeline → GraphRAG Sync: reconciles graph
src/brain/ingest/__init__.py - Local Web UI → Ingest Routes: serves routes
- Ingest Routes → Ingest Jobs: starts jobs
src/brain/ui/routes_ingest.py - Ingest Jobs → Extractor Dispatch: extracts uploads
src/brain/ui/ingest_service.py - Ingest Jobs → Ingest Pipeline: ingests documents
src/brain/ui/ingest_service.py - Brain CLI → Search Pipeline: searches corpus
src/brain/search_pipeline.py - Search Pipeline → Hybrid Search: runs retrieval
src/brain/search_pipeline.py - Hybrid Search → Postgres Store: retrieves records
- Search Pipeline → Recall and Ask: supports recall and ask
src/brain/search_pipeline.py - MCP Server → Search Pipeline: searches knowledge
- Local Web UI → Search Pipeline: searches corpus
- User or Agent → Recall and Ask: requests passages
- Ingest Pipeline → Markdown Vault: exports notes
src/brain/ingest/__init__.py - User or Agent → Proactive Synthesis: runs synthesis
- Proactive Synthesis → Markdown Vault: publishes pages
- User or Agent → GraphRAG Sync: uses GraphRAG
- Embedding Backends → Ollama Service: requests embeddings
src/brain/embeddings.py - LLM Enrichment → Ollama Service: requests enrichment
src/brain/enrichment.py - Runtime Configuration → Embedding Backends: selects backend
src/brain/embeddings.py - Runtime Configuration → Postgres Store: configures connection
src/brain/db.py