# biswajitbiswal-in/secondself architecture

> Architecture diagram of the GitHub repository biswajitbiswal-in/secondself, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/biswajitbiswal-in/secondself
- Repository: https://github.com/biswajitbiswal-in/secondself

## Overview

SecondSelf is a local-first personal knowledge system: people capture notes, URLs, and files; captures are organized into PARA notes, semantically linked, and represented in an interactive graph; questions retrieve relevant notes and produce cited answers. The README documents these major capabilities and the pipeline order. The sampled source confirms key UI-to-backend and orchestrator calls; some backend wiring is unsampled, so those relationships are omitted where not established.

Main entry points: [`lib/storage.py`](https://github.com/biswajitbiswal-in/secondself/blob/main/lib/storage.py) (File storage), [`app.py`](https://github.com/biswajitbiswal-in/secondself/blob/main/app.py) (Streamlit app), [`ask.py`](https://github.com/biswajitbiswal-in/secondself/blob/main/ask.py) (RAG answering), [`pipeline.py`](https://github.com/biswajitbiswal-in/secondself/blob/main/pipeline.py) (Pipeline orchestrator)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interface["User interface"]
  node_app["Streamlit app<br/>[app.py]"]
  node_capture_ui["Capture page<br/>[capture.py]"]
  node_ask_ui["Ask page<br/>[ask.py]"]
  node_graph_ui["Graph view<br/>[graph.py]"]
  node_dashboard["Knowledge dashboard<br/>[dashboard.py]"]
  node_graph_viewer["Interactive viewer<br/>[graph.html]"]
end

subgraph group_knowledge["Knowledge pipeline"]
  node_capture["Capture intake<br/>[capture.py]"]
  node_extract["Content extraction<br/>[extract.py]"]
  node_pipeline["Pipeline orchestrator<br/>[pipeline.py]"]
  node_classify["PARA classification<br/>[classify.py]"]
  node_link["Semantic linking<br/>[link.py]"]
  node_graph_build["Graph builder<br/>[build_graph.py]"]
end

subgraph group_qa["Search and answers"]
  node_ask["RAG answering<br/>[ask.py]"]
  node_keyword_search["Keyword search<br/>[search.py]"]
end

subgraph group_state["Local knowledge state"]
  node_storage["File storage<br/>[storage.py]"]
  node_raw[("Raw captures")]
  node_wiki[("PARA notes")]
  node_index[("Processing index<br/>[index.json]")]
  node_embeddings_data[("Note embeddings<br/>[embeddings.pkl]")]
  node_graph_data[("Knowledge graph<br/>[graph.json]")]
end

subgraph group_integrations["AI integrations"]
  node_llm{{"Groq LLM"}}
  node_embedding_model{{"Sentence-transformers"}}
end

node_person(("Knowledge keeper"))

node_person -->|"uses"| node_app
node_person -->|"submits content"| node_capture_ui
node_person -->|"asks question"| node_ask_ui
node_app -->|"renders"| node_capture_ui
node_app -->|"renders"| node_ask_ui
node_app -->|"renders"| node_graph_ui
node_app -->|"renders"| node_dashboard
node_capture_ui -->|"captures"| node_capture
node_capture -->|"extracts text"| node_extract
node_capture -->|"saves capture"| node_storage
node_storage -->|"writes and reads"| node_raw
node_pipeline -->|"runs classification"| node_classify
node_pipeline -->|"runs linking"| node_link
node_pipeline -->|"builds graph"| node_graph_build
node_classify -->|"reads captures; writes notes"| node_storage
node_classify -->|"classifies content"| node_llm
node_link -->|"reads and updates notes"| node_storage
node_link -->|"computes similarity"| node_embedding_model
node_link -->|"persists vectors"| node_embeddings_data
node_graph_build -->|"reads notes"| node_storage
node_graph_build -->|"writes graph"| node_graph_data
node_pipeline -->|"tracks processing"| node_index
node_ask_ui -->|"submits question"| node_ask
node_ask -->|"retrieves notes"| node_storage
node_ask -->|"embeds query"| node_embedding_model
node_ask -->|"retrieves vectors"| node_embeddings_data
node_ask -->|"synthesizes answer"| node_llm
node_graph_ui -->|"loads graph"| node_graph_data
node_graph_ui -->|"displays graph"| node_graph_viewer
node_dashboard -->|"reads note stats"| node_storage
node_keyword_search -->|"searches notes"| node_storage

click node_app "https://github.com/biswajitbiswal-in/secondself/blob/main/app.py"
click node_capture_ui "https://github.com/biswajitbiswal-in/secondself/blob/main/ui/pages/capture.py"
click node_ask_ui "https://github.com/biswajitbiswal-in/secondself/blob/main/ui/pages/ask.py"
click node_graph_ui "https://github.com/biswajitbiswal-in/secondself/blob/main/ui/pages/graph.py"
click node_dashboard "https://github.com/biswajitbiswal-in/secondself/blob/main/ui/pages/dashboard.py"
click node_capture "https://github.com/biswajitbiswal-in/secondself/blob/main/capture.py"
click node_extract "https://github.com/biswajitbiswal-in/secondself/blob/main/lib/extract.py"
click node_pipeline "https://github.com/biswajitbiswal-in/secondself/blob/main/pipeline.py"
click node_classify "https://github.com/biswajitbiswal-in/secondself/blob/main/classify.py"
click node_link "https://github.com/biswajitbiswal-in/secondself/blob/main/link.py"
click node_graph_build "https://github.com/biswajitbiswal-in/secondself/blob/main/build_graph.py"
click node_ask "https://github.com/biswajitbiswal-in/secondself/blob/main/ask.py"
click node_keyword_search "https://github.com/biswajitbiswal-in/secondself/blob/main/search.py"
click node_storage "https://github.com/biswajitbiswal-in/secondself/blob/main/lib/storage.py"
click node_raw "https://github.com/biswajitbiswal-in/secondself/tree/main/data"
click node_wiki "https://github.com/biswajitbiswal-in/secondself/tree/main/wiki"
click node_index "https://github.com/biswajitbiswal-in/secondself/blob/main/data/index.json"
click node_embeddings_data "https://github.com/biswajitbiswal-in/secondself/blob/main/data/embeddings.pkl"
click node_graph_data "https://github.com/biswajitbiswal-in/secondself/blob/main/data/graph.json"
click node_graph_viewer "https://github.com/biswajitbiswal-in/secondself/blob/main/static/graph.html"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_app,node_capture_ui,node_ask_ui,node_graph_ui,node_dashboard,node_graph_viewer toneBlue
class node_capture,node_extract,node_pipeline,node_classify,node_link,node_graph_build toneAmber
class node_ask,node_keyword_search toneMint
class node_storage,node_raw,node_wiki,node_index,node_embeddings_data,node_graph_data toneRose
class node_llm,node_embedding_model,node_person toneIndigo
```

## Components

### User interface

- [Streamlit app](https://github.com/biswajitbiswal-in/secondself/blob/main/app.py): `app.py`
- [Capture page](https://github.com/biswajitbiswal-in/secondself/blob/main/ui/pages/capture.py): `ui/pages/capture.py`
- [Ask page](https://github.com/biswajitbiswal-in/secondself/blob/main/ui/pages/ask.py): `ui/pages/ask.py`
- [Graph view](https://github.com/biswajitbiswal-in/secondself/blob/main/ui/pages/graph.py): `ui/pages/graph.py`
- [Knowledge dashboard](https://github.com/biswajitbiswal-in/secondself/blob/main/ui/pages/dashboard.py): `ui/pages/dashboard.py`
- [Interactive viewer](https://github.com/biswajitbiswal-in/secondself/blob/main/static/graph.html): `static/graph.html`

### Knowledge pipeline

- [Capture intake](https://github.com/biswajitbiswal-in/secondself/blob/main/capture.py): `capture.py`
- [Content extraction](https://github.com/biswajitbiswal-in/secondself/blob/main/lib/extract.py): `lib/extract.py`
- [Pipeline orchestrator](https://github.com/biswajitbiswal-in/secondself/blob/main/pipeline.py): `pipeline.py`
- [PARA classification](https://github.com/biswajitbiswal-in/secondself/blob/main/classify.py): `classify.py`
- [Semantic linking](https://github.com/biswajitbiswal-in/secondself/blob/main/link.py): `link.py`
- [Graph builder](https://github.com/biswajitbiswal-in/secondself/blob/main/build_graph.py): `build_graph.py`

### Search and answers

- [RAG answering](https://github.com/biswajitbiswal-in/secondself/blob/main/ask.py): `ask.py`
- [Keyword search](https://github.com/biswajitbiswal-in/secondself/blob/main/search.py): `search.py`

### Local knowledge state

- [File storage](https://github.com/biswajitbiswal-in/secondself/blob/main/lib/storage.py): `lib/storage.py`
- [Raw captures](https://github.com/biswajitbiswal-in/secondself/tree/main/data): `data`
- [PARA notes](https://github.com/biswajitbiswal-in/secondself/tree/main/wiki): `wiki`
- [Processing index](https://github.com/biswajitbiswal-in/secondself/blob/main/data/index.json): `data/index.json`
- [Note embeddings](https://github.com/biswajitbiswal-in/secondself/blob/main/data/embeddings.pkl): `data/embeddings.pkl`
- [Knowledge graph](https://github.com/biswajitbiswal-in/secondself/blob/main/data/graph.json): `data/graph.json`

### AI integrations

- **Groq LLM**
- **Sentence-transformers**

### Other components

- **Knowledge keeper**

## Connections

- Knowledge keeper → Streamlit app: uses
- Knowledge keeper → Capture page: submits content
- Knowledge keeper → Ask page: asks question
- Streamlit app → Capture page: renders
- Streamlit app → Ask page: renders
- Streamlit app → Graph view: renders
- Streamlit app → Knowledge dashboard: renders
- Capture page → Capture intake: captures
- Capture intake → Content extraction: extracts text
- Capture intake → File storage: saves capture
- File storage → Raw captures: writes and reads
- Pipeline orchestrator → PARA classification: runs classification
- Pipeline orchestrator → Semantic linking: runs linking
- Pipeline orchestrator → Graph builder: builds graph
- PARA classification → File storage: reads captures; writes notes
- PARA classification → Groq LLM: classifies content
- Semantic linking → File storage: reads and updates notes
- Semantic linking → Sentence-transformers: computes similarity
- Semantic linking → Note embeddings: persists vectors
- Graph builder → File storage: reads notes
- Graph builder → Knowledge graph: writes graph
- Pipeline orchestrator → Processing index: tracks processing
- Ask page → RAG answering: submits question
- RAG answering → File storage: retrieves notes
- RAG answering → Sentence-transformers: embeds query
- RAG answering → Note embeddings: retrieves vectors
- RAG answering → Groq LLM: synthesizes answer
- Graph view → Knowledge graph: loads graph
- Graph view → Interactive viewer: displays graph
- Knowledge dashboard → File storage: reads note stats
- Keyword search → File storage: searches notes