# eugeneyan/obsidian-copilot architecture

> Architecture diagram of the GitHub repository eugeneyan/obsidian-copilot, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/eugeneyan/obsidian-copilot
- Repository: https://github.com/eugeneyan/obsidian-copilot

## Overview

##### Project type and purpose
- Obsidian-copilot is a full-stack retrieval-augmented generation (RAG) tool that integrates with Obsidian (a markdown note-taking app).
- It consists of two primary parts:
a. A Python-based backend service for indexing and retrieval (using OpenSearch and semantic search via Hugging Face embeddings).
b. An Obsidian plugin (TypeScript/JavaScript) that calls the backend to draft or reflect on notes in real time.

##### File structure
- Top-level:
- Dockerfile, docker-compose.yml, Makefile, build.sh – container build & orchestration
- src/ – Python backend code (FastAPI app, logger, and indexing scripts under src/prep)
- plugin/ – Obsidian plugin code (main.ts/js, manifest.json, styles.css, version bump, package.json)
- data/ – placeholder for any data artifacts
- assets/ – readme images
- requirements.txt – Python dependencies
- The presence of build_opensearch_index.py and build_semantic_index.py indicates two parallel indexing pipelines.
- docker-compose.yml spins up an OpenSearch container that the backend will use.

##### README for additional insights
- It describes RAG flow: build indices → start OpenSearch → run retrieval app → install plugin
- It references the Hugging Face cache and vault path environment variables
- Makefile targets: build (docker), opensearch (stand up container), build-artifacts (run indexing), run (start API), install-plugin

##### Main components & interactions
a. Obsidian Vault (user’s markdown notes)
b. Indexing Pipelines (Python scripts):
– build_opensearch_index.py → keyword-based indexing into OpenSearch
– build_semantic_index.py → embeddings-based vector store (could be a simple local store or wrapped by same service)
c. OpenSearch Container (launched via docker-compose)
d. Semantic Store / Hugging Face Cache (local filesystem)
e. Retrieval API Service (src/app.py, served by Uvicorn/FastAPI in a Docker container)
f. Obsidian Plugin (runs in Obsidian, written in TS/JS)
g. External Services (Hugging Face transformers, optionally the Internet for downloading models)

Relationships:
- Indexing Pipelines read from Obsidian Vault → push to OpenSearch & Semantic Store
- Retrieval API reads from both indices, merges results, and exposes REST endpoints
- Obsidian Plugin calls the Retrieval API over HTTP → renders suggestions in the editor

##### Architectural patterns & design principles
- Separation of concerns: indexing vs. retrieval vs. presentation (plugin)
- RAG pattern: blending sparse (keyword) and dense (embedding) search
- Containerization: each major service runs in its own container (API, OpenSearch)
- Extensibility: plugin manifest, version bump script, ESBuild config

##### Elements
- Labeled nodes: Obsidian Vault, Index Builder, OpenSearch, Semantic Store, Retrieval API, Obsidian Plugin
- Annotation callouts: “Built by build_opensearch_index.py”, “Built by build_semantic_index.py”, “Served via Uvicorn”

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph "Obsidian Vault & Cache"
        Vault["Obsidian Vault"]:::disk
        HFCache["Hugging Face Cache"]:::disk
    end

    subgraph "Indexing Pipelines"
        VDict["Vault Dict Builder<br>(build_vault_dict.py)"]:::compute
        KIndex["Keyword Indexer<br>(build_opensearch_index.py)"]:::compute
        SIndex["Semantic Indexer<br>(build_semantic_index.py)"]:::compute
    end

    Vault -->|"reads notes"| VDict
    VDict -->|"feeds"| KIndex
    VDict -->|"feeds"| SIndex
    KIndex -->|"indexes →"| OpenSearch
    SIndex -->|"stores embeddings →"| SemanticStore

    subgraph "Data Stores"
        OpenSearch["OpenSearch<br>Container"]:::service
        SemanticStore["Semantic Store<br>(data/)"]:::disk
    end

    OpenSearch -->|"search"| RetrievalAPI
    SemanticStore -->|"embedding lookup"| RetrievalAPI

    subgraph "Retrieval Backend"
        RetrievalAPI["Retrieval API<br>(src/app.py)"]:::service
        Logger["Logger Module<br>(src/logger.py)"]:::compute
    end

    RetrievalAPI -->|"logs via"| Logger

    subgraph "Obsidian Plugin"
        Main["Plugin Main<br>(main.ts/js)"]:::ui
        Manifest["Manifest<br>(manifest.json)"]:::ui
        ESBuild["Build Config<br>(esbuild.config.mjs)"]:::ui
        Styles["Stylesheet<br>(styles.css)"]:::ui
        VersionBump["Version Bump<br>(version-bump.mjs)"]:::ui
        Package["Package.json<br>(package.json)"]:::ui
    end

    Main -->|"HTTP Request"| RetrievalAPI
    RetrievalAPI -->|"HTTP Response"| Main

    subgraph "Deployment & Orchestration"
        MF["Makefile"]:::config
        DC["docker-compose.yml"]:::config
        DF["Dockerfile"]:::config
        BS["build.sh"]:::config
    end

    MF --> DC
    DC --> OpenSearch
    DF --> RetrievalAPI
    BS --> KIndex
    BS --> SIndex

    classDef disk fill:#ccffcc,stroke:#006600;
    classDef service fill:#cce5ff,stroke:#004080;
    classDef compute fill:#e6e6e6,stroke:#666666;
    classDef ui fill:#ffe0b3,stroke:#cc7a00;
    classDef config fill:#f2f2f2,stroke:#999999;

    click KIndex "https://github.com/eugeneyan/obsidian-copilot/blob/main/src/prep/build_opensearch_index.py"
    click SIndex "https://github.com/eugeneyan/obsidian-copilot/blob/main/src/prep/build_semantic_index.py"
    click VDict "https://github.com/eugeneyan/obsidian-copilot/blob/main/src/prep/build_vault_dict.py"
    click RetrievalAPI "https://github.com/eugeneyan/obsidian-copilot/blob/main/src/app.py"
    click Logger "https://github.com/eugeneyan/obsidian-copilot/blob/main/src/logger.py"
    click MF "https://github.com/eugeneyan/obsidian-copilot/tree/main/Makefile"
    click DC "https://github.com/eugeneyan/obsidian-copilot/blob/main/docker-compose.yml"
    click DF "https://github.com/eugeneyan/obsidian-copilot/tree/main/Dockerfile"
    click BS "https://github.com/eugeneyan/obsidian-copilot/blob/main/build.sh"
    click SemanticStore "https://github.com/eugeneyan/obsidian-copilot/tree/main/data/"
    click Main "https://github.com/eugeneyan/obsidian-copilot/blob/main/plugin/main.ts"
    click Manifest "https://github.com/eugeneyan/obsidian-copilot/blob/main/plugin/manifest.json"
    click ESBuild "https://github.com/eugeneyan/obsidian-copilot/blob/main/plugin/esbuild.config.mjs"
    click Styles "https://github.com/eugeneyan/obsidian-copilot/blob/main/plugin/styles.css"
    click VersionBump "https://github.com/eugeneyan/obsidian-copilot/blob/main/plugin/version-bump.mjs"
    click Package "https://github.com/eugeneyan/obsidian-copilot/blob/main/plugin/package.json"
```
