# arkohut/pensieve architecture

> Architecture diagram of the GitHub repository arkohut/pensieve, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-02-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/arkohut/pensieve
- Repository: https://github.com/arkohut/pensieve

## Overview

Based on your project’s file structure and README, Pensieve (formerly known as Memos) is a full‐stack application focused on passive screen recording, image indexing (using OCR, vector embeddings, and visual search via VLM), and a convenient web interface for retrieval. Its architecture is split across several distinct areas: screen capture, event-based processing, plugin-driven data extraction/indexing, data persistence (using either SQLite or PostgreSQL), and a modern Svelte-based front end.

##### Main Components and Layers
- Screenshot Recording Component:
- “memos record” functionality (and the “screen_recorder” module) that actively captures screenshots.
- Event Watching and Processing Component:
- “memos watch” service that listens for new screenshot events.
- This part dynamically submits indexing jobs and delegates processing tasks.
- Data Processing and Plugin Layer:
- Components for OCR and VLM plugins (located under memos/plugins/ocr and memos/plugins/vlm).
- Where embedding modules and the simple tokenizer exist.
- Connections to the external ML backends (in memos_ml_backends) which serve specialized functionalities for ML-powered indexing.
- API/Web Server Component:
- “memos serve” (and memos/main.py) which runs the web server providing access and a dashboard.
- It processes indexing/search requests.
- Frontend Component:
- Database Storage:
- A persistence layer that might use SQLite by default, with optional enhanced performance using PostgreSQL (with pgvector extension).
- Background Services & CLI:
- A note that the system can be manipulated from the command line (via “memos init”, “memos start”, “memos migrate”, etc.) and that the service can run in separate processes.

##### Establish Relationships and Integration
- Screenshots are captured by the recording service → sent to the watch/processing module.
- Processors apply OCR, embeddings, and VLM plugin operations to extract text and image metadata.
- Processed results are cached/indexed and then stored in the database.
- The API/web server reads from the database and serves data to the frontend.
- Optionally, annotate that the CLI commands can trigger configuration, migration, or manual re-indexing of screenshots.
- Optional routes such as switching from local SQLite to a PostgreSQL backend via configuration editing.

##### Point Out Architectural Patterns and Design Principles
- Separation of Concerns: Distinguish between services for capturing, processing, indexing, and serving data.
- Modular Extensibility: Emphasize the plugins (for OCR, VLM, etc.) demonstrating a modular design that allows adding or replacing functionality.
- Event-Driven Processing: Use an event or watch pattern for asynchronous processing of screenshots.
- Client-Server Architecture: Highlight the Svelte-based frontend communicating with a Python-powered backend via a REST or similar API.
- Scalability Aspect: Note the separation of ML processing tasks and the dedicated database layer, especially when using PostgreSQL for large-scale data.

##### Relevant Technologies and Libraries
- Python Core with its CLI commands, configuration (YAML-based), and migrations using Alembic.
- Use of Docker for PostgreSQL with the pgvector extension.
- Svelte for the front end.
- External integrations such as Ollama for multimodal visual search.
- Tools like alembic for database migration and logging modules for tracking operations.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Screenshot Capture Layer
    subgraph "Screenshot Capture Layer"
        R1["Recording Service (memos record)"]:::service
        R2["Screen Recorder Module (screen_recorder)"]:::service
    end

    %% Event Processing Layer
    subgraph "Event Processing Layer"
        E1["Event Watcher Service (memos watch)"]:::service
        CLI1["CLI Commands (memos cmds & memos/commands.py)"]:::cli
    end

    %% Data Processing & Plugin Layer
    subgraph "Data Processing & Plugin Layer"
        P1["OCR Plugin (memos/plugins/ocr)"]:::plugin
        P2["VLM Plugin (memos/plugins/vlm)"]:::plugin
        P3["Embedding Module (memos/embedding.py)"]:::plugin
        P4["Simple Tokenizer (memos/simple_tokenizer)"]:::plugin
        P5["External ML Backends (memos_ml_backends)"]:::plugin
    end

    %% API/Web Server Layer
    subgraph "API/Web Server Layer"
        A1["Web Server (memos/main.py & memos/server.py)"]:::service
    end

    %% Database Layer
    subgraph "Database Layer"
        DB["Database Storage (SQLite/PostgreSQL)"]:::db
    end

    %% Frontend Layer
    subgraph "Frontend Layer"
        F1["Web UI (Svelte) (web)"]:::frontend
    end

    %% Background Services & CLI
    subgraph "Background Services & CLI"
        CLI2["CLI Utility (memos_app.py)"]:::cli
    end

    %% Data Flow
    R1 -->|"captures"| E1
    R2 -->|"captures"| E1

    E1 -->|"submits"| P1
    E1 -->|"submits"| P2
    E1 -->|"submits"| P3
    E1 -->|"submits"| P4
    E1 -->|"submits"| P5

    P1 -->|"stores"| DB
    P2 -->|"stores"| DB
    P3 -->|"stores"| DB
    P4 -->|"stores"| DB
    P5 -->|"stores"| DB

    DB -->|"retrieves"| A1
    A1 -->|"serves"| F1

    %% CLI interactions
    CLI1 -->|"triggers"| E1
    CLI1 -->|"migrates"| DB
    CLI2 -->|"re-indexes"| E1
    CLI2 -->|"configures"| DB

    %% Click Events
    click R1 "https://github.com/arkohut/pensieve/blob/master/memos/record.py"
    click R2 "https://github.com/arkohut/pensieve/tree/master/screen_recorder/"
    click CLI1 "https://github.com/arkohut/pensieve/tree/master/memos/cmds/"
    click P1 "https://github.com/arkohut/pensieve/tree/master/memos/plugins/ocr/"
    click P2 "https://github.com/arkohut/pensieve/tree/master/memos/plugins/vlm/"
    click P3 "https://github.com/arkohut/pensieve/blob/master/memos/embedding.py"
    click P4 "https://github.com/arkohut/pensieve/tree/master/memos/simple_tokenizer/"
    click P5 "https://github.com/arkohut/pensieve/tree/master/memos_ml_backends/"
    click A1 "https://github.com/arkohut/pensieve/blob/master/memos/main.py"
    click DB "https://github.com/arkohut/pensieve/tree/master/memos/databases/"
    click F1 "https://github.com/arkohut/pensieve/tree/master/web/"
    click CLI2 "https://github.com/arkohut/pensieve/blob/master/memos_app.py"

    %% Styles
    classDef service fill:#e3f2fd,stroke:#2196f3,stroke-width:2px;
    classDef plugin fill:#e8f5e9,stroke:#43a047,stroke-width:2px;
    classDef db fill:#fff3e0,stroke:#fb8c00,stroke-width:2px;
    classDef frontend fill:#fce4ec,stroke:#e91e63,stroke-width:2px;
    classDef cli fill:#f3e5f5,stroke:#9c27b0,stroke-width:2px;
```
