How arkohut/pensieve works
Based on your project’s file structure and README, Pensieve (formerly known as Memos) is a full‐stack application focused on passive screen recording, image indexing (using OCR, vector embeddings, and visual search via VLM), and a convenient web interface for retrieval. Its architecture is split across several distinct areas: screen capture, event-based processing, plugin-driven data extraction/indexing, data persistence (using either SQLite or PostgreSQL), and a modern Svelte-based front end.
Main Components and Layers
• Screenshot Recording Component:
• “memos record” functionality (and the “screen_recorder” module) that actively captures screenshots.
• Event Watching and Processing Component:
• “memos watch” service that listens for new screenshot events.
• This part dynamically submits indexing jobs and delegates processing tasks.
• Data Processing and Plugin Layer:
• Components for OCR and VLM plugins (located under memos/plugins/ocr and memos/plugins/vlm).
• Where embedding modules and the simple tokenizer exist.
• Connections to the external ML backends (in memos_ml_backends) which serve specialized functionalities for ML-powered indexing.
• API/Web Server Component:
• “memos serve” (and memos/main.py) which runs the web server providing access and a dashboard.
• It processes indexing/search requests.
• Frontend Component:
• Database Storage:
• A persistence layer that might use SQLite by default, with optional enhanced performance using PostgreSQL (with pgvector extension).
• Background Services & CLI:
• A note that the system can be manipulated from the command line (via “memos init”, “memos start”, “memos migrate”, etc.) and that the service can run in separate processes.
Establish Relationships and Integration
• Screenshots are captured by the recording service → sent to the watch/processing module.
• Processors apply OCR, embeddings, and VLM plugin operations to extract text and image metadata.
• Processed results are cached/indexed and then stored in the database.
• The API/web server reads from the database and serves data to the frontend.
• Optionally, annotate that the CLI commands can trigger configuration, migration, or manual re-indexing of screenshots.
• Optional routes such as switching from local SQLite to a PostgreSQL backend via configuration editing.
Point Out Architectural Patterns and Design Principles
• Separation of Concerns: Distinguish between services for capturing, processing, indexing, and serving data.
• Modular Extensibility: Emphasize the plugins (for OCR, VLM, etc.) demonstrating a modular design that allows adding or replacing functionality.
• Event-Driven Processing: Use an event or watch pattern for asynchronous processing of screenshots.
• Client-Server Architecture: Highlight the Svelte-based frontend communicating with a Python-powered backend via a REST or similar API.
• Scalability Aspect: Note the separation of ML processing tasks and the dedicated database layer, especially when using PostgreSQL for large-scale data.
Relevant Technologies and Libraries
• Python Core with its CLI commands, configuration (YAML-based), and migrations using Alembic.
• Use of Docker for PostgreSQL with the pgvector extension.
• Svelte for the front end.
• External integrations such as Ollama for multimodal visual search.
• Tools like alembic for database migration and logging modules for tracking operations.