# mediar-ai/screenpipe architecture

> Architecture diagram of the GitHub repository mediar-ai/screenpipe, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mediar-ai/screenpipe
- Repository: https://github.com/mediar-ai/screenpipe

## Overview

##### Main Components Identification

a) Core Recording Components:
- screenpipe-vision (Screen capture and OCR)
- screenpipe-audio (Audio recording and processing)
- screenpipe-actions (Keyboard/mouse monitoring)

b) Data Processing & Storage:
- screenpipe-server (Central server component)
- SQLite database (Local data storage)
- Video/Audio cache system

c) Plugin System:
- Pipe framework
- Plugin sandbox environment
- Plugin store integration

d) User Interface:
- Tauri-based desktop app (screenpipe-app-tauri)
- Web-based UI components
- System tray integration

**┌─ User Interface Layer ─────────────────────┐**
- Desktop App (Tauri)                      │
- Web UI Components                        │
- System Tray                              │
- Plugin Layer ────────────────────────────┤
- Plugin Manager                           │
- Plugin Store                             │
- Plugin Sandbox                           │
- Core Services Layer ────────────────────┤
- Vision Service                           │
- Audio Service                            │
- Action Service                           │
- Processing Layer ─────────────────────────┤
- OCR Processing                           │
- Speech-to-Text                           │
- Speaker Identification                   │
- Storage Layer ──────────────────────────┤
- SQLite Database                          │
- Video Cache                              │
- Audio Cache                              │
- ┘

##### Data Flow Representation

- Screen/audio capture flow → processing → storage
- Plugin data access paths
- User interface interactions
- Real-time processing pipelines

##### Integration Points

- API endpoints
- Plugin extension points
- External service connections (e.g., AI services)

##### Monitoring & Control

- Health check flows
- Resource monitoring paths
- System control points

##### Data security boundaries

This system design should reflect Screenpipe's core value proposition of continuous recording while highlighting its extensible architecture through the plugin system.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Styles
    classDef ui fill:#gray,stroke:#333
    classDef core fill:#blue,stroke:#333
    classDef plugin fill:#purple,stroke:#333
    classDef storage fill:#yellow,stroke:#333
    classDef processing fill:#green,stroke:#333

    %% User Interface Layer
    subgraph UI["User Interface Layer"]
        Desktop["Desktop App (Tauri)"]
        WebUI["Web UI Components"]
        Tray["System Tray"]
        Settings["Settings Interface"]
    end

    %% Plugin Layer
    subgraph Plugins["Plugin Layer"]
        PluginMgr["Plugin Manager"]
        PluginStore["Plugin Store"]
        PluginSandbox["Plugin Sandbox"]
    end

    %% Core Services Layer
    subgraph CoreServices["Core Services Layer"]
        Vision["Vision Service"]
        Audio["Audio Service"]
        Action["Action Service"]
        ServerCore["Server Core"]
        HealthMon["Health Monitor"]
    end

    %% Processing Layer
    subgraph Processing["Processing Layer"]
        OCR["OCR Processing"]
        STT["Speech-to-Text"]
        SpeakerID["Speaker Identification"]
        VideoProc["Video Processing"]
    end

    %% Storage Layer
    subgraph Storage["Storage Layer"]
        DB["SQLite Database"]
        VideoCache["Video Cache"]
        AudioCache["Audio Cache"]
        DBMigrations["Database Migrations"]
    end

    %% Relationships
    Desktop --> PluginMgr
    Desktop --> CoreServices
    Tray --> ServerCore
    Vision --> OCR
    Audio --> STT
    Audio --> SpeakerID
    ServerCore --> VideoProc
    VideoProc --> VideoCache
    Audio --> AudioCache
    PluginMgr --> PluginSandbox
    PluginStore --> PluginMgr
    ServerCore --> DB
    DB --> DBMigrations
    HealthMon --> ServerCore

    %% Click Events
    click Desktop "https://github.com/mediar-ai/screenpipe/tree/main/screenpipe-app-tauri"
    click WebUI "https://github.com/mediar-ai/screenpipe/tree/main/screenpipe-app-tauri/components"
    click Tray "https://github.com/mediar-ai/screenpipe/blob/main/screenpipe-app-tauri/src-tauri/src/tray.rs"
    click Settings "https://github.com/mediar-ai/screenpipe/tree/main/screenpipe-app-tauri/components/settings"
    click PluginStore "https://github.com/mediar-ai/screenpipe/blob/main/screenpipe-app-tauri/components/pipe-store.tsx"
    click Vision "https://github.com/mediar-ai/screenpipe/tree/main/screenpipe-vision"
    click Audio "https://github.com/mediar-ai/screenpipe/tree/main/screenpipe-audio"
    click Action "https://github.com/mediar-ai/screenpipe/tree/main/screenpipe-actions"
    click ServerCore "https://github.com/mediar-ai/screenpipe/blob/main/screenpipe-server/src/core.rs"
    click HealthMon "https://github.com/mediar-ai/screenpipe/blob/main/screenpipe-server/src/resource_monitor.rs"
    click OCR "https://github.com/mediar-ai/screenpipe/blob/main/screenpipe-vision/src/custom_ocr.rs"
    click STT "https://github.com/mediar-ai/screenpipe/blob/main/screenpipe-audio/src/stt.rs"
    click SpeakerID "https://github.com/mediar-ai/screenpipe/blob/main/screenpipe-audio/src/pyannote/identify.rs"
    click VideoProc "https://github.com/mediar-ai/screenpipe/blob/main/screenpipe-server/src/video.rs"
    click VideoCache "https://github.com/mediar-ai/screenpipe/blob/main/screenpipe-server/src/video_cache.rs"
    click DB "https://github.com/mediar-ai/screenpipe/blob/main/screenpipe-server/src/db.rs"
    click DBMigrations "https://github.com/mediar-ai/screenpipe/tree/main/screenpipe-server/src/migrations"
    click PluginMgr "https://github.com/mediar-ai/screenpipe/blob/main/screenpipe-core/src/pipes.rs"
    click PluginSandbox "https://github.com/mediar-ai/screenpipe/tree/main/pipes"

    %% Legend
    subgraph Legend
        UI_L["UI Components"]:::ui
        Core_L["Core Services"]:::core
        Plugin_L["Plugin System"]:::plugin
        Storage_L["Storage"]:::storage
        Processing_L["Processing"]:::processing
    end
```
