# deta/surf architecture

> Architecture diagram of the GitHub repository deta/surf, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/deta/surf
- Repository: https://github.com/deta/surf

## Overview

##### Project Type & Purpose
- Deta Surf is a desktop “full-stack” Electron application combining:
- A Svelte/TypeScript UI (renderer + preload + service libraries)
- An Electron main process for window + IPC + orchestration
- Rust-based local servers (AI “brain” & HTTP API)
- Local data persistence (SFFS flat-file system + SQLite migrations)
- Background workers (image processing, OCR, parsing)

##### Top-Level Architecture Components
a. Electron Layer
- Renderer (app/src/renderer/*): Svelte components for Core, PDF, Resource, Settings
- Preload (app/src/preload/*): TypeScript bridge exposing safe IPC & service APIs
- Main (app/src/main/*): Bootstraps windows, manages menus, permissions, web requests, launches workers
b. Service Libraries (packages/services & packages/types)
- UI-side business logic for AI prompts, context management, IPC messaging, notebooks, tabs
c. Rust Servers
- packages/backend-server: Rust HTTP server exposing endpoints for embeddings, AI interactions, KV, messaging
- packages/backend: Rust library for AI “brain”, LLM clients, session/store management, worker handlers
d. Data Storage & Workers
- SFFS (Surf Flat File System): local open-format library storage
- SQLite migrations (packages/backend/migrations)
- Workers: imageProcessor.ts and Rust-spawned background tasks (OCR, parsing)
e. External Services
- LLM providers (OpenAI, local LLMs, cloud models)
- Web services (YouTube, webpages, RSS) via web-parser
- Browser context (via webContents)

##### Key Interactions & Data Flows
1. User clicks UI button → Renderer Svelte component
2. Renderer → Preload exposes IPC call or service method
3. Preload → Main process (via Electron’s IPC)
4. Main → Either:
- Directly invoke local worker (imageProcessor, OCR)
- HTTP call to local Rust server (backend-server)
5. Rust server → embeds & AI “brain” library (packages/backend) → persists to SQLite/SFFS
6. Rust server → returns JSON/stream to Main
7. Main → Preload (IPC response) → Renderer updates UI

##### Architectural Patterns & Principles
- Layered architecture (UI → Preload → Main → Backend → Data)
- Separation of concerns: UI vs. business logic vs. data store
- Local-first philosophy: all data and AI inference run locally when possible
- Monorepo with scope-based packages (services, types, core UI, Rust backends)
- Use of IPC for secure context bridging
- Worker threads & spawned Rust processes for CPU-bound tasks

##### Final Advice
- Keep details: mention workers (OCR, imageProcessor), migrations, service handlers
- Separation: UI, IPC, orchestration, AI backend, storage, external

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% UI Layer
    subgraph "UI Layer"
        UI["UI Renderer (Svelte)"]:::ui
        Services["Service Libraries"]:::service
        Types["Shared Types"]:::service
    end

    %% IPC Layer
    subgraph "IPC Layer"
        Preload["Preload Bridge (IPC Layer)"]:::ipc
    end

    %% Main Process
    subgraph "Electron Main Process"
        Main["Electron Main Process"]:::main
        SFFS1["SFFS Storage (main)"]:::data
    end

    %% Backend Services
    subgraph "Backend Services"
        RustHTTP["Rust HTTP Server"]:::backend
        AILib["AI “Brain” Library"]:::backend
        LLMapi["LLM API Clients"]:::backend
        LLMlocal["Local LLM Interfaces"]:::backend
        WebParser["Web Parser Microservice"]:::backend
        Migrations["SQLite Migrations"]:::data
        WorkerHandlers["Background Worker Handlers"]:::worker
    end

    %% Workers
    subgraph "Workers"
        ImgWorker["Image Processing Worker"]:::worker
        OCR["OCR Models"]:::worker
    end

    %% External Services
    subgraph "External Services"
        ExternalLLM["External LLM Providers"]:::external
        WebServices["YouTube/Web/RSS"]:::external
        BrowserCtx["Browser Context"]:::external
    end

    %% Connections
    UI -->|"uses"| Services
    UI -->|"uses"| Types
    UI -->|"user action"| Preload
    Preload -->|"IPC.send/handle"| Main
    Main -->|"spawn(imageProcessor)"| ImgWorker
    Main -->|"spawn(ocr)"| OCR
    Main -->|"- - HTTP POST /embeddings/- -"| RustHTTP
    RustHTTP <-->|"stream AI completions"| Main
    RustHTTP -->|"internal call"| AILib
    AILib -->|"uses"| LLMapi
    AILib -->|"uses"| LLMlocal
    AILib -->|"persist"| SFFS1
    AILib -->|"migrate"| Migrations
    Main -->|"respond IPC"| Preload
    Preload -->|"update UI"| UI
    Services -->|"IPC helpers"| Preload
    Types -->|"types for"| Services
    Types -->|"types for"| UI
    Main -->|"uses storage"| SFFS1
    Main -->|"calls"| WebParser
    LLMapi -->|"- - call external LLM - -"| ExternalLLM
    UI -->|"webview"| BrowserCtx
    Main -->|"webRequests"| BrowserCtx
    Main -->|"schedule handlers"| WorkerHandlers

    %% Click Events
    click UI "https://github.com/deta/surf/tree/main/app/src/renderer"
    click Services "https://github.com/deta/surf/tree/main/packages/services/src/lib"
    click Types "https://github.com/deta/surf/tree/main/packages/types/src"
    click Preload "https://github.com/deta/surf/tree/main/app/src/preload"
    click Main "https://github.com/deta/surf/tree/main/app/src/main"
    click RustHTTP "https://github.com/deta/surf/tree/main/packages/backend-server/src"
    click AILib "https://github.com/deta/surf/tree/main/packages/backend/src/ai/brain"
    click LLMapi "https://github.com/deta/surf/tree/main/packages/backend/src/ai/llm"
    click LLMlocal "https://github.com/deta/surf/tree/main/packages/backend/src/ai/local"
    click SFFS1 "https://github.com/deta/surf/blob/main/app/src/main/sffs.ts"
    click SFFS2 "https://github.com/deta/surf/blob/main/packages/services/src/lib/sffs.ts"
    click Migrations "https://github.com/deta/surf/tree/main/packages/backend/migrations"
    click ImgWorker "https://github.com/deta/surf/blob/main/app/src/main/workers/imageProcessor.ts"
    click OCR "https://github.com/deta/surf/tree/main/app/resources/ocrs"
    click OCR "https://github.com/deta/surf/tree/main/app/resources/tessdata"
    click WorkerHandlers "https://github.com/deta/surf/tree/main/packages/backend/src/worker"
    click WebParser "https://github.com/deta/surf/tree/main/packages/web-parser/src"

    %% Styles
    classDef ui fill:#D0EBFF,stroke:#0B74DE
    classDef ipc fill:#A5F3E2,stroke:#0B6E58
    classDef main fill:#A8D0FF,stroke:#1C3AA9
    classDef backend fill:#D2F8D2,stroke:#228B22
    classDef service fill:#FFF4C2,stroke:#D4A017
    classDef data fill:#FFF3B0,stroke:#C0A000
    classDef worker fill:#FFE4B5,stroke:#ED9121
    classDef external fill:#E0E0E0,stroke:#A0A0A0
```
