# hoarder-app/hoarder architecture

> Architecture diagram of the GitHub repository hoarder-app/hoarder, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hoarder-app/hoarder
- Repository: https://github.com/hoarder-app/hoarder

## Overview

##### Main Components Overview
The system should be divided into these major components:

a. Frontend Applications:
- Web App (Next.js)
- Browser Extensions (Chrome/Firefox)
- Mobile Apps (iOS/Android)
- CLI Interface

b. Backend Services:
- API Layer (tRPC)
- Authentication Service (NextAuth)
- Worker Services
- Asset Preprocessing
- Crawler
- Feed Processing
- OpenAI Integration
- Search Indexing
- Video Processing

c. Storage & Databases:
- Primary Database
- Meilisearch (Full-text search)
- Asset Storage

##### Component Relationships

- Client apps → API Layer (tRPC)
- API Layer → Authentication Service
- API Layer → Storage Services
- Workers ← → Primary Database
- Workers ← → Asset Storage
- Search Service ← → Meilisearch

##### Detailed Component Breakdown

a. Frontend Layer:
**┌─ Client Applications ─────────────┐**
- ┌─────────┐ ┌────────┐ ┌───────┐ │
- Web (Next│ │Browser │ │Mobile │ │
- .js)  │ │  Ext   │ │ Apps  │ │
- ┘ └────────┘ └───────┘ │
- ┬───────────────────┘
- ↓
- [tRPC API Gateway]

b. Backend Services:
**┌─ API & Processing Layer ──────────────────┐**
- ┌─────────┐ ┌──────────┐ ┌─────────────┐ │
- Auth     │ │Bookmark  │ │Worker       │ │
- Service  │ │Processing│ │Orchestration│ │
- ┘ └──────────┘ └─────────────┘ │
- ┬──────────────┬───────────┘
- ↓              ↓
**┌─ Storage Layer ─────────────────────────┐**
- ┌─────────┐ ┌──────────┐ ┌───────────┐ │
- Database │ │Meilisearch│ │Asset      │ │
- (SQL)    │ │(Search)   │ │Storage    │ │
- ┘ └──────────┘ └───────────┘ │
- ┘

##### Additional Elements

Supporting components:
- Load balancers (if applicable)
- Caching layers
- External service connections (OpenAI, etc.)
- Security boundaries
- Monitoring/logging systems

Remember to highlight the self-hosting aspect of the architecture and how components can be deployed in different configurations (Docker, Kubernetes, etc.).

This system design should emphasize the modular nature of the application and how different components can be scaled or modified independently, which is particularly important for a self-hosted application.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Styles
    classDef frontend fill:#2374ab,color:#fff
    classDef backend fill:#48a9a6,color:#fff
    classDef storage fill:#e4cc37,color:#000
    classDef worker fill:#57837b,color:#fff
    classDef auth fill:#c1666b,color:#fff
    classDef external fill:#808080,color:#fff

    %% Frontend Layer
    subgraph Clients
        WebApp["Web App (Next.js)"]:::frontend
        BrowserExt["Browser Extensions"]:::frontend
        MobileApp["Mobile Apps"]:::frontend
        CLI["CLI Interface"]:::frontend
    end

    %% API Layer
    subgraph APILayer
        TRPC["tRPC API Gateway"]:::backend
        AuthService["Authentication Service"]:::auth
        APIRoutes["API Routes"]:::backend
    end

    %% Worker Services
    subgraph Workers
        AssetWorker["Asset Preprocessing"]:::worker
        CrawlerWorker["Crawler"]:::worker
        FeedWorker["Feed Processing"]:::worker
        OpenAIWorker["OpenAI Integration"]:::worker
        SearchWorker["Search Indexing"]:::worker
        VideoWorker["Video Processing"]:::worker
    end

    %% Storage Layer
    subgraph Storage
        DB[(Primary Database)]:::storage
        Search[(Meilisearch)]:::storage
        Assets[(Asset Storage)]:::storage
    end

    %% External Services
    OpenAI["OpenAI/Ollama"]:::external

    %% Connections
    WebApp --> TRPC
    BrowserExt --> TRPC
    MobileApp --> TRPC
    CLI --> TRPC
    
    TRPC --> AuthService
    TRPC --> APIRoutes
    
    APIRoutes --> DB
    APIRoutes --> Search
    APIRoutes --> Assets
    
    Workers --> DB
    Workers --> Assets
    Workers --> Search
    
    OpenAIWorker --> OpenAI

    %% Click Events
    click WebApp "https://github.com/hoarder-app/hoarder/tree/main/apps/web"
    click BrowserExt "https://github.com/hoarder-app/hoarder/tree/main/apps/browser-extension"
    click MobileApp "https://github.com/hoarder-app/hoarder/tree/main/apps/mobile"
    click CLI "https://github.com/hoarder-app/hoarder/tree/main/apps/cli"
    click TRPC "https://github.com/hoarder-app/hoarder/tree/main/packages/trpc"
    click AuthService "https://github.com/hoarder-app/hoarder/tree/main/apps/web/app/api/auth"
    click AssetWorker "https://github.com/hoarder-app/hoarder/blob/main/apps/workers/assetPreprocessingWorker.ts"
    click CrawlerWorker "https://github.com/hoarder-app/hoarder/blob/main/apps/workers/crawlerWorker.ts"
    click FeedWorker "https://github.com/hoarder-app/hoarder/blob/main/apps/workers/feedWorker.ts"
    click OpenAIWorker "https://github.com/hoarder-app/hoarder/blob/main/apps/workers/openaiWorker.ts"
    click SearchWorker "https://github.com/hoarder-app/hoarder/blob/main/apps/workers/searchWorker.ts"
    click VideoWorker "https://github.com/hoarder-app/hoarder/blob/main/apps/workers/videoWorker.ts"
    click DB "https://github.com/hoarder-app/hoarder/tree/main/packages/db"
    click APIRoutes "https://github.com/hoarder-app/hoarder/tree/main/apps/web/app/api"

    %% Legend
    subgraph Legend
        Frontend["Frontend Components"]:::frontend
        BackendAPI["Backend Services"]:::backend
        WorkerService["Worker Services"]:::worker
        StorageSystem["Storage Systems"]:::storage
        AuthComponent["Auth Components"]:::auth
        ExternalService["External Services"]:::external
    end
```
