How hoarder-app/hoarder works
Main Components Overview
The system should be divided into these major components:
a. Frontend Applications:
• Web App (Next.js)
• Browser Extensions (Chrome/Firefox)
• Mobile Apps (iOS/Android)
• CLI Interface
b. Backend Services:
• API Layer (tRPC)
• Authentication Service (NextAuth)
• Worker Services
• Asset Preprocessing
• Crawler
• Feed Processing
• OpenAI Integration
• Search Indexing
• Video Processing
c. Storage & Databases:
• Primary Database
• Meilisearch (Full-text search)
• Asset Storage
Component Relationships
• Client apps → API Layer (tRPC)
• API Layer → Authentication Service
• API Layer → Storage Services
• Workers ← → Primary Database
• Workers ← → Asset Storage
• Search Service ← → Meilisearch
Detailed Component Breakdown
a. Frontend Layer:
┌─ Client Applications ─────────────┐
• ┌─────────┐ ┌────────┐ ┌───────┐ │
• Web (Next│ │Browser │ │Mobile │ │
• .js) │ │ Ext │ │ Apps │ │
• ┘ └────────┘ └───────┘ │
• ┬───────────────────┘
• ↓
• [tRPC API Gateway]
b. Backend Services:
┌─ API & Processing Layer ──────────────────┐
• ┌─────────┐ ┌──────────┐ ┌─────────────┐ │
• Auth │ │Bookmark │ │Worker │ │
• Service │ │Processing│ │Orchestration│ │
• ┘ └──────────┘ └─────────────┘ │
• ┬──────────────┬───────────┘
• ↓ ↓
┌─ Storage Layer ─────────────────────────┐
• ┌─────────┐ ┌──────────┐ ┌───────────┐ │
• Database │ │Meilisearch│ │Asset │ │
• (SQL) │ │(Search) │ │Storage │ │
• ┘ └──────────┘ └───────────┘ │
• ┘
Additional Elements
Supporting components:
• Load balancers (if applicable)
• Caching layers
• External service connections (OpenAI, etc.)
• Security boundaries
• Monitoring/logging systems
Remember to highlight the self-hosting aspect of the architecture and how components can be deployed in different configurations (Docker, Kubernetes, etc.).
This system design should emphasize the modular nature of the application and how different components can be scaled or modified independently, which is particularly important for a self-hosted application.