# dropbox/hackpad architecture

> Architecture diagram of the GitHub repository dropbox/hackpad, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dropbox/hackpad
- Repository: https://github.com/dropbox/hackpad

## Overview

##### Project Type and Purpose
- Hackpad is a full-stack, real-time collaborative wiki built on top of the open-source Etherpad editor.
- It comprises multiple clients (web browser, iOS app, macOS app), a Node.js based real-time server, supporting infrastructure services in Java/Scala, databases (MySQL, Solr), caches, static asset servers, and external integrations (OAuth providers, Dropbox, APNs, Stripe, email, analytics).

##### High-Level Components
A. Clients
- Web browser + Static assets (HTML/CSS/JS served by Node.js or CDN)
- Native iOS app (Objective-C code under client/ios/Hackpad, HackpadKit for sync)
- Native macOS app (Objective-C code under client/osx/Hackpad)
B. Load Balancer & Reverse Proxy
- NGINX (contrib/nginx.conf) or Docker routed entrypoints (bin/docker-entrypoint.sh)
C. API & Real-Time Servers
- Node.js/Etherpad server (etherpad/src/main.js + control/, collab/ modules)
- Java/Scala OpenOffice conversion microservice (infrastructure/com.etherpad.openofficeservice)
D. Data Stores & Search
- MySQL (pad content, users, migrations under etherpad/src/etherpad/db_migrations)
- Solr (full-text search, under etherpad/solr)
E. Caching & Message Layers
- Memcached (s3 cache for assets, under infrastructure/lib)
- In-process pub/sub for collaboration (Etherpad’s easysync2)
F. Static Assets & Themes
- Static file server or CDN (etherpad/src/static, themes under src/themes)
- Asset pipeline tools (YUICOMPRESSOR, LESS under infrastructure/yuicompressor)
G. Background & Scheduled Jobs
- Cron scripts (contrib/cron) for cleanup, migrations
- Runit or Docker “run.sh” wrappers (contrib/runit, bin/run.sh)
H. External Integrations
- OAuth2 / Facebook SDK / Google SDK
- Dropbox sync (importexport/dropbox.js)
- APNs (java-apns)
- Stripe (billing tables, pro features)
- Email gateway, Mixpanel analytics

##### Relationships and Data Flows
- Web & native clients ←→ Load Balancer / Proxy via HTTPS
- Static assets (CSS/JS) served directly
- API calls (REST endpoints under control/api_v1_control.js)
- WebSocket (or long-polling) connections for real-time collab (collab_server.js)
- Node.js server ↔ MySQL for read/write pad data
- Node.js server ↔ Solr via HTTP for indexing/search
- Node.js server ↔ Memcached for session storage and caching
- Node.js server → Background conversion service over internal RPC/HTTP
- Background jobs (cron/runit) → Database maintenance, migrations
- Node.js server → External OAuth providers (Facebook, Google), email & billing services

##### Architectural Patterns & Principles
- Layered MVC: controllers in control/, models in pad/, helpers under helpers.js
- Event-driven: collaboration over websockets, internal pub/sub, asynchronous I/O
- Modular: Etherpad’s module system (framework-src/modules), plug-in architecture
- Microservice for heavy tasks (OpenOffice conversion in Scala/Java)
- Static asset pipeline: LESS → CSS, YUICompressor → minified JS/CSS

##### Essential Elements to Capture
- Round-cornered rectangles for each component with a title (e.g., “Etherpad Node.js Cluster”)
- Boundary lines showing trust zones (public internet vs. internal network vs. datacenter)

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Public Internet"
        Browser["Web Browser"]:::client
        IOS["iOS App"]:::client
        Mac["macOS App"]:::client
    end

    LB["Load Balancer / Proxy"]:::server

    subgraph "Application Layer"
        NodeCluster["Etherpad Node.js Cluster"]:::server
        OOService["OpenOffice Conversion Service"]:::server
    end

    StaticCDN["Static CDN / Asset Server"]:::server

    subgraph "Data Layer"
        MySQL[(MySQL<br/>3306)]:::database
        Solr[(Solr<br/>9001)]:::database
        Memcached[(Memcached)]:::database
    end

    subgraph "Background Jobs"
        Cron["Cron Scripts"]:::server
        Runit["Runit Wrappers"]:::server
        RunSh["Shell Wrapper"]:::server
    end

    subgraph "External Services"
        OAuth["OAuth2 Providers"]:::external
        Dropbox["Dropbox Sync"]:::external
        APNs["APNs Service"]:::external
        Stripe["Stripe Billing"]:::external
        Email["Email Gateway"]:::external
        Analytics["Analytics"]:::external
    end

    Browser -->|HTTPS| LB
    IOS -->|HTTPS| LB
    Mac -->|HTTPS| LB

    LB -->|routes traffic| NodeCluster
    LB -->|serves static| StaticCDN

    NodeCluster -->|read/write| MySQL
    NodeCluster -->|index/search| Solr
    NodeCluster -->|cache| Memcached
    NodeCluster -->|RPC/HTTP| OOService
    NodeCluster -->|trigger jobs| Cron
    NodeCluster -->|trigger jobs| Runit
    NodeCluster -->|trigger jobs| RunSh
    NodeCluster -->|OAuth2 auth| OAuth
    NodeCluster -->|sync| Dropbox
    NodeCluster -->|push| APNs
    NodeCluster -->|billing| Stripe
    NodeCluster -->|send emails| Email
    NodeCluster -->|track events| Analytics

    click Browser "https://github.com/dropbox/hackpad/tree/master/etherpad/src/static"
    click IOS "https://github.com/dropbox/hackpad/tree/master/client/ios/Hackpad"
    click Mac "https://github.com/dropbox/hackpad/tree/master/client/osx/Hackpad"
    click LB "https://github.com/dropbox/hackpad/blob/master/contrib/nginx.conf"
    click LB "https://github.com/dropbox/hackpad/blob/master/bin/docker-entrypoint.sh"
    click NodeCluster "https://github.com/dropbox/hackpad/blob/master/etherpad/src/main.js"
    click OOService "https://github.com/dropbox/hackpad/blob/master/infrastructure/com.etherpad.openofficeservice"
    click MySQL "https://github.com/dropbox/hackpad/tree/master/etherpad/src/etherpad/db_migrations"
    click Solr "https://github.com/dropbox/hackpad/tree/master/etherpad/solr/collection1"
    click Memcached "https://github.com/dropbox/hackpad/blob/master/infrastructure/lib/memcached-2.6rc1.jar"
    click Cron "https://github.com/dropbox/hackpad/tree/master/contrib/cron/cron.daily/etherpad-remove-ooconvert"
    click Runit "https://github.com/dropbox/hackpad/tree/master/contrib/runit/run"
    click RunSh "https://github.com/dropbox/hackpad/blob/master/bin/run.sh"
    click OAuth "https://github.com/dropbox/hackpad/blob/master/infrastructure/framework-src/modules/oauth.js"
    click Dropbox "https://github.com/dropbox/hackpad/blob/master/etherpad/src/etherpad/importexport/dropbox.js"
    click APNs "https://github.com/dropbox/hackpad/blob/master/etherpad/src/etherpad/pro/pro_apns.js"
    click Stripe "https://github.com/dropbox/hackpad/blob/master/etherpad/src/etherpad/db_migrations/m0058_add_stripeJson.js"
    click Email "https://github.com/dropbox/hackpad/blob/master/infrastructure/framework-src/modules/email.js"
    click Analytics "https://github.com/dropbox/hackpad/tree/master/etherpad/src/etherpad/statistics"

    classDef client fill:#add8e6,stroke:#000,stroke-width:1px
    classDef server fill:#90ee90,stroke:#000,stroke-width:1px
    classDef database fill:#ffa500,stroke:#000,stroke-width:1px
    classDef external fill:#dda0dd,stroke:#000,stroke-width:1px
```
