# rapidpro/mailroom architecture

> Architecture diagram of the GitHub repository rapidpro/mailroom, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rapidpro/mailroom
- Repository: https://github.com/rapidpro/mailroom

## Overview

##### Main Components

a. Core Services:
- Web Server (HTTP API Layer)
- Message Processing Engine (Mailroom Core)
- Flow Engine (GoFlow)
- Task Workers/Handlers
- Queue Management System

b. External Dependencies:
- PostgreSQL Database (Primary + Read-only replica)
- Redis
- ElasticSearch
- AWS S3
- SMTP Server
- Firebase Cloud Messaging
- External Services (Courier, RapidPro)

##### Component Relationships

**[External Clients] → [Web Server Layer (/:8090)]**
- ↓
**[RapidPro Platform] ← [Core Services]**
- ↓
**[Message Engine] ← → [Flow Engine (GoFlow)]**
- ↓
**[Task Workers] → [Queue Management]**
- ↓
**[Service Handlers]**
- ↓
**[External Services Layer]**

##### Key Architectural Elements

a. Data Flow Paths:
- Task distribution through the queue system
- Data persistence flows to databases
- Webhook and external service interactions

- Core Services (Rectangle): Web, Engine, Workers
- External Services (Cloud): Courier, FCM, SMTP
- Queue System (Queue symbol): Redis-based queuing
- Processing Units (Hexagon): Task handlers, Flow engine

##### Specific Implementation Details

a. Core Modules (group together):
- Models (/core/models)
- Handlers (/core/handlers)
- Tasks (/core/tasks)
- Queue Management (/core/queue)
- Runner (/core/runner)

b. Service Integration Points:
- IVR Services (Twiml, Vonage)
- Ticket Services (Zendesk, RocketChat, Mailgun)
- Message Services (Courier integration)

##### Important Architectural Patterns

- Worker Pool Pattern: Show task distribution system
- Queue-based Architecture: Highlight message processing flow
- Service Layer Pattern: External service integration
- Repository Pattern: Data access layer

Top Level:
**[Client Layer] → [API Gateway]**
- ↓
**[Processing Layer]**
- ↓
**[Storage Layer]**

Middle Layer (Processing):
**[Task Handlers] ← → [Queue Manager]**
- ↓
**[Flow Engine] ← → [Message Processor]**

Bottom Layer (Storage/External):
**[PostgreSQL] [Redis] [ElasticSearch] [S3]**
**[External Services: Courier, FCM, SMTP]**

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph "Client Layer"
        CL[External Clients]:::client
        RP[RapidPro Platform]:::client
    end

    subgraph "API Gateway"
        WS["Web Server (HTTP API)"]:::server
        click WS "https://github.com/rapidpro/mailroom/blob/main/web/server.go"
    end

    subgraph "Processing Layer"
        subgraph "Core Engine"
            ME["Message Processing Engine"]:::core
            click ME "https://github.com/rapidpro/mailroom/blob/main/core/msgio/send.go"
            FE["Flow Engine (GoFlow)"]:::core
            click FE "https://github.com/rapidpro/mailroom/blob/main/core/goflow/engine.go"
        end

        subgraph "Task Management"
            TW["Task Workers"]:::worker
            click TW "https://github.com/rapidpro/mailroom/blob/main/core/tasks/handler/worker.go"
            QM["Queue Management"]:::queue
            click QM "https://github.com/rapidpro/mailroom/blob/main/core/queue/queue.go"
        end

        subgraph "Service Handlers"
            WH["Webhook Handler"]:::handler
            click WH "https://github.com/rapidpro/mailroom/blob/main/core/handlers/webhook_called.go"
            MH["Message Handler"]:::handler
            click MH "https://github.com/rapidpro/mailroom/blob/main/core/handlers/msg_created.go"
            CH["Contact Handler"]:::handler
            click CH "https://github.com/rapidpro/mailroom/blob/main/core/handlers/contact_groups_changed.go"
        end
    end

    subgraph "Storage Layer"
        DB[(PostgreSQL)]:::storage
        RD[(Redis)]:::storage
        ES[(ElasticSearch)]:::storage
        S3[("AWS S3")]:::storage
    end

    subgraph "External Services"
        CR["Courier"]:::external
        FCM["Firebase Cloud\nMessaging"]:::external
        SMTP["SMTP Server"]:::external
        IVR["IVR Services"]:::external
        click IVR "https://github.com/rapidpro/mailroom/tree/main/services/ivr"
        TKT["Ticket Services"]:::external
        click TKT "https://github.com/rapidpro/mailroom/tree/main/services/tickets"
    end

    %% Connections
    CL --> WS
    RP --> WS
    WS --> ME
    ME <--> FE
    ME --> TW
    TW --> QM
    QM --> WH & MH & CH
    WH & MH & CH --> CR
    ME --> DB
    ME --> RD
    ME --> ES
    ME --> S3
    CH --> FCM
    MH --> SMTP
    WH --> IVR
    CH --> TKT

    %% Styles
    classDef client fill:#E1F5FE,stroke:#0288D1
    classDef server fill:#E8F5E9,stroke:#388E3C
    classDef core fill:#FFF3E0,stroke:#F57C00
    classDef worker fill:#F3E5F5,stroke:#7B1FA2
    classDef queue fill:#FFEBEE,stroke:#D32F2F
    classDef handler fill:#E8EAF6,stroke:#3949AB
    classDef storage fill:#ECEFF1,stroke:#546E7A
    classDef external fill:#FCE4EC,stroke:#C2185B
```
