# tjcsl/director4 architecture

> Architecture diagram of the GitHub repository tjcsl/director4, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tjcsl/director4
- Repository: https://github.com/tjcsl/director4

## Overview

##### Project type and purpose
- This is a distributed full-stack platform (“Director 4.0”) for managing and hosting websites at scale.
- It splits into three cooperating microservices:
– manager: a Django-based public web UI and API
– orchestrator: a service that spins up and manages Docker site containers
– router: the edge service on load-balancers handling custom domains and proxying

##### File structure
- Top-level folders:
– manager/: Django project (apps, settings, static, templates, tests)
– orchestrator/: Python service orchestrating Docker (consumers, docker, configs, views)
– router/: Python service for Nginx integration and certbot
– shared/, shell/, resources/: utility libraries and helper scripts
– ci/, docs/, scripts/, vagrant-config/: CI, documentation, provisioning
- Key config/build files:
– Pipfile/Pipfile.lock per component
– fabfile.py, scripts/*.sh for automation
– docs/ARCHITECTURE.md, DEPLOYING.md for deeper architecture notes

##### README for additional insights
- Three primary components listed
- Dependencies: Docker Engine, RabbitMQ, Celery, Redis (Channels), PostgreSQL, Nginx
- References an architecture doc and Vagrant setup

a. Main components
– “User” (browser or API client)
– Nginx reverse-proxy/load-balancer on each service host
– Appservers:
- Manager’s Django appserver (uWSGI/Gunicorn)
- Orchestrator’s appserver (e.g., Flask/Starlette)
– Manager subsystem:
- PostgreSQL database
- RabbitMQ message broker
- Celery worker pool
- Redis channel layer for Django Channels (websockets/notifications)
– Orchestrator subsystem:
- Docker Engine daemon for site container lifecycle
- Docker Registry (image storage)
– Router subsystem:
- Nginx configuration manager
- Certbot (Let's Encrypt) integration
– External services:
- Docker Hub or private registry
- SMTP for email notifications
- DNS (for custom domain routing)

b. Relationships and data flows
1. User → Router Nginx → routed to appropriate site container or Manager UI
2. User → Manager Nginx → Django appserver → PostgreSQL (web UI/API)
3. Manager enqueues long-running tasks → RabbitMQ → Celery workers
4. Manager appserver publishes Channel messages → Redis → websocket clients
5. Orchestrator subscribes to RabbitMQ tasks → orchestrator appserver → Docker Engine
6. Orchestrator logs/events → stored or forwarded to Manager via API or message queue
7. Router monitors site health/status (via API or RabbitMQ/Redis)

c. Architectural patterns and principles
- Microservice decomposition: clear separation of UI, orchestration, and edge routing
- Message-driven decoupling: RabbitMQ + Celery for background jobs
- Pub/Sub for realtime updates: Django Channels + Redis
- Infrastructure as code: Docker + Nginx + Certbot configs + Vagrant for dev
- MVC in Manager: Django apps (auth, sites, docs, request, shell_server)
- Single responsibility for each service (manager vs orchestrator vs router)

d. Key technologies
- Python 3.7+, Django, Celery, RabbitMQ, Redis, PostgreSQL
- Docker Engine + Docker Registry
- Nginx, Certbot
- Vagrant for local/provisional provisioning
- uWSGI/Gunicorn (appserver layer)

##### Detail
- Services (including Celery, Redis, Certbot, Docker Engine)
- How the Manager and Orchestrator appservers communicate via RabbitMQ and/or REST

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External User
    User["User"]:::external

    %% External Services
    subgraph "External Services"
        DNS["DNS"]:::external
        DockerHub["Docker Hub"]:::external
        SMTP["SMTP"]:::external
    end

    %% Router Host
    subgraph "Router Host"
        RouterProxy["Router Nginx (LB)"]:::proxy
        RouterApp["Router Service App"]:::app
        Certbot["Certbot"]:::infra
    end

    %% Manager Host
    subgraph "Manager Host"
        MgmtProxy["Manager Nginx"]:::proxy
        MgmtApp["Manager Appserver"]:::app
        Postgres["PostgreSQL"]:::data
        Rabbit["RabbitMQ Broker"]:::message
        Celery["Celery Workers"]:::app
        Redis["Redis Channel Layer"]:::data
    end

    %% Orchestrator Host
    subgraph "Orchestrator Host"
        OrchApp["Orchestrator Appserver"]:::app
        DockerEng["Docker Engine"]:::infra
        Registry["Docker Registry"]:::infra
        Consumers["Orchestrator Message Consumers"]:::message
    end

    %% Flows
    User -->|DNS resolution| DNS
    DNS --> RouterProxy
    User -->|HTTP| RouterProxy
    RouterProxy -->|Proxies| RouterApp
    RouterApp -->|Obtain certs| Certbot
    RouterProxy -->|Forwards to Manager| MgmtProxy
    RouterProxy -->|Forwards to Sites| OrchApp

    User -->|HTTP| MgmtProxy
    MgmtProxy -->|WSGI/ASGI| MgmtApp
    MgmtApp -->|SQL queries| Postgres
    MgmtApp -->|Enqueue tasks| Rabbit
    MgmtApp -->|Websocket events| Redis
    Rabbit -->|consume tasks| Celery

    Rabbit -->|task events| Consumers
    Consumers -->|invokes endpoints| OrchApp
    OrchApp -->|manages containers| DockerEng
    DockerEng -->|stores images| Registry
    OrchApp -->|publish events| Rabbit

    MgmtApp -->|send emails| SMTP

    DockerHub -->|pull images| Registry

    %% Click Events
    click MgmtApp "https://github.com/tjcsl/director4/blob/master/manager/director/utils/appserver.py"
    click Celery "https://github.com/tjcsl/director4/blob/master/manager/director/celery.py"
    click Redis "https://github.com/tjcsl/director4/blob/master/manager/director/routing.py"
    click Rabbit "https://github.com/tjcsl/director4/blob/master/manager/director/apps/sites/tasks.py"
    click OrchApp "https://github.com/tjcsl/director4/blob/master/orchestrator/orchestrator/app.py"
    click DockerEng "https://github.com/tjcsl/director4/blob/master/orchestrator/orchestrator/docker/containers.py"
    click Registry "https://github.com/tjcsl/director4/blob/master/orchestrator/orchestrator/docker/registry.py"
    click Consumers "https://github.com/tjcsl/director4/tree/master/orchestrator/orchestrator/consumers/"
    click RouterApp "https://github.com/tjcsl/director4/blob/master/router/router/app.py"
    click Certbot "https://github.com/tjcsl/director4/blob/master/router/router/certbot.py"

    %% Styles
    classDef external fill:#f9f,stroke:#333,stroke-width:1px
    classDef proxy fill:#bbdefb,stroke:#1e88e5,stroke-width:1px
    classDef app fill:#c8e6c9,stroke:#2e7d32,stroke-width:1px
    classDef data fill:#ffe0b2,stroke:#ef6c00,stroke-width:1px
    classDef message fill:#e1bee7,stroke:#8e24aa,stroke-width:1px
    classDef infra fill:#e0e0e0,stroke:#424242,stroke-width:1px
```
