How tjcsl/director4 works
Project type and purpose
• This is a distributed full-stack platform (“Director 4.0”) for managing and hosting websites at scale.
• It splits into three cooperating microservices:
– manager: a Django-based public web UI and API
– orchestrator: a service that spins up and manages Docker site containers
– router: the edge service on load-balancers handling custom domains and proxying
File structure
• Top-level folders:
– manager/: Django project (apps, settings, static, templates, tests)
– orchestrator/: Python service orchestrating Docker (consumers, docker, configs, views)
– router/: Python service for Nginx integration and certbot
– shared/, shell/, resources/: utility libraries and helper scripts
– ci/, docs/, scripts/, vagrant-config/: CI, documentation, provisioning
• Key config/build files:
– Pipfile/Pipfile.lock per component
– fabfile.py, scripts/*.sh for automation
– docs/ARCHITECTURE.md, DEPLOYING.md for deeper architecture notes
README for additional insights
• Three primary components listed
• Dependencies: Docker Engine, RabbitMQ, Celery, Redis (Channels), PostgreSQL, Nginx
• References an architecture doc and Vagrant setup
a. Main components
– “User” (browser or API client)
– Nginx reverse-proxy/load-balancer on each service host
– Appservers:
• Manager’s Django appserver (uWSGI/Gunicorn)
• Orchestrator’s appserver (e.g., Flask/Starlette)
– Manager subsystem:
• PostgreSQL database
• RabbitMQ message broker
• Celery worker pool
• Redis channel layer for Django Channels (websockets/notifications)
– Orchestrator subsystem:
• Docker Engine daemon for site container lifecycle
• Docker Registry (image storage)
– Router subsystem:
• Nginx configuration manager
• Certbot (Let's Encrypt) integration
– External services:
• Docker Hub or private registry
• SMTP for email notifications
• DNS (for custom domain routing)
b. Relationships and data flows
1. User → Router Nginx → routed to appropriate site container or Manager UI
2. User → Manager Nginx → Django appserver → PostgreSQL (web UI/API)
3. Manager enqueues long-running tasks → RabbitMQ → Celery workers
4. Manager appserver publishes Channel messages → Redis → websocket clients
5. Orchestrator subscribes to RabbitMQ tasks → orchestrator appserver → Docker Engine
6. Orchestrator logs/events → stored or forwarded to Manager via API or message queue
7. Router monitors site health/status (via API or RabbitMQ/Redis)
c. Architectural patterns and principles
• Microservice decomposition: clear separation of UI, orchestration, and edge routing
• Message-driven decoupling: RabbitMQ + Celery for background jobs
• Pub/Sub for realtime updates: Django Channels + Redis
• Infrastructure as code: Docker + Nginx + Certbot configs + Vagrant for dev
• MVC in Manager: Django apps (auth, sites, docs, request, shell_server)
• Single responsibility for each service (manager vs orchestrator vs router)
d. Key technologies
• Python 3.7+, Django, Celery, RabbitMQ, Redis, PostgreSQL
• Docker Engine + Docker Registry
• Nginx, Certbot
• Vagrant for local/provisional provisioning
• uWSGI/Gunicorn (appserver layer)
Detail
• Services (including Celery, Redis, Certbot, Docker Engine)
• How the Manager and Orchestrator appservers communicate via RabbitMQ and/or REST