How dmoj/online-judge works
Project Type and Purpose
• This is a full-stack, monolithic Django web application augmented by asynchronous and real-time microservices to form the DMOJ online judge and contest platform.
• The core handles user and contest management, problem rendering, submissions, and administration. Background workers execute judging tasks; a separate judge bridge interfaces with an external judge-server cluster; real-time updates flow over WebSockets.
Main Components
a. Clients and Entry Points
• Web Browser (users) – HTTP(S) and WebSocket clients for site UI, live submission updates, and admin interfaces.
• Load Balancer / Reverse Proxy – distributes incoming HTTP and WS connections to Django app servers or WebSocket servers; can also serve static assets or forward to a CDN.
b. Web Application Layer
• Django Application Servers – handle HTTP requests (views, forms, REST/API), render Jinja2 templates, process authentication (OAuth, two-factor), manage sessions, and serve static/media.
• WSGI / ASGI Interfaces – synchronous WSGI (dmoj/wsgi.py) for HTTP; ASGI or custom WebSocket daemon scripts (websocket/*.js) for real-time channels.
c. Data and Persistence
• Relational Database (PostgreSQL/MySQL) – stores models: users, problems, contests, submissions, scores, etc.
• Caching Layer (Redis/Memcached) – optional; used by Django and Celery for caching querysets, throttles, and session data.
• File Storage – for user uploads, generated PDFs, static resources (could be local or S3).
d. Asynchronous Task Processing
• Celery Workers – process background jobs: submission dispatch, system testing, PDF generation, notifications, event posting.
• Message Broker (RabbitMQ/Redis/AMQP) – queues jobs between Django and Celery; also used by event_poster_amqp.
e. Judge Integration
• Judge Bridge Daemon (judge/bridge/daemon.py) – a long-running process that consumes judging requests from the broker and forwards them to one or more external judge-server instances.
• Judge-Server Cluster (external microservices) – sandboxed execution environments that compile and run submitted code, enforce resource limits, and return results.
f. Real-Time Event Delivery
• Event Poster (judge/event_poster_ws.py or AMQP variant) – publishes status changes (submission progress, contest scoreboard updates) onto WebSocket channels or message broker topics.
• WebSocket Servers / Consumers – maintain persistent connections to browsers and push live updates.
g. External Services
• OAuth Providers (Google, GitHub, Facebook) for social login
• Stanford MOSS for plagiarism detection
• Email / Mailgun for notifications
• MathJax CDN for rendering LaTeX
Relationships and Interactions
1. User → HTTP(S) → Load Balancer → Django App → Database / Cache
2. Django → Celery via Message Broker → Celery Worker → Database / Judge Bridge
3. Judge Bridge ↔ AMQP → Judge-Server Cluster → Judge Bridge → Database
4. Submission status changes → Celery or Django event_poster → Message Broker / WS → WebSocket Server → Web Browser
5. Django → External OAuth / MOSS / Email APIs
Architectural Patterns and Principles
• MVC (Django models-views-templates) for the core web app.
• Asynchronous, event-driven architecture for long-running judge tasks (Celery + message broker).
• Microservices for isolated, scalable code execution (judge-server cluster).
• Pub/Sub pattern for real-time updates (AMQP or Redis channels feeding WebSockets).
• Clear separation of concerns: presentation (HTTP + WS), application logic (Django), background processing (Celery), execution (judge-server), persistent storage.