How johnnyau19/cloud-scheduling works
Project Type and Purpose
• This is a Deep Reinforcement Learning–based job dispatching (cloud scheduling) project.
• Core goal: train an RL agent (Double DQN via Stable-Baselines3) in a custom Gym-style environment to dispatch “jobs” (customers) to “servers” (checkout counters) based on noisy workload estimates, then compare its throughput against a heuristic (EEFT), benchmark runtimes, export the trained model to ONNX, and finally serve it via a FastAPI HTTP interface.
File Structure
Top‐level directories & files:
• job_simulator_env/…
– gym-style env modules (config, job, simulator)
• baseline/eeft.py
• train.py
• evaluate.py
• benchmark_runtime.py
• export_model_onnx.py
• server.py
• model/ (best_model.zip, best_model.onnx)
• dockerfile
• requirements.txt / pyproject.toml
Indicators of architectural choices:
• Separation of concerns between
– Offline pipeline (simulation, training, evaluation, benchmarking)
– Model export
– Online inference service
• Use of ONNX for portable model representation
• Dockerization for reproducible environments
Additional README Insights
• Environment generates job sequences with Gaussian noise on estimated/actual workloads.
• Training uses Double DQN (Stable-Baselines3).
• Evaluation compares RL vs EEFT on identical episodes.
• Real-time plotting with matplotlib.
• Docker images: one for offline scripts (v1), one for FastAPI service (v2).
Main Components
a. job_simulator_env (Simulator)
b. Offline Pipeline
– Training Module (train.py → Stable-Baselines3)
– Baseline Policy Module (baseline/eeft.py)
– Evaluation Module (evaluate.py)
– Benchmarking Module (benchmark_runtime.py)
c. Model Artifacts
– best_model.zip (SB3 checkpoint)
– export_model_onnx.py → best_model.onnx
d. Online Service
– server.py (FastAPI)
– Exposed REST endpoint for “/dispatch”
e. External Clients
– CLI scripts or Web clients calling the API
Component Relationships & Data Flows
1. Simulator generates Job Streams →
2. Training Module consumes streams → produces best_model.zip → stored in model/
3. Baseline Module also consumes streams (with same seed) → computes EEFT actions
4. Evaluation Module runs both agents on identical streams → collects throughput → real-time plots
5. Benchmark Module measures latency of EEFT vs RL
6. Export Module loads best_model.zip → exports best_model.onnx
7. FastAPI Service loads best_model.onnx at startup → listens for client requests → for each request (estimated workload), runs inference → returns selected server index
Architectural Patterns & Design Principles
• Modular separation: environment vs offline training/eval vs model export vs online inference
• Offline vs online pipelines clearly demarcated
• Use of a standard Gym interface for the env to allow plug-and-play of different agents
• ONNX as an interoperability layer for serving
Technologies & Frameworks
• Python 3
• Gym-style custom env
• Stable-Baselines3 (Double DQN)
• ONNX for model export
• FastAPI for REST API
• matplotlib for plotting
• Docker for containerization