How pytorch/torchx works
Project Type and Purpose
• TorchX is an open‐source SDK and CLI tool (a universal job launcher) for PyTorch workloads.
• It provides a high-level “Runner + Scheduler + Runtime” abstraction, letting users define ML jobs once and run them on Kubernetes, Slurm, AWS Batch, Docker, GCP Batch, or locally.
• Key use cases: fast iteration during research, end-to-end production ML pipelines.
File Structure (High-Level Layers)
• CLI Layer (torchx/cli): Handles user commands, parses arguments, invokes the Runner API.
• SDK / Core API (torchx/runner, torchx/specs, torchx/apps, torchx/components): Defines JobSpecs, component abstractions, and the Runner interface.
• Scheduling Layer (torchx/schedulers): A plugin model where each scheduler (AWS Batch, Kubernetes, Slurm, Docker, etc.) implements an API for launching jobs.
• Runtime Layer (torchx/runtime): Contains container build logic and tracking instrumentation.
• Workspace Layer (torchx/workspace): Manages code packaging/upload (e.g., Docker images, directory staging).
• Tracking Layer (torchx/tracker): Pluggable backends (e.g., MLflow, fsspec) for capturing metrics, logs, artifacts.
• Utility Modules (torchx/util): Shared helper functions (session, I/O, logging).
• Examples & Tests: Illustrate usage and verify correctness.
README for Insights
• Describes TorchX as a “universal job launcher.”
• Mentions supported schedulers, quickstart link, and dependencies.
Main Components and Their Interactions
a. Actors and Components
– User (CLI or Python SDK)
– CLI Frontend (torchx/cli)
– Runner API (torchx/runner + torchx/specs)
– Component Library (torchx/components + torchx/apps + torchx/pipelines)
– Scheduler Plugins (torchx/schedulers/*)
– Workspace (torchx/workspace)
– Runtime Container Builder (torchx/runtime)
– Tracking Backend (torchx/tracker)
– External Systems: Kubernetes cluster, Slurm cluster, AWS Batch, Docker daemon/registry, GCP Batch, Local OS
b. Data & Control Flows
1. User invokes torchx run … (CLI) or calls Runner.run(...) (Python).
2. CLI → Runner: parse args into a JobSpec.
3. Runner → Scheduler: select appropriate scheduler plugin based on --scheduler flag.
4. Scheduler → Workspace: stage code/artifacts (e.g., build Docker image or prepare directory).
5. Scheduler → Runtime: build container if needed, inject tracking hooks.
6. Scheduler → External System: submit job (K8s Job, Slurm sbatch, AWS Batch job, Docker container run).
7. External System → Tracker: emit status/events/metrics back to TorchX tracking layer.
8. CLI/SDK polls Tracker or watches logs for updates.
c. Architectural Patterns & Principles
– Plugin Architecture: schedulers and trackers are swappable modules following a common API.
– Separation of Concerns: CLI vs. Runner vs. Scheduler vs. Workspace vs. Runtime vs. Tracker.
– Declarative Specs: users define JobSpecs that the runner/scheduler materializes.
– Extensibility: new schedulers, new trackers, new components.