How dptech-corp/uni-core works
Project type and purpose
• Uni-Core is a high-performance, distributed PyTorch framework optimized for Transformer-based models.
• It provides Python APIs, command-line tools, fused CUDA kernels, distributed multi-GPU/multi-node training, mixed-precision support, optimizers, schedulers, data loaders, logging, checkpointing, and Docker support.
File structure
• Top-level directories:
• csrc: native C++/CUDA extension sources (kernels for Adam, layernorm, softmax_dropout, etc.)
• unicore: core Python package (subpackages: data, distributed, logging, losses, models, modules, optim, tasks, trainer, utils)
• unicore_cli: command-line entrypoints (train.py)
• examples/bert: reference example showing how to instantiate datasets, models, tasks, and start training
• docker: Dockerfiles for RDMA/GPU environments
• tests: unit tests for Python modules
• .github/workflows: CI for building wheels across CUDA versions, publishing, and Docker
• setup.py, requirements.txt: packaging and dependencies
• Patterns:
• Python package with clear separation of concerns (data / model / module / task / training orchestrator)
• Native extensions for performance-critical kernels, loaded at runtime
• CLI wrapper around Python API for user entrypoint
• CI pipelines for multi-CUDA builds and releases
README for additional insights
• Confirms project features: distributed training, mixed precision, fused CUDA, checkpoint management, logging, buffered data loader, gradient accumulation, optimizers, LR schedulers
• Mentions dependency on PyTorch and matching CUDA version
• References Docker image and example usage
a. Main components
1. User Interfaces
• CLI (unicore_cli/train.py)
• Example scripts (examples/bert/model.py, task.py)
2. Uni-Core Python Library (unicore/…)
• Data Layer: data loaders & datasets (unicore/data)
• Model Layer: modules & model abstractions (unicore/modules, unicore/models)
• Task Layer: high-level training definitions (unicore/tasks)
• Trainer: orchestration of epoch/step loop, gradient accumulation (unicore/trainer.py)
• Optimization: optimizers & schedulers (unicore/optim)
• Logging: metrics, progress bars (unicore/logging)
• Checkpointing & utils (unicore/checkpoint_utils, unicore/utils)
• Distributed Support: wrappers for DDP, multi-node utils (unicore/distributed)
3. Native Extension
• csrc: CUDA kernels compiled into shared objects loaded by Python
• Fused ops: Adam, layernorm, RMSNorm, softmax+dropout, rounding, multi-tensor
4. External Dependencies
• PyTorch (autograd, DDP, mixed precision API)
• CUDA runtime & GPU hardware
• MPI/RDMA (via docker/rdma) for multi-node communication
5. Infrastructure
• Docker environment for consistent GPU/OS setup
• GitHub Actions for building/publishing wheels
b. Relationships and interactions
• User invokes CLI or example script → Python API
• Python API uses Data Layer to feed mini-batches
• Model Layer builds computational graph with PyTorch modules
• Trainer orchestrates forward/backward passes, calls fused CUDA kernels via native extension
• Optimizer & LR scheduler update parameters (FP32/FP16/bf16)
• DDP wrappers distribute training loops across GPUs/nodes, use CUDA kernels and RDMA for communication
• Logging & checkpointing record metrics and save model state
c. Architectural patterns and principles
• Modular layer separation (Data, Model, Task, Trainer, Optim, Logging)
• Extension of PyTorch (wrapper pattern) rather than reinventing tensor APIs
• Native plugin for hot-path kernels (performance)
• CLI + Python API dual entrypoints for flexibility
• CI/CD ensures multi-CUDA compatibility and reliable releases
d. Key technologies, frameworks, libraries
• Python 3.x, PyTorch
• CUDA (cu102–cu118)
• C++/CUDA fusion kernels
• Docker + nvidia-docker
• GitHub Actions for cross-CUDA builds