How doanva84/nemo1 works
Main Components
• Core Library (“nemo/core”)
– Central classes (ModelPT, Dataset, Exportable, Lightning integration, neural types, connectors).
– Python-based configuration (Hydra-Runner, base_config).
• Domain Collections (“nemo/collections”)
– ASR, TTS, NLP/LLM, VLM/multimodal, Audio, Diffusion, Vision, etc.
– Each collection: data modules, models, losses, metrics, modules, parts (submodules + utils).
• Training Engine (“nemo/lightning”)
– PyTorch Lightning overrides, strategies (TP, PP, FSDP, MoE).
• Export & Deployment (“nemo/export” + “nemo/deploy”)
– ONNX, TensorRT-LLM, Triton, FastAPI microservices, Riva integration, VLLM exporter.
• CLI, Examples & Scripts
– “examples/…” for end-user sample code.
– “scripts/…” for data prep, checkpoint conversion, model training, deployment helper.
– “tools/…” for stand-alone utilities (e.g., CTC segmentation, forced aligner).
• Packaging & Containers
– Dockerfiles under “docker/…”, pip packaging (setup.py, pyproject.toml).
• CI/CD & Quality Checks
– GitHub Actions workflows (.github/workflows) for build/test/publish.
– Linters, code formatting, coverage, labeler, version bump bots.
• External Dependencies & Services
– PyTorch, Hydra, Torch-Lightning, NVIDIA Megatron Core, Transformer Engine.
– Cloud/cluster: AWS EKS, Google GKE, Slurm, NGC containers.
– Model Hub: Hugging Face, NVIDIA NGC.
Relationships & Data Flow
– User invokes an example script (or CLI command) → loads a Hydra config → Core Library instantiates DataModule + ModelPT → Training Engine schedules across GPUs → Model outputs to disk or memory.
– Export component reads a trained checkpoint → converter (ONNX/TensorRT) → exported artifact.
– Deployment service (FastAPI/Triton) loads exported model → serves inference requests from external clients.
– CI pipelines trigger on git events → run linters/tests → build Docker images → publish Python wheel to PyPI and container to NGC.
Architectural Patterns & Principles
– Modular plugin pattern: each “collection” is self-contained but plugs into the Core API.
– Hydra-based configuration: decouples code from config, supports Python-based configs in v2.0.
– Layered architecture:
Layer 1: User Interface (examples, CLI, scripts)
Layer 2: Core & Collections (library code)
Layer 3: Training Engine (Lightning + Megatron)
Layer 4: Export & Deployment
Layer 5: CI/CD & Packaging
– Separation of concerns: data, model, training, export, and deployment all live in separate modules.