How nvidia-nemo/nemotron works
Nemotron is an open-model development repository for building agentic AI models. The principal workflow is: an engineer invokes the CLI, selects a reusable step or recipe, prepares or generates data, trains a model, evaluates and converts it, then exports or deploys the resulting artifacts. The graph preserves documented curation, SDG, BYOB, RL, multimodal, and deployment capabilities. Some recipe-to-runner relationships are represented at subsystem level because only selected source files were sampled.
Main entry points: _runners Training Runners · artifact.py Model Artifacts · api.py Data Preparation · nemotron.py Nemotron CLI
Components 22
User Interface
- Nemotron CLI
src/nemotron/cli/bin/nemotron.py - Step Commands
src/nemotron/cli/commands/steps/run_cmd.py
Pipeline Orchestration
- Step Catalog
src/nemotron/steps/index.py - RunSpec Pipeline
src/nemo_runspec/pipeline.py - Training Recipes
src/nemotron/recipes
Data Lifecycle
- Data Curation
src/nemotron/steps/curate/nemo_curator/step.py - Synthetic Data
src/nemotron/recipes/data/sdg/long-document - Data Preparation
src/nemotron/data_prep/api.py - Preparation Stages
src/nemotron/data_prep/stages - Benchmark Generation
src/nemotron/steps/byob/scripts/runtime.py - MCQ Pipeline
src/nemotron/steps/byob/runtime/benchmark_families/mcq/pipeline.py
Model Lifecycle
- Training Runners
src/nemotron/steps/_runners - Model Recipes
src/nemotron/recipes/super3 - Model Evaluation
src/nemotron/steps/eval/__init__.py - Model Conversion
src/nemotron/steps/convert - Embedding Rerank
src/nemotron/recipes/embed
Runtime Operations
- Artifact Kit
src/nemotron/kit/artifact.py - Export Deployment
src/nemotron/recipes/embed/stage5_deploy/deploy.py - Environment Profiles
src/nemotron/steps/env/env_toml/step.py - Model Artifacts
src/nemotron/kit/artifact.py
Other components
- Engineer
- Slurm Cluster
Connections 24
- Engineer → Nemotron CLI: invokes
- Nemotron CLI → Step Commands: dispatches
- Nemotron CLI → Step Catalog: discovers
- Step Commands → Step Catalog: resolves
- Step Commands → RunSpec Pipeline: launches
- RunSpec Pipeline → Slurm Cluster: submits
- Training Recipes → Data Preparation: prepares
- Training Recipes → Training Runners: trains
- Data Curation → Data Preparation: curates
- Synthetic Data → Data Preparation: generates
- Data Preparation → Preparation Stages: runs
- Preparation Stages → Training Runners: feeds
- Benchmark Generation → MCQ Pipeline: dispatches
- MCQ Pipeline → Model Artifacts: writes
- Training Runners → Model Artifacts: produces
- Training Runners → Model Evaluation: evaluates
- Training Runners → Model Conversion: converts
- Model Evaluation → Model Artifacts: reports
- Model Conversion → Artifact Kit: registers
- Embedding Rerank → Model Evaluation: evaluates
- Embedding Rerank → Export Deployment: deploys
- Artifact Kit → Model Artifacts: tracks
- Environment Profiles → RunSpec Pipeline: configures
- Export Deployment → Model Artifacts: exports