How nvidia-merlin/dataloader works
Project type and purpose
• This is a Python library (“merlin-dataloader”) for high-performance GPU-accelerated data loading in recommender-system training.
• It provides zero-copy transfers from disk (Parquet files) into GPU memory and then into TensorFlow, PyTorch, or JAX via DLPack.
• Its core use-case is large-scale, distributed recommender-model training with per-epoch shuffling, embedding support, padding, and multi-GPU/distributed backends.
Top-level structure and architectural cues
• merlin/dataloader: core operators, versioning, and framework-agnostic loader base.
• merlin/dataloader/ops: GPU-optimized ops (embeddings, padding).
• merlin/dataloader/utils: framework-specific trainers and wrappers (TensorFlow Horovod, Torch Distributed).
• merlin/loader: thin framework adapters for JAX, TensorFlow, PyTorch.
• examples: usage notebooks showing end-to-end flow.
• docs: Sphinx-based documentation.
• tests: unit and integration tests against framework loaders and examples.
Key README insights
• Inputs: on-disk Parquet dataset → merlin.io.Dataset (not part of this repo, but assumed upstream).
• Loader instantiates with batch_size and framework choice.
• Core flow: read Parquet → GPU buffer → DLPack → framework tensors → model.fit/loop.
Main components
a. Data Source
• Parquet files on disk (HDFS/S3/local FS)
b. Data Reader & Buffering
• merlin.dataloader: reads batches with GPU-side reader (e.g., cuDF or custom reader)
• In-GPU staging buffers
c. Data-Processing Ops
• Embedding lookups, padding, shuffling
• Operate on GPU arrays
d. DLPack Interface
• Zero-copy bridge to framework tensors
e. Framework Loaders
• LoaderTensorFlow: tf.data.Iterator or Keras Sequence
• LoaderTorch: torch.utils.data.DataLoader subclass with distributed support
• LoaderJAX: iterator yielding jax.numpy arrays
f. External Services & Dependencies
• CUDA Runtime, cuDF (if used), DLPack protocol
• Horovod or torch.distributed for distributed training
g. Consumer
• User model training loop (model.fit, training script)
Architectural patterns and design principles
• Plugin/adapter pattern for framework neutrality (loader_base + per-framework implementations).
• Single Responsibility: separate modules for I/O, ops, adapters.
• Zero-copy and in-GPU data movement to minimize CPU ↔ GPU overhead.
• Extensibility: ops directory can be extended with new GPU operations.
• Modular testing: unit tests for each loader and op.
Details to capture
• GPU buffer staging between Reader and Ops.
• DLPack zero-copy transfer boundary.
• The three framework loader paths in parallel.
• Points of extensibility (ops plugins, new frameworks).
• Distributed-training hooks (Horovod/TorchDist).
• Distinguished modules: merlin.dataloader vs merlin.loader.