How amzn/mrc-net-6d-pose works
Project Type and Purpose
• This is an open-source research codebase implementing “MRC-Net: 6-DoF Pose Estimation with MultiScale Residual Correlation.”
• Core purpose: end-to-end two-stage Siamese network for object pose estimation (classification → rendering-guided regression), with multiscale feature correlation between real and rendered images.
Top-Level File Structure Analysis
• scripts/
• install_deps.sh – environment setup (conda, pip)
• prepare_data.sh – dataset download & cache generation
• run_training.sh – multi-GPU training launch
• run_inference.sh – inference with pretrained weights
• lib/
• data_utils.py – dataset loader, batching, augmentation
• warmup_lr.py – learning-rate scheduler utility
• Core modules at project root:
• bop_dataset.py – dataset‐specific reader (BOP format)
• precompute_quaternion_labels.py – discretize orientations for classification stage
• models.py – model definitions (classification branch, regression branch, multiscale correlation layers)
• inference.py – inference logic (load model, render input, forward pass, output pose)
• main.py – training entrypoint (parses config, builds model, training loop)
• config.py – hyperparameters and paths
• utils.py – miscellaneous helpers (logging, checkpointing)
• misc/
Architectural Patterns and Design Principles
• Two-stage pipeline (classification → rendering → regression)
• Siamese network structure: parallel feature extractors on real versus rendered images
• Multiscale residual correlation: cross-correlate feature maps at several scales
• Modular design: clear separation of data loading, model definition, training loop, and inference
• Script-driven workflow: reproducible environment setup, data prep, train, infer
Main Components
A. Environment & Scripts Layer
B. Data Layer
• “Raw Dataset (BOP format)” node
• “Precomputed Quaternion Labels” node (precompute_quaternion_labels.py)
• “Data Loader & Augmentations” (lib/data_utils.py)
C. Training Pipeline
• “Config Manager” (config.py)
• “Model Builder” (models.py)
• “Classification Stage”
• “Renderer” (external module called by models or inference)
• “Regression Stage”
• “Loss & Optimizer” (warmup_lr.py + training loop in main.py)
• “Checkpointing & Logging” (utils.py)
D. Inference Pipeline
• “Load Checkpoint & Config” (scripts/run_inference.sh + inference.py)
• “Render Classification Output”
• “Feature Extraction & Multiscale Correlation”
• “Pose Output”
E. External Dependencies
• PyTorch, CUDA/GPUs
• Rendering library (e.g., PyOpenGL, pyrender)
• Python environment (requirements.txt)