# amzn/mrc-net-6d-pose architecture

> Architecture diagram of the GitHub repository amzn/mrc-net-6d-pose, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/amzn/mrc-net-6d-pose
- Repository: https://github.com/amzn/mrc-net-6d-pose

## Overview

##### Project Type and Purpose
- This is an open-source research codebase implementing “MRC-Net: 6-DoF Pose Estimation with MultiScale Residual Correlation.”
- Core purpose: end-to-end two-stage Siamese network for object pose estimation (classification → rendering-guided regression), with multiscale feature correlation between real and rendered images.

##### Top-Level File Structure Analysis
- scripts/
- install_deps.sh – environment setup (conda, pip)
- prepare_data.sh – dataset download & cache generation
- run_training.sh – multi-GPU training launch
- run_inference.sh – inference with pretrained weights
- lib/
- data_utils.py – dataset loader, batching, augmentation
- warmup_lr.py – learning-rate scheduler utility
- Core modules at project root:
- bop_dataset.py – dataset‐specific reader (BOP format)
- precompute_quaternion_labels.py – discretize orientations for classification stage
- models.py – model definitions (classification branch, regression branch, multiscale correlation layers)
- inference.py – inference logic (load model, render input, forward pass, output pose)
- main.py – training entrypoint (parses config, builds model, training loop)
- config.py – hyperparameters and paths
- utils.py – miscellaneous helpers (logging, checkpointing)
- misc/

##### Architectural Patterns and Design Principles
- Two-stage pipeline (classification → rendering → regression)
- Siamese network structure: parallel feature extractors on real versus rendered images
- Multiscale residual correlation: cross-correlate feature maps at several scales
- Modular design: clear separation of data loading, model definition, training loop, and inference
- Script-driven workflow: reproducible environment setup, data prep, train, infer

##### Main Components
A. Environment & Scripts Layer
B. Data Layer
- “Raw Dataset (BOP format)” node
- “Precomputed Quaternion Labels” node (precompute_quaternion_labels.py)
- “Data Loader & Augmentations” (lib/data_utils.py)
C. Training Pipeline
- “Config Manager” (config.py)
- “Model Builder” (models.py)
- “Classification Stage”
- “Renderer” (external module called by models or inference)
- “Regression Stage”
- “Loss & Optimizer” (warmup_lr.py + training loop in main.py)
- “Checkpointing & Logging” (utils.py)
D. Inference Pipeline
- “Load Checkpoint & Config” (scripts/run_inference.sh + inference.py)
- “Render Classification Output”
- “Feature Extraction & Multiscale Correlation”
- “Pose Output”
E. External Dependencies
- PyTorch, CUDA/GPUs
- Rendering library (e.g., PyOpenGL, pyrender)
- Python environment (requirements.txt)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Setup & Data Layer
    subgraph "Setup & Data" 
        direction TB
        EnvSetup["Environment Setup"]:::setup
        DataPrep["Data Preparation"]:::setup
    end

    subgraph "Data Layer"
        direction TB
        RawData["Raw Dataset (BOP format)"]:::data
        PrecompLabels["Precomputed Quaternion Labels"]:::data
        DataLoader["Data Loader & Augmentations"]:::data
    end

    %% Training Pipeline
    subgraph "Training Pipeline" 
        direction TB
        ConfigMgr["Config Manager"]:::training
        ModelBuilder["Model Builder"]:::training
        ClassStage["Classification Stage"]:::training
        RendererTrain["Renderer"]:::external
        RegrStage["Regression Stage"]:::training
        LossOpt["Loss & Optimizer"]:::training
        Checkpoint["Checkpointing & Logging"]:::training
    end

    %% Inference Pipeline
    subgraph "Inference Pipeline"
        direction TB
        LoadCheckpoint["Load Checkpoint & Config"]:::inference
        RenderClass["Render Classification Output"]:::inference
        FeatureCorr["Feature Extraction & Multiscale Correlation"]:::inference
        PoseOut["Pose Output"]:::inference
    end

    %% External Dependencies
    GPU["GPU Cluster"]:::external
    RenderEngine["Rendering Engine"]:::external
    EnvSpec["Project Dependencies"]:::external

    %% Connections Setup & Data to Data Layer
    EnvSetup -->|sets up env| EnvSpec
    DataPrep -->|generates caches| RawData
    DataPrep -->|generates caches| PrecompLabels

    %% Data flow to Training
    RawData -->|images| DataLoader
    PrecompLabels -->|labels| DataLoader
    DataLoader -->|batches| ConfigMgr

    %% Training sequence
    ConfigMgr -->|configs| ModelBuilder
    ModelBuilder -->|builds network| ClassStage
    ClassStage -->|features| RendererTrain
    RendererTrain -->|rendered images| RegrStage
    RegrStage -->|predictions| LossOpt
    LossOpt -->|gradients| ModelBuilder
    LossOpt -->|checkpoints| Checkpoint

    %% Training to GPU and Render Engine
    ClassStage -.->|uses| GPU
    RegrStage -.->|uses| GPU
    RendererTrain -.->|calls| RenderEngine

    %% Inference flow
    Checkpoint -->|weights| LoadCheckpoint
    LoadCheckpoint -->|inputs| RenderClass
    RenderClass -->|images| FeatureCorr
    FeatureCorr -->|feature maps| PoseOut
    PoseOut -->|final pose| GPU

    %% Styles
    classDef setup fill:#D0E6FF,stroke:#2B6CB0,color:#2B6CB0;
    classDef data fill:#E6F7FF,stroke:#3182CE,color:#3182CE;
    classDef training fill:#E6FFEA,stroke:#2F855A,color:#2F855A;
    classDef inference fill:#FFF4E6,stroke:#DD6B20,color:#DD6B20;
    classDef external fill:#F0F0F0,stroke:#718096,color:#718096;

    %% Click Events
    click EnvSetup "https://github.com/amzn/mrc-net-6d-pose/blob/main/scripts/install_deps.sh"
    click DataPrep "https://github.com/amzn/mrc-net-6d-pose/blob/main/scripts/prepare_data.sh"
    click RawData "https://github.com/amzn/mrc-net-6d-pose/blob/main/bop_dataset.py"
    click PrecompLabels "https://github.com/amzn/mrc-net-6d-pose/blob/main/precompute_quaternion_labels.py"
    click DataLoader "https://github.com/amzn/mrc-net-6d-pose/blob/main/lib/data_utils.py"
    click EnvSpec "https://github.com/amzn/mrc-net-6d-pose/blob/main/requirements.txt"
    click ConfigMgr "https://github.com/amzn/mrc-net-6d-pose/blob/main/config.py"
    click ModelBuilder "https://github.com/amzn/mrc-net-6d-pose/blob/main/models.py"
    click Checkpoint "https://github.com/amzn/mrc-net-6d-pose/blob/main/utils.py"
    click LoadCheckpoint "https://github.com/amzn/mrc-net-6d-pose/blob/main/scripts/run_inference.sh"
    click Checkpoint "https://github.com/amzn/mrc-net-6d-pose/blob/main/main.py"
    click LossOpt "https://github.com/amzn/mrc-net-6d-pose/blob/main/lib/warmup_lr.py"
```
