# megapose6d/megapose6d architecture

> Architecture diagram of the GitHub repository megapose6d/megapose6d, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/megapose6d/megapose6d
- Repository: https://github.com/megapose6d/megapose6d

## Overview

##### Main components
- Data & Datasets: This component encompasses the synthetic dataset (webdatasets, Google Scanned Objects, ShapeNet, etc.) along with pre-processed meshes and metadata. The “src/megapose/datasets” directory and the “local_data” folder indicate data storage and handling.
- Pre-processing & Utils: Several utility libraries and conversion/transform modules in “src/megapose/utils” and “src/megapose/lib3d” handle camera geometry, mesh operations, and coordinate transforms.
- Model Components:
- Inference Module: In “src/megapose/inference”, you find detectors, pose estimators, and various refiners (ICP, TEASER++, etc.). This module is responsible for taking in an image (and possibly depth) plus camera parameters, object meshes, and then outputting a 6D pose.
- Training Module: In “src/megapose/training”, the training scripts, loss functions, and configuration files for training (e.g., pose_models_cfg.py, training_config.py) reside.
- Model Architectures: The “src/megapose/models” directory contains the backbone networks (ResNet variants, Mask R-CNN, etc.) which are used within training or inference pipelines.
- Evaluation & Metrics: “src/megapose/evaluation” and its “meters” subdirectory are dedicated to evaluating model performance on pose estimation tasks – including scripts for running evaluation and metrics computation.
- Visualization: The “src/megapose/visualization” modules along with the notebooks (e.g., “megapose_estimator_visualization.ipynb”) provide tools for visualizing different aspects of the results ranging from detection overlays to mesh renderings.
- Scripting & Orchestration: The “src/megapose/scripts” directory includes all the higher-level scripts (e.g., downloading data, distributed training/inference, end-to-end pose estimation) to connect these modules seamlessly.
- Environment and Deployment: Directories like “conda” and “docker” indicate that the project can be deployed using either a conda-managed environment or Docker containers. This is an essential layer that encapsulates dependencies and runtime configurations.

##### Component Relationships and Interactions
- Input Flow: The system starts with data input (image files, camera parameters, object meshes, and detections from JSON files). Represent these as data sources that feed into the pre-processing unit.
- Core Processing:
▪ For Training – prepared datasets feed into training scripts (found in “training/train_megapose.py”). The training module uses model architectures defined in “models” and applies losses (e.g., “megapose_forward_loss.py”) and configurations.
▪ Evaluation then compares predictions (from “evaluation/evaluation.py” and “evaluation/eval_config.py”) against ground-truth data.
- Output & Visualization:
▪ The outputs (pose estimation JSON files, rendered overlay images) are visualized via modules like “visualization/meshcat_visualizer.py” or Bokeh-based plotters.
- Infrastructure and Deployment: Show that both conda and Docker are supported. Represent these as wrapper/container layers ensuring reproducibility and managing external dependencies.

##### Architectural Patterns
- Pipeline Structure: The end-to-end inference and training process represents a clear pipeline architecture (data ingestion → model processing → evaluation → visualization).
- Environment Abstraction: Consider adding a layer or notation for environment management (Conda and Docker) that envelops all code-related components.

##### Relevant Technologies and Frameworks
- Python-based modules and utilities.
- Deep learning frameworks (assumed to be PyTorch or similar, as seen from the “torchvision_resnet.py” file under models).
- Third party libraries (e.g., Bokeh for plotting in “bokeh_plotter.py” and MeshCat for 3D visualization).
- External tools for dataset rendering (BlenderProc is mentioned in the README for synthetic dataset generation).
- Containerization (Docker) and environment management (conda).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Top Level Input
    Input["Input Data"]

    %% Data & Datasets Subgraph
    subgraph "Data & Datasets"
        DRepo["Data Repository"]
        LD["Local Data"]
    end

    %% Pre-processing & Utils Subgraph
    subgraph "Pre-processing & Utils"
        U["Utils"]
        L3["Lib3D"]
    end

    %% Core Processing Subgraph
    subgraph "Core Processing"
        Inf["Inference Module"]
        Train["Training Module"]
        MA["Model Architectures"]
        subgraph "Evaluation & Metrics"
            Eval["Evaluation Scripts"]
            Meters["Metrics/Meters"]
        end
    end

    %% Visualization Subgraph
    subgraph "Visualization"
        VisMod["Visualization Modules"]
        VisNB["Visualization Notebooks"]
    end

    %% Scripting & Orchestration Node
    Scripting["Scripting & Orchestration"]

    %% Environment & Deployment Subgraph
    subgraph "Environment & Deployment"
        Conda["Conda Environment"]
        Docker["Docker Container"]
    end

    %% Connections
    Input --> DRepo
    Input --> LD

    DRepo --> U
    DRepo --> L3
    LD --> U
    LD --> L3

    U --> Inf
    U --> Train
    L3 --> Inf
    L3 --> Train

    Scripting --> U
    Scripting --> Inf
    Scripting --> Train
    Scripting --> Eval

    Train --> MA
    MA --> Eval
    Eval --> Meters

    Inf --> VisMod
    Eval --> VisMod
    VisMod --> VisNB

    Conda --> Scripting
    Docker --> Scripting

    %% Styles
    classDef data fill:#ADD8E6,stroke:#000,stroke-width:2px;
    classDef pre fill:#FFFF99,stroke:#000,stroke-width:2px;
    classDef core fill:#90EE90,stroke:#000,stroke-width:2px;
    classDef eval fill:#FFA07A,stroke:#000,stroke-width:2px;
    classDef vis fill:#DDA0DD,stroke:#000,stroke-width:2px;
    classDef script fill:#FFB6C1,stroke:#000,stroke-width:2px;
    classDef env fill:#D3D3D3,stroke:#000,stroke-width:2px;

    class Input,DRepo,LD data;
    class U,L3 pre;
    class Inf,Train,MA core;
    class Eval,Meters eval;
    class VisMod,VisNB vis;
    class Scripting script;
    class Conda,Docker env;

    %% Click Events
    click DRepo "https://github.com/megapose6d/megapose6d/tree/master/src/megapose/datasets"
    click LD "https://github.com/megapose6d/megapose6d/tree/master/local_data"
    click U "https://github.com/megapose6d/megapose6d/tree/master/src/megapose/utils"
    click L3 "https://github.com/megapose6d/megapose6d/tree/master/src/megapose/lib3d"
    click Inf "https://github.com/megapose6d/megapose6d/tree/master/src/megapose/inference"
    click Train "https://github.com/megapose6d/megapose6d/tree/master/src/megapose/training"
    click MA "https://github.com/megapose6d/megapose6d/tree/master/src/megapose/models"
    click Eval "https://github.com/megapose6d/megapose6d/tree/master/src/megapose/evaluation"
    click Meters "https://github.com/megapose6d/megapose6d/tree/master/src/megapose/evaluation/meters"
    click VisMod "https://github.com/megapose6d/megapose6d/tree/master/src/megapose/visualization"
    click VisNB "https://github.com/megapose6d/megapose6d/blob/master/notebooks/megapose_estimator_visualization.ipynb"
    click Scripting "https://github.com/megapose6d/megapose6d/tree/master/src/megapose/scripts"
    click Conda "https://github.com/megapose6d/megapose6d/tree/master/conda"
    click Docker "https://github.com/megapose6d/megapose6d/tree/master/docker"
```
