# microsoft/singleshotpose architecture

> Architecture diagram of the GitHub repository microsoft/singleshotpose, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-04-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/microsoft/singleshotpose
- Repository: https://github.com/microsoft/singleshotpose

## Overview

1. **What this repository is**
   This is a research codebase for **single-shot 6D object pose estimation**. The main idea is to use a YOLO-style CNN to predict 2D image locations of projected 3D bounding-box keypoints, then recover the 6D pose with a **PnP** step. The repo contains both the **single-object LINEMOD** pipeline and a **multi-object OCCLUSION** variant.

2. **Top-level architecture split**
   The repo is organized around three layers:
   - **Core single-object code** at the root
   - **Multi-object extension** in `multi_obj_pose_estimation/`
   - A full **Python 2 compatibility fork** in `py2/`
   
   The duplicated structure shows that the project supports the same pipeline across two runtimes / PyTorch-era implementations.

3. **Core execution flow**
   The primary runtime path is:
   - `train.py` for training
   - `valid.py` for evaluation / inference
   - `cfg.py` for loading dataset and model configuration
   - `darknet.py` for building the YOLO-like network
   - `dataset.py` and `image.py` for data loading / augmentation / image handling
   - `region_loss.py` for the custom loss used during training
   - `utils.py` for shared math, geometry, and post-processing helpers

   This is a classic config-driven training stack rather than an app with services or APIs.

4. **Model boundary: Darknet-style network definition**
   The network architecture is not hardcoded in Python; it is primarily defined by **Darknet config files** under `cfg/`, such as:
   - `yolo-pose.cfg`
   - `yolo-pose-pre.cfg`
   
   These configs define the CNN layers, detection head, and training schedule. `darknet.py` translates that configuration into a PyTorch model. This boundary is important: the **model topology is externalized** from the training code.

5. **Data boundary: `.data` files and dataset assets**
   Object-specific dataset metadata lives in `cfg/*.data` files, such as `ape.data`, `duck.data`, etc. These files encode the paths and camera/object parameters needed for training and testing. The repo also expects:
   - train/test split text files
   - label files
   - `.ply` 3D object models
   - optional mask/background assets
   
   So the data pipeline is split between **file-based dataset descriptors** and **Python loading code**.

6. **Single-object pose estimation pipeline**
   For a single object, the pipeline is:
   - read image + label + camera/object metadata
   - augment and normalize samples in `dataset.py`
   - feed batches into the YOLO-like model from `darknet.py`
   - compute detection/keypoint regression loss in `region_loss.py`
   - save learned weights to a model file
   - run `valid.py` to decode predictions and evaluate pose quality
   
   The key output is not a bounding box alone, but the projected keypoints used to reconstruct pose.

7. **Multi-object extension**
   `multi_obj_pose_estimation/` mirrors the core stack with separate modules:
   - `darknet_multi.py`
   - `dataset_multi.py`
   - `image_multi.py`
   - `region_loss_multi.py`
   - `train_multi.py`
   - `valid_multi.py`
   - `utils_multi.py`
   
   This is a distinct architecture variant for the OCCLUSION dataset where the model handles **multiple object classes** and uses multi-class training configs in `multi_obj_pose_estimation/cfg/`.

8. **How multi-object differs architecturally**
   The multi-object version adds an anchor-based assignment and class-aware output structure. The README notes that each label contains **21 values**: class + 9 projected points + x/y range. This means the multi-object head has to predict both:
   - **pose-related keypoints**
   - **class / object identity**
   - **size or anchor-related terms**
   
   So the main difference is not the backbone, but the **output encoding and training target assignment**.

9. **Shared geometry and 3D assets**
   The pose logic depends on 3D object model files (`.ply`) and camera intrinsics from the config files. `MeshPly.py` likely handles parsing mesh files or related 3D geometry utilities. This geometry layer is critical because pose is recovered from predicted 2D locations against known 3D reference points.

10. **Training vs inference responsibilities**
   Training scripts are responsible for:
   - loading dataset definitions
   - initializing weights
   - running optimization
   - checkpointing model weights
   
   Validation scripts are responsible for:
   - loading trained weights
   - running forward inference
   - post-processing predictions into pose estimates
   
   The repo keeps these paths separate, which matters when drawing the architecture: training is not embedded in inference code.

11. **Configuration and reproducibility**
   The repo relies heavily on external config files for reproducibility:
   - model config: layer structure, loss settings, learning rate schedule
   - data config: paths, camera intrinsics, object diameter, train/test splits
   
   This makes the codebase highly parameterized and dataset-dependent, which is central to how the architecture operates across LINEMOD, OCCLUSION, and custom datasets.

12. **Legacy compatibility branch**
   The `py2/` directory is not a separate product; it is a **legacy implementation snapshot** for older Python 2.7 / PyTorch 0.3.1 environments. It mirrors the same high-level modules as the root code, so it should be viewed as an alternate runtime branch rather than an independent subsystem.

13. **External dependencies and runtime context**
   The architecture assumes a GPU training environment with **PyTorch 0.4.1**, CUDA, and common CV libraries like NumPy, SciPy, PIL, and OpenCV. Pretrained weights and dataset tarballs are external inputs expected at setup time, so the repository is only one part of the end-to-end pose estimation system.

14. **Diagram takeaway**
   If you draw this repo, the central flow is:
   **config + dataset assets → dataset loader/augmentation → Darknet-style CNN → custom region loss during training / PnP-based pose decoding during validation**
   
   The main architectural boundaries are between:
   - configuration vs code
   - data/geometry vs model
   - training vs inference
   - single-object vs multi-object variants
   - modern code vs Python 2 legacy fork

Main entry points: [`darknet.py`](https://github.com/microsoft/singleshotpose/blob/master/darknet.py) (darknet.py), [`valid.py`](https://github.com/microsoft/singleshotpose/blob/master/valid.py) (valid.py), [`multi_obj_pose_estimation/darknet_multi.py`](https://github.com/microsoft/singleshotpose/blob/master/multi_obj_pose_estimation/darknet_multi.py) (darknet_multi.py), [`train.py`](https://github.com/microsoft/singleshotpose/blob/master/train.py) (train.py)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_core["Core pipeline"]
  node_cfg_py["cfg.py<br/>config loader<br/>[cfg.py]"]
  node_core_cfg["cfg files<br/>model/data config"]
  node_train_py["train.py<br/>training entry<br/>[train.py]"]
  node_valid_py["valid.py<br/>validation entry<br/>[valid.py]"]
  node_darknet_py["darknet.py<br/>model builder<br/>[darknet.py]"]
  node_dataset_py[("dataset.py<br/>data pipeline<br/>[dataset.py]")]
  node_image_py["image.py<br/>image utils<br/>[image.py]"]
  node_loss_py{{"region_loss.py<br/>custom loss<br/>[region_loss.py]"}}
  node_utils_py["utils.py<br/>geometry utils<br/>[utils.py]"]
  node_mesh_py["MeshPly.py<br/>3D geometry<br/>[MeshPly.py]"]
end

subgraph group_multi["Multi-object pipeline"]
  node_multi_cfg["multi cfg<br/>multi-object config"]
  node_train_multi["train_multi.py<br/>training entry<br/>[train_multi.py]"]
  node_valid_multi["valid_multi.py<br/>validation entry<br/>[valid_multi.py]"]
  node_darknet_multi["darknet_multi.py<br/>model builder<br/>[darknet_multi.py]"]
  node_dataset_multi[("dataset_multi.py<br/>data pipeline<br/>[dataset_multi.py]")]
  node_image_multi["image_multi.py<br/>image utils<br/>[image_multi.py]"]
  node_loss_multi{{"region_loss_multi.py<br/>custom loss"}}
  node_utils_multi["utils_multi.py<br/>geometry utils<br/>[utils_multi.py]"]
end

subgraph group_legacy["Legacy fork"]
  node_py2_fork["py2/<br/>legacy fork"]
end

node_cfg_py -->|"configures"| node_train_py
node_cfg_py -->|"configures"| node_valid_py
node_core_cfg -->|"defines"| node_darknet_py
node_core_cfg -->|"describes"| node_dataset_py
node_train_py -->|"loads"| node_dataset_py
node_dataset_py -->|"uses"| node_image_py
node_train_py -->|"builds"| node_darknet_py
node_darknet_py -->|"optimizes"| node_loss_py
node_darknet_py -->|"decodes"| node_utils_py
node_loss_py -->|"uses"| node_utils_py
node_valid_py -->|"runs"| node_darknet_py
node_valid_py -->|"post-processes"| node_utils_py
node_valid_py -->|"poses"| node_mesh_py
node_multi_cfg -->|"defines"| node_darknet_multi
node_train_multi -->|"loads"| node_dataset_multi
node_train_multi -->|"builds"| node_darknet_multi
node_dataset_multi -->|"uses"| node_image_multi
node_darknet_multi -->|"optimizes"| node_loss_multi
node_valid_multi -->|"runs"| node_darknet_multi
node_valid_multi -->|"post-processes"| node_utils_multi
node_py2_fork -.->|"mirrors"| node_cfg_py
node_py2_fork -.->|"mirrors"| node_train_multi

click node_cfg_py "https://github.com/microsoft/singleshotpose/blob/master/cfg.py"
click node_core_cfg "https://github.com/microsoft/singleshotpose/tree/master/cfg"
click node_train_py "https://github.com/microsoft/singleshotpose/blob/master/train.py"
click node_valid_py "https://github.com/microsoft/singleshotpose/blob/master/valid.py"
click node_darknet_py "https://github.com/microsoft/singleshotpose/blob/master/darknet.py"
click node_dataset_py "https://github.com/microsoft/singleshotpose/blob/master/dataset.py"
click node_image_py "https://github.com/microsoft/singleshotpose/blob/master/image.py"
click node_loss_py "https://github.com/microsoft/singleshotpose/blob/master/region_loss.py"
click node_utils_py "https://github.com/microsoft/singleshotpose/blob/master/utils.py"
click node_mesh_py "https://github.com/microsoft/singleshotpose/blob/master/MeshPly.py"
click node_multi_cfg "https://github.com/microsoft/singleshotpose/tree/master/multi_obj_pose_estimation/cfg"
click node_train_multi "https://github.com/microsoft/singleshotpose/blob/master/multi_obj_pose_estimation/train_multi.py"
click node_valid_multi "https://github.com/microsoft/singleshotpose/blob/master/multi_obj_pose_estimation/valid_multi.py"
click node_darknet_multi "https://github.com/microsoft/singleshotpose/blob/master/multi_obj_pose_estimation/darknet_multi.py"
click node_dataset_multi "https://github.com/microsoft/singleshotpose/blob/master/multi_obj_pose_estimation/dataset_multi.py"
click node_image_multi "https://github.com/microsoft/singleshotpose/blob/master/multi_obj_pose_estimation/image_multi.py"
click node_loss_multi "https://github.com/microsoft/singleshotpose/blob/master/multi_obj_pose_estimation/region_loss_multi.py"
click node_utils_multi "https://github.com/microsoft/singleshotpose/blob/master/multi_obj_pose_estimation/utils_multi.py"
click node_py2_fork "https://github.com/microsoft/singleshotpose/tree/master/py2"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cfg_py,node_core_cfg,node_train_py,node_valid_py,node_darknet_py,node_dataset_py,node_image_py,node_loss_py,node_utils_py,node_mesh_py toneBlue
class node_multi_cfg,node_train_multi,node_valid_multi,node_darknet_multi,node_dataset_multi,node_image_multi,node_loss_multi,node_utils_multi toneAmber
class node_py2_fork toneMint
```

## Components

### Core pipeline

Single-object LINEMOD training and validation stack.

- [cfg.py](https://github.com/microsoft/singleshotpose/blob/master/cfg.py): config loader · Loads dataset and model configuration for the single-object pipeline. · `cfg.py`
- [cfg files](https://github.com/microsoft/singleshotpose/tree/master/cfg): model/data config · Darknet model definitions and per-object .data descriptors. · `cfg`
- [train.py](https://github.com/microsoft/singleshotpose/blob/master/train.py): training entry · Runs optimization, checkpointing, and model export for single-object pose estimation. · `train.py`
- [valid.py](https://github.com/microsoft/singleshotpose/blob/master/valid.py): validation entry · Loads trained weights and decodes predictions into pose estimates. · `valid.py`
- [darknet.py](https://github.com/microsoft/singleshotpose/blob/master/darknet.py): model builder · Builds the YOLO-style network from external Darknet config files. · `darknet.py`
- [dataset.py](https://github.com/microsoft/singleshotpose/blob/master/dataset.py): data pipeline · Reads images, labels, and metadata; applies augmentation and batching. · `dataset.py`
- [image.py](https://github.com/microsoft/singleshotpose/blob/master/image.py): image utils · Image loading, preprocessing, and augmentation helpers. · `image.py`
- [region_loss.py](https://github.com/microsoft/singleshotpose/blob/master/region_loss.py): custom loss · Pose-specific detection and keypoint regression loss used during training. · `region_loss.py`
- [utils.py](https://github.com/microsoft/singleshotpose/blob/master/utils.py): geometry utils · Shared math, decoding, and post-processing helpers. · `utils.py`
- [MeshPly.py](https://github.com/microsoft/singleshotpose/blob/master/MeshPly.py): 3D geometry · Parses or supports .ply mesh assets used for pose recovery. · `MeshPly.py`

### Multi-object pipeline

OCCLUSION multi-class variant with mirrored modules.

- [multi cfg](https://github.com/microsoft/singleshotpose/tree/master/multi_obj_pose_estimation/cfg): multi-object config · OCCLUSION dataset descriptors and multi-object Darknet configs. · `multi_obj_pose_estimation/cfg`
- [train_multi.py](https://github.com/microsoft/singleshotpose/blob/master/multi_obj_pose_estimation/train_multi.py): training entry · Multi-object training loop for class-aware pose estimation. · `multi_obj_pose_estimation/train_multi.py`
- [valid_multi.py](https://github.com/microsoft/singleshotpose/blob/master/multi_obj_pose_estimation/valid_multi.py): validation entry · Multi-object inference and pose evaluation pipeline. · `multi_obj_pose_estimation/valid_multi.py`
- [darknet_multi.py](https://github.com/microsoft/singleshotpose/blob/master/multi_obj_pose_estimation/darknet_multi.py): model builder · Builds the multi-object YOLO-style network variant. · `multi_obj_pose_estimation/darknet_multi.py`
- [dataset_multi.py](https://github.com/microsoft/singleshotpose/blob/master/multi_obj_pose_estimation/dataset_multi.py): data pipeline · Loads multi-object samples and target encodings for OCCLUSION. · `multi_obj_pose_estimation/dataset_multi.py`
- [image_multi.py](https://github.com/microsoft/singleshotpose/blob/master/multi_obj_pose_estimation/image_multi.py): image utils · Multi-object image preprocessing and augmentation helpers. · `multi_obj_pose_estimation/image_multi.py`
- [region_loss_multi.py](https://github.com/microsoft/singleshotpose/blob/master/multi_obj_pose_estimation/region_loss_multi.py): custom loss · Multi-class pose loss with anchor-aware target assignment. · `multi_obj_pose_estimation/region_loss_multi.py`
- [utils_multi.py](https://github.com/microsoft/singleshotpose/blob/master/multi_obj_pose_estimation/utils_multi.py): geometry utils · Shared decoding and post-processing helpers for the multi-object branch. · `multi_obj_pose_estimation/utils_multi.py`

### Legacy fork

Python 2 compatibility branch that mirrors the core architecture.

- [py2/](https://github.com/microsoft/singleshotpose/tree/master/py2): legacy fork · Python 2 compatibility snapshot mirroring the core and multi-object pipelines. · `py2`

## Connections

- cfg.py → train.py: configures · Provides training configuration and dataset parameters.
- cfg.py → valid.py: configures · Provides evaluation and dataset parameters.
- cfg files → darknet.py: defines · Darknet config files define the core network topology.
- cfg files → dataset.py: describes · Data files and split metadata feed the dataset loader.
- train.py → dataset.py: loads · Training consumes augmented batches from the dataset pipeline.
- dataset.py → image.py: uses · Dataset loading relies on image preprocessing helpers.
- train.py → darknet.py: builds · Training constructs the model from the Darknet wrapper.
- darknet.py → region_loss.py: optimizes · Model outputs are trained against the custom region loss.
- darknet.py → utils.py: decodes · Utilities support geometry and post-processing inside the model flow.
- region_loss.py → utils.py: uses · Loss computation shares geometric helper logic.
- valid.py → darknet.py: runs · Validation loads the trained model for inference.
- valid.py → utils.py: post-processes · Validation decodes predictions into pose estimates.
- valid.py → MeshPly.py: poses · Pose recovery depends on 3D mesh geometry.
- multi cfg → darknet_multi.py: defines · Multi-object configs define the branch-specific topology and data descriptors.
- train_multi.py → dataset_multi.py: loads · Multi-object training consumes OCCLUSION batches.
- train_multi.py → darknet_multi.py: builds · Training constructs the multi-object model.
- dataset_multi.py → image_multi.py: uses · Dataset loading relies on image preprocessing helpers.
- darknet_multi.py → region_loss_multi.py: optimizes · Multi-object outputs are trained with the branch-specific loss.
- valid_multi.py → darknet_multi.py: runs · Validation loads the multi-object model for inference.
- valid_multi.py → utils_multi.py: post-processes · Utilities decode predictions into multi-object pose outputs.
- py2/ → cfg.py: mirrors · Legacy branch mirrors the core configuration and runtime structure.
- py2/ → train_multi.py: mirrors · Legacy branch also contains the multi-object training variant.
