# jeffacce/dynamo_ssl architecture

> Architecture diagram of the GitHub repository jeffacce/dynamo_ssl, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jeffacce/dynamo_ssl
- Repository: https://github.com/jeffacce/dynamo_ssl

## Overview

This project is a research–oriented visual pretraining system for visuo-motor control. It integrates dataset management, simulation environments, deep learning model training, and evaluation components into one coherent suite. The system’s aim is to pretrain visual encoders (using methods such as dynamics pretraining and variants of self‐supervised learning) and then test them in several simulated robotic environments (e.g. sim kitchen, block pushing, LIBERO Goal, Push-T).

##### Main Components
- Environment and Simulation Modules:
– The “envs” directory has multiple simulation environments (e.g. sim_kitchen, block pushing, libero, pusht). Also include the assets, physics simulation wrappers, robots, and associated BDDL files.
- Dataset Management:
– “datasets” contains modules for various dataset loaders (including a template “your_dataset.py”). This is connected to configuration files that define dataset paths via environment variables.
- Configuration System:
– The “configs” and “eval_configs” directories hold YAML configuration files for defining environment settings, training parameters, SSL options, inverse dynamics/projector settings, and encoder architectures. They drive the behavior of both training and evaluation pipelines.
- Models and Architectures:
– The “models” module encapsulates the visual encoder (with variants like ResNet and multiview stacking), projector models, self-supervised learning modules (“ssl”), and also the transformer components (e.g. the vq_behavior_transformer). These represent the “brains” of the system.
- Training and Evaluation Pipelines:
– The “train.py” script is the entry point for training the encoder models.
– The “online_eval.py” script is used for evaluating the trained model snapshots in simulation.
- Workspaces:
– The “workspaces” folder contains experiment-specific code that handles both training diagnostics (linear probes, nearest neighbor mappings) and other per-environment pipelines.
- Utilities and Supporting Scripts:
– The “utils” package and additional scripts (e.g., “online_eval.py”) provide key functionalities, such as inference, logging (with wandb), video creation, and sometimes dataset preparation.

##### Relationships and Data Flow
- Datasets feeding data into training pipelines, which in turn supply batches to the models.
- Models with the “train.py” training loop, clearly indicating that the deep learning encoder along with auxiliary (projector and SSL) components are being optimized using the data.
- Additional utilities like logging with wandb and video generation that ties back to both training and evaluation modules.

##### Architectural Patterns and Technologies
- Modular Design – The system is decomposed into well‐separated modules by function (dataset handling, simulation, configuration, training, evaluation, and models), which is typical for full research pipelines.
- Config-Driven Execution – The heavy use of YAML configuration files means the system design should emphasize a central “configuration hub” that governs both training and evaluation.
- Technologies/Libraries – Emphasize the use of Conda environments (conda_env.yml), wandb for logging, and potentially PyTorch (implied in model modules) and simulation libraries (MuJoCo, PyBullet or similar for sim environments) even though not explicitly mentioned.

##### Additional Detailed Elements
- Separation between training and evaluation workflows.
- Incorporate the concept of “workspaces” as experimental setups that combine data passes, diagnostics, and model evaluations.
- Optionally, indicate common model pipeline components: an encoder branch, a projection head (inverse dynamics, identity, etc.), and an optional EMA or SimSiam branch (as mentioned in training configs for your dataset).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Configuration Hub
    subgraph "Configuration Hub"
        CF["Configs"]:::config
        EF["Eval Configs"]:::config
    end

    %% Data Management
    subgraph "Data Management"
        DS["Datasets"]:::dataset
        subgraph "Environments & Simulation"
            SK["Sim Kitchen"]:::env
            BP["Block Pushing"]:::env
            LB["Libero"]:::env
            PT["Push-T"]:::env
        end
    end

    %% Training Pipeline
    subgraph "Training Pipeline"
        TS["Training Script"]:::train
        MD["Models (Encoder/Projector/SSL/Transformer)"]:::model
    end

    %% Evaluation Pipeline
    subgraph "Evaluation Pipeline"
        ES["Evaluation Script"]:::eval
    end

    %% Workspaces
    subgraph "Workspaces"
        WS["Workspaces"]:::workspace
    end

    %% Utilities
    subgraph "Utilities"
        UT["Utilities (Inference, Video, Logging)"]:::utility
    end

    %% Relationships and Data Flow
    CF -->|"ConfigurationInput"| TS
    EF -->|"ConfigurationInput"| TS
    CF -->|"ConfigurationInput"| ES
    EF -->|"ConfigurationInput"| ES
    DS -->|"DatasetStream"| TS
    TS -->|"TrainingUpdate"| MD
    MD -->|"ModelSnapshot"| ES
    SK -->|"SimulationInput"| ES
    BP -->|"SimulationInput"| ES
    LB -->|"SimulationInput"| ES
    PT -->|"SimulationInput"| ES
    WS -->|"DiagnosticsFeedback"| TS
    UT -->|"LoggingAndOutput"| TS
    UT -->|"LoggingAndOutput"| ES

    %% Click Events
    click SK "https://github.com/jeffacce/dynamo_ssl/blob/main/envs/sim_kitchen.py"
    click BP "https://github.com/jeffacce/dynamo_ssl/tree/main/envs/block_pushing"
    click LB "https://github.com/jeffacce/dynamo_ssl/tree/main/envs/libero"
    click PT "https://github.com/jeffacce/dynamo_ssl/blob/main/envs/pusht.py"
    click DS "https://github.com/jeffacce/dynamo_ssl/tree/main/datasets"
    click CF "https://github.com/jeffacce/dynamo_ssl/tree/main/configs"
    click EF "https://github.com/jeffacce/dynamo_ssl/tree/main/eval_configs"
    click MD "https://github.com/jeffacce/dynamo_ssl/tree/main/models"
    click TS "https://github.com/jeffacce/dynamo_ssl/blob/main/train.py"
    click ES "https://github.com/jeffacce/dynamo_ssl/blob/main/online_eval.py"
    click WS "https://github.com/jeffacce/dynamo_ssl/tree/main/workspaces"
    click UT "https://github.com/jeffacce/dynamo_ssl/tree/main/utils"

    %% Styles
    classDef config fill:#cce5ff,stroke:#004085,stroke-width:2px;
    classDef dataset fill:#d4edda,stroke:#155724,stroke-width:2px;
    classDef env fill:#fff3cd,stroke:#856404,stroke-width:2px;
    classDef model fill:#d1ecf1,stroke:#0c5460,stroke-width:2px;
    classDef train fill:#f8d7da,stroke:#721c24,stroke-width:2px;
    classDef eval fill:#d6d8d9,stroke:#1b1e21,stroke-width:2px;
    classDef workspace fill:#fdfd96,stroke:#b8860b,stroke-width:2px;
    classDef utility fill:#e2e3e5,stroke:#6c757d,stroke-width:2px;
```
