# sastry-group/latte-mv architecture

> Architecture diagram of the GitHub repository sastry-group/latte-mv, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sastry-group/latte-mv
- Repository: https://github.com/sastry-group/latte-mv

## Overview

##### Main Components and Modules
- Data Ingestion & Preprocessing:
– Identify the modules that handle raw video and sensor data. For example, BallTracker and PaddleTracker contain scripts (e.g., main.py, preprocess.py, clip.py) for handling input videos and extracting ball (and paddle) tracking data.

- 3D Reconstruction:
– HumanPoseTracker and RallyClipper are responsible for converting 2D detections (from ball, keypoints, table corners, etc.) into 3D reconstructions. They are configured with environments (via env.yml) and use external models (like PHALP for pose estimation).

- Anticipatory Model Training:
– The GenerativeModel module (with its own train.py, inference.py, and analysis scripts) is where the transformer is trained for anticipatory predictions. This model leverages data from the reconstructed outputs to predict ball trajectories.

- Simulation & Validation:
– MujocoRecon is used for validating the anticipatory predictions in a simulated environment. This phase uses MuJoCo for 3D physics simulation and validates the reconstructions using a robotic system.
– Represent this as a “Simulation & Evaluation” component that gets input from the anticipatory output.

- Visualization:
– Include a “Visualization” component linked to the output of both the anticipation and simulation modules.

- Supporting Utilities & Environment Setup:
– There are configuration files (env.yml, condaenv files) and build scripts that set up the experimental environments.
– Optionally show a “Configuration & Utilities” layer that supports all other modules.

##### Relationships and Data Flow
– From video input to preprocessing (BallTracker, PaddleTracker).
– From preprocessing to 3D reconstruction (HumanPoseTracker & RallyClipper).
– From reconstructed output to transformer training/inference (GenerativeModel).
– From predictions to simulation & evaluation (MujocoRecon).
– Back to visualization at multiple points in the pipeline.
– Pre-trained models and datasets (downloaded and later re-used) indicated alongside the corresponding module.
– Conda environments for each module showing the modularity/tracking of dependencies.

##### Architectural Patterns and Design Principles
- This is a staged pipeline, so illustrate the workflow as sequential stages rather than a monolithic architecture.
- Separation of concerns by isolating each phase into its submodule/directory.
– Data processing (e.g., ball and pose tracking modules)
– Model training and inference (the transformer module in GenerativeModel)
– Simulation (MuJoCo-based evaluation)
– Visualization and analysis.

##### Relevant Technologies, Frameworks, and Libraries
- Anaconda and Conda environments are central to managing dependencies. Note the use of environment files (env.yml, condaenv) across modules.
- MuJoCo is the key simulation library for Phase 3.
- Transformer-based models (for anticipatory prediction) are trained here.
- External libraries like PHALP for human pose estimation and perhaps TensorFlow/PyTorch (indirectly implied in training logs/events.out.tfevents files) should be mentioned.
- Pretrained models (downloaded via “gdown”) and their integration points.

## Diagram (Mermaid)

```mermaid
flowchart TD
    A["Data Input"]:::input

    subgraph "Phase 1: Data Ingestion & Preprocessing"
        BT["BallTracker"]:::preprocessing
        PT["PaddleTracker"]:::preprocessing
    end

    PD["Preprocessed Data"]:::data

    subgraph "Phase 2: 3D Reconstruction"
        HPT["Human Pose Tracker"]:::reconstruction
        RC["Rally Clipper"]:::reconstruction
    end

    subgraph "Phase 3: Anticipation Module"
        GM["Anticipation Transformer"]:::anticipation
    end

    subgraph "Phase 4: Simulation & Evaluation"
        MR["MuJoCo Simulation"]:::simulation
    end

    subgraph "Phase 5: Visualization & Analysis"
        VIZ["Visualization"]:::visualization
    end

    CU["Config & Utilities"]:::config

    %% Data flow connections
    A --> BT
    A --> PT
    BT --> PD
    PT --> PD
    PD --> HPT
    PD --> RC
    HPT --> GM
    RC --> GM
    GM --> MR
    GM --> VIZ
    MR --> VIZ

    %% Configuration support (dashed lines)
    CU --- BT
    CU --- PT
    CU --- HPT
    CU --- RC
    CU --- GM
    CU --- MR
    CU --- VIZ

    %% Click Events
    click BT "https://github.com/sastry-group/latte-mv/blob/main/BallTracker/main.py"
    click PT "https://github.com/sastry-group/latte-mv/blob/main/PaddleTracker/main.py"
    click HPT "https://github.com/sastry-group/latte-mv/blob/main/HumanPoseTracker/demo.py"
    click RC "https://github.com/sastry-group/latte-mv/blob/main/RallyClipper/main.py"
    click GM "https://github.com/sastry-group/latte-mv/blob/main/GenerativeModel/train.py"
    click MR "https://github.com/sastry-group/latte-mv/blob/main/MujocoRecon/mujoco_sim.py"
    click VIZ "https://github.com/sastry-group/latte-mv/blob/main/BallTracker/utils/visualize.py"
    click CU "https://github.com/sastry-group/latte-mv/blob/main/BallTracker/condaenv.rn2t0qhz.requirements.txt"

    %% Styles
    classDef input fill:#ADD8E6,stroke:#333,stroke-width:2px;
    classDef preprocessing fill:#FFD700,stroke:#333,stroke-width:2px;
    classDef reconstruction fill:#90EE90,stroke:#333,stroke-width:2px;
    classDef anticipation fill:#FFA07A,stroke:#333,stroke-width:2px;
    classDef simulation fill:#DDA0DD,stroke:#333,stroke-width:2px;
    classDef visualization fill:#87CEFA,stroke:#333,stroke-width:2px;
    classDef config fill:#F0E68C,stroke:#333,stroke-width:2px;
    classDef data fill:#E6E6FA,stroke:#333,stroke-width:2px;
```
