# penn-pal-lab/liv architecture

> Architecture diagram of the GitHub repository penn-pal-lab/liv, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/penn-pal-lab/liv
- Repository: https://github.com/penn-pal-lab/liv

## Overview

##### Project Type and Purpose
- LIV is a research‐oriented Python library for learning and evaluating language‐conditioned vision representations for robotic control.
- It supports:
- Pre‐training and fine‐tuning of a multi‐modal model (LIV built atop CLIP)
- Reward‐curve generation (animated plots)
- Inference examples

##### Top‐Level Component Breakdown
a. Configuration Management (Hydra)
- Directory: liv/cfgs
- YAML files for dataset, training, launcher, output
b. Data Pipeline
- Script to download & manifest datasets (liv/dataset)
- DataLoader utilities (liv/utils/data_loaders.py)
- Supported datasets: EpicKitchen, realrobot, custom
c. Model Layer
- LIV model wrapper (liv/models/model_liv.py)
- Underlying CLIP submodule (liv/models/clip/clip)
d. Training Orchestration
- Entry point: train_liv.py
- Trainer logic: liv/trainer.py
- Checkpointing & evaluation modes
e. Logging & Visualization
- Logger (liv/utils/logger.py)
- Plotter for reward curves (liv/utils/plotter.py)
f. Examples & Inference
- Static inference: liv/examples/liv_static.py
- Dynamic reward‐curve generation: liv/examples/liv_example.py
g. Packaging & Setup
- setup.py at root
- liv/models/clip/setup.py for CLIP

##### Relationships & Data Flow
- User invokes CLI (train_liv.py or example scripts)
- Hydra loads configs → selects dataset & training regime
- Data pipeline: read manifest → DataLoader yields (frames, text) batches
- Forward pass through LIV model (vision/text encoders + similarity head)
- Loss computation & backward pass in Trainer
- Checkpoints saved; Logger records metrics
- Plotter generates (animated) reward‐curve outputs
- For inference, examples load a saved LIV model and compute embeddings/values

##### Architectural Patterns & Principles
- Modular separation of concerns: config, data, model, training, utils
- Hydra for reproducible, composable configurations
- PyTorch’s standard training loop pattern encapsulated in trainer.py
- Plugin‐style dataset support via YAML entries in liv/cfgs/dataset

##### Components & Technologies to Label
- Python 3.9, PyTorch, Torchvision, Hydra, CLIP
- Directories as packages: liv/, liv/models/, liv/utils/, liv/cfgs/, liv/dataset/, liv/examples
- External artifacts: raw video frame directories, manifest.csv, pre-trained CLIP weights, output folders

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% CLI & Config
    subgraph "CLI & Configuration"
        CLI["train_liv.py"]:::config
        Config["Config Manager (liv/cfgs)"]:::config
    end

    %% Data Pipeline
    subgraph "Data Pipeline"
        Download["download_data.sh"]:::data
        Manifest["generate_manifest.py"]:::data
        DataLoader["DataLoader Utilities"]:::data
    end

    %% Model Layer
    subgraph "Model Layer"
        LIVModel["LIV Wrapper (liv/models/model_liv.py)"]:::model
        CLIP["CLIP Submodule (liv/models/clip)"]:::model
    end

    %% Training Orchestration
    subgraph "Training Orchestration"
        Trainer["Trainer Logic (liv/trainer.py)"]:::training
    end

    %% Utilities
    subgraph "Logging & Visualization"
        Logger["Logger (liv/utils/logger.py)"]:::utils
        Plotter["Plotter (liv/utils/plotter.py)"]:::utils
    end

    %% Examples & Inference
    subgraph "Examples & Inference"
        StaticExample["Static Inference (liv/examples/liv_static.py)"]:::examples
        DynamicExample["Dynamic Reward-Curve (liv/examples/liv_example.py)"]:::examples
    end

    %% Outputs
    subgraph "Outputs"
        Checkpoints["Model Checkpoints"]:::output
        Rewards["Reward Animations"]:::output
    end

    %% Flow
    CLI -->|"load configs"| Config
    Config -->|"dataset & training settings"| Download
    Config --> Manifest
    Download -->|"raw data"| DataLoader
    Manifest -->|"manifest.csv"| DataLoader
    DataLoader -->|"batch of (frames, text)"| LIVModel
    LIVModel -->|"features"| CLIP
    CLIP -->|"embeddings"| LIVModel
    LIVModel -->|"forward pass"| Trainer
    Trainer -->|"checkpoint"| Checkpoints
    Trainer -->|"metrics"| Logger
    Logger -->|"logs"| Plotter
    Plotter -->|"reward curves"| Rewards

    %% Examples Flow
    StaticExample -->|"load model"| LIVModel
    StaticExample -->|"compute embeddings"| Rewards
    DynamicExample -->|"load model"| LIVModel
    DynamicExample -->|"generate animations"| Rewards

    %% Click Events
    click CLI "https://github.com/penn-pal-lab/liv/blob/main/liv/train_liv.py"
    click Config "https://github.com/penn-pal-lab/liv/tree/main/liv/cfgs/"
    click Download "https://github.com/penn-pal-lab/liv/blob/main/liv/dataset/download_data.sh"
    click Manifest "https://github.com/penn-pal-lab/liv/blob/main/liv/dataset/generate_manifest.py"
    click DataLoader "https://github.com/penn-pal-lab/liv/blob/main/liv/utils/data_loaders.py"
    click LIVModel "https://github.com/penn-pal-lab/liv/blob/main/liv/models/model_liv.py"
    click CLIP "https://github.com/penn-pal-lab/liv/tree/main/liv/models/clip/"
    click Trainer "https://github.com/penn-pal-lab/liv/blob/main/liv/trainer.py"
    click Logger "https://github.com/penn-pal-lab/liv/blob/main/liv/utils/logger.py"
    click Plotter "https://github.com/penn-pal-lab/liv/blob/main/liv/utils/plotter.py"
    click StaticExample "https://github.com/penn-pal-lab/liv/blob/main/liv/examples/liv_static.py"
    click DynamicExample "https://github.com/penn-pal-lab/liv/blob/main/liv/examples/liv_example.py"
    click SetupRoot "https://github.com/penn-pal-lab/liv/blob/main/setup.py"
    click SetupClip "https://github.com/penn-pal-lab/liv/blob/main/liv/models/clip/setup.py"

    %% Styles
    classDef config fill:#bbdefb,stroke:#1e88e5,color:#0d47a1;
    classDef data fill:#c8e6c9,stroke:#388e3c,color:#1b5e20;
    classDef model fill:#ffe0b2,stroke:#fb8c00,color:#e65100;
    classDef training fill:#e1bee7,stroke:#8e24aa,color:#4a148c;
    classDef utils fill:#eeeeee,stroke:#757575,color:#424242;
    classDef examples fill:#fff9c4,stroke:#fbc02d,color:#f57f17;
    classDef output fill:#fce4ec,stroke:#d81b60,color:#880e4f;
```
