# yueyang1996/labo architecture

> Architecture diagram of the GitHub repository yueyang1996/labo, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/yueyang1996/labo
- Repository: https://github.com/yueyang1996/labo

## Overview

- A “LaBo” training/testing procedure (based on concept bottlenecks and submodular optimization), and
- A standard “linear probe” approach for interpretable image classification.

##### Main Components of the System
a. Experiment Driver and Interface:
– main.py acts as the primary entry point that dispatches experiments.
– Shell scripts (labo_train.sh, labo_test.sh, linear_probe.sh) wrap around the training/test routines.
b. Configuration Management:
– The cfg/ directory holds configuration files for both LaBo (in cfg/asso_opt) and linear probe experiments (in cfg/linear_probe). These configurations set system arguments, hyperparameters, and experiment settings.
c. Data Module:
– Data loaders are implemented in data.py (for LaBo) and data_lp.py (for linear probe).
– The datasets/ directory contains dataset-specific information (splits, concepts, and instructions on image storage), meaning that the data ingestion and preparation pipelines are crucial.
d. Model Implementations:
– LaBo Model: In models/asso_opt/asso_opt.py, this module implements the concept bottleneck model that utilizes language models and submodular optimization.
– Linear Probe: In models/linear_probe/linear_probe.py, this module implements the linear classification layer over image features.
– Concept Selection: In models/select_concept/select_algo.py, the algorithm for concept selection is provided.
e. Experiment Management and Checkpointing:
– The exp/ directory is used as a workspace to store model checkpoints and configuration snapshots from the training experiments.
– The output/ directory captures performance logs and results.
f. Utility and External Services:
– utils.py encapsulates auxiliary functions (preprocessing, feature extraction).
– External experiment logging/tracking with wandb is indicated in the README.

##### Relationships and Data Flows
– The configuration files, once parsed, direct two parallel branches:
- Data Pipeline: Data loaders (data.py / data_lp.py) use dataset definitions from the datasets/ folder.
- Model Pipeline: Depending on the experiment type, the corresponding model file is invoked (either asso_opt.py for LaBo or linear_probe.py for linear probe). When using LaBo, the concept selection module (select_algo.py) also comes into play.
– These pipelines finally converge in training or testing workflows (with main.py orchestrating) and then produce outputs that are stored in the exp/ and output/ directories.
– External modules like wandb are integrated during training to log metrics—this can be shown as a side component connecting to the training process.

##### Architectural Patterns and Design Principles
– Modularity: The project clearly separates configuration, data loading, model implementation, and experiment management.
– Separation of Concerns: Different directories deal with different aspects (models vs. datasets vs. configs).
– Extendibility: By having dedicated config files per dataset and shot variations, and separating the concept selection logic, changes or additions in new datasets or methods can be easily encapsulated.
– Reusability: Shared utilities in utils.py and separate data loading for different experiments foster a reusable design.

##### Relevant Technologies and Frameworks
– Python 3.9.13 is specified; the project likely leverages common deep learning frameworks (e.g., PyTorch) though that isn’t explicit in the tree.
– Experiment tracking is done using wandb.
– The configuration structure hints at potential use of argument parsers or configuration tools to manage experiment settings.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Experiment Interface
    subgraph "Experiment Interface"
        EI1["main.py"]:::interface
        EI2["labo_train.sh"]:::interface
        EI3["labo_test.sh"]:::interface
        EI4["linear_probe.sh"]:::interface
    end

    %% Configuration Manager
    subgraph "Configuration Manager"
        CM1["LaBo Config (cfg/asso_opt)"]:::config
        CM2["Linear Probe Config (cfg/linear_probe)"]:::config
    end

    %% Data Module
    subgraph "Data Module"
        D1["Data Loader (data.py)"]:::data
        D2["Data Loader (data_lp.py)"]:::data
        D3["Datasets Repository (datasets/)"]:::data
    end

    %% Model Components
    subgraph "Model Components"
        M1["LaBo Model (models/asso_opt)"]:::model
        M2["Linear Probe (models/linear_probe)"]:::model
        M3["Concept Selection (models/select_concept)"]:::model
    end

    %% Experiment Management and Checkpointing
    subgraph "Experiment Management"
        EM1["Experiment Storage (exp/)"]:::storage
        EM2["Output Logs (output/)"]:::storage
    end

    %% Utility Functions
    U1["Utilities (utils.py)"]:::util

    %% External Services
    ES1["wandb Logging"]:::external

    %% Connections
    %% From Experiment Interface to Configuration Manager
    EI1 -->|"LoadConfig"| CM1
    EI1 -->|"LoadConfig"| CM2

    %% From Configuration Manager to Data Module (configuring data loaders)
    CM1 -->|"DatasetConfig"| D1
    CM2 -->|"DatasetConfig"| D2
    %% Common dataset repository available for both experiments
    CM1 -->|"Ref.Datasets"| D3
    CM2 -->|"Ref.Datasets"| D3

    %% From Data Module to Model Components (training data flow)
    D1 -->|"TrainingData"| M1
    D1 -->|"TrainingData"| M3
    D2 -->|"TrainingData"| M2

    %% Direct configuration for model settings
    CM1 -->|"ModelSettings"| M1
    CM2 -->|"ModelSettings"| M2

    %% Concept Selection influences LaBo Model
    M3 -->|"SelectConcepts"| M1

    %% From Model Components to Experiment Management (checkpointing and logs)
    M1 -->|"SaveCheckpoints"| EM1
    M1 -->|"SaveLogs"| EM2
    M2 -->|"SaveCheckpoints"| EM1
    M2 -->|"SaveLogs"| EM2

    %% External Logging from Model Components
    M1 -->|"LogMetrics"| ES1
    M2 -->|"LogMetrics"| ES1

    %% Utility Functions support Data and Model Modules (dotted lines)
    D1 -.->|"Preprocess"| U1
    M1 -.->|"FeatureExtract"| U1
    M2 -.->|"FeatureExtract"| U1

    %% Click Events
    click EI1 "https://github.com/yueyang1996/labo/blob/main/main.py"
    click EI2 "https://github.com/yueyang1996/labo/blob/main/labo_train.sh"
    click EI3 "https://github.com/yueyang1996/labo/blob/main/labo_test.sh"
    click EI4 "https://github.com/yueyang1996/labo/blob/main/linear_probe.sh"
    click CM1 "https://github.com/yueyang1996/labo/tree/main/cfg/asso_opt"
    click CM2 "https://github.com/yueyang1996/labo/tree/main/cfg/linear_probe"
    click D1 "https://github.com/yueyang1996/labo/blob/main/data.py"
    click D2 "https://github.com/yueyang1996/labo/blob/main/data_lp.py"
    click D3 "https://github.com/yueyang1996/labo/tree/main/datasets/"
    click M1 "https://github.com/yueyang1996/labo/blob/main/models/asso_opt/asso_opt.py"
    click M2 "https://github.com/yueyang1996/labo/blob/main/models/linear_probe/linear_probe.py"
    click M3 "https://github.com/yueyang1996/labo/blob/main/models/select_concept/select_algo.py"
    click EM1 "https://github.com/yueyang1996/labo/tree/main/exp/"
    click EM2 "https://github.com/yueyang1996/labo/tree/main/output/"
    click U1 "https://github.com/yueyang1996/labo/blob/main/utils.py"

    %% Styles
    classDef interface fill:#bbd1f7,stroke:#2a65a0,stroke-width:2px;
    classDef config fill:#d0f0c0,stroke:#3a773a,stroke-width:2px;
    classDef data fill:#f7d6bb,stroke:#a0652a,stroke-width:2px;
    classDef model fill:#f7f7bb,stroke:#a0a02a,stroke-width:2px;
    classDef storage fill:#ffeebb,stroke:#a08a2a,stroke-width:2px;
    classDef util fill:#e0e0e0,stroke:#707070,stroke-dasharray: 5 5,stroke-width:2px;
    classDef external fill:#fbb6c2,stroke:#a02a50,stroke-width:2px;
```
