# zijialewislu/cvpr2024-fact architecture

> Architecture diagram of the GitHub repository zijialewislu/cvpr2024-fact, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/zijialewislu/cvpr2024-fact
- Repository: https://github.com/zijialewislu/cvpr2024-fact

## Overview

##### Project's Purpose and Type
- The README clearly indicates that it’s an action segmentation framework built for processing video datasets (Breakfast, GTEA, EgoProceL, Epic-Kitchens).
- The main functionalities include training (train.py), evaluation (eval.py), handling various configurations (configs directory), and model definition (models directory).
- It’s a full-stack ML project with well-separated components for configuration, model, dataset handling, training, and evaluation.

##### File Structure and Map It to Architectural Components
- Top-level files like train.py, eval.py, and home.py set the entry points for training and evaluation.
- The "configs" folder contains YAML configuration files (e.g., breakfast.yaml, gtea.yaml, etc.) that dictate experiment parameters – consider this as the Config Manager.
- The "models" folder defines the learning models, with files that establish core building blocks (blocks.py, blocks_SepVerbNoun.py) and loss computation (loss.py). This is your Model Component.
- The "utils" folder includes scripts for dataset preparation (dataset.py), experiment evaluation (evaluate.py), training utilities (train_tools.py), and other helper functions (utils.py) – these form the Utility Engine supporting data operations, training flow, and analytics.
- The README also references external data preparation (download/extraction scripts in extract_epic_kitchens.py) and integration with wandb for metrics visualization.

##### Main Components and Their Interactions
- Data Input & Preprocessing:
- The system will ingest datasets (Breakfast, GTEA, etc.) that are to be stored in a designated data folder (created separately).
- Use the utils/dataset.py module along with extraction scripts (e.g., extract_epic_kitchens.py) to load and process raw video features.
- Configuration Management:
- The configs folder (with YAML files and helper utilities in configs/utils.py) acts as the central manager for all experiment settings.
- Model Definition & Architecture:
- The models folder defines various architectural components, including basic model structures and specialized blocks for cross-attention modeling.
- The loss module within models ensures the network is optimized correctly.
- Training Pipeline:
- The main training logic resides in train.py, which leverages configurations from the configs folder and model definitions from the models folder.
- Supplementary tools in utils/train_tools.py help drive the training process.
- Evaluation and Logging:
- Evaluation is handled by eval.py and supporting utilities found in utils/evaluate.py.
- The integration with wandb is used for logging metrics, visualizing losses, and tracking experiments.
- External Dependencies and Data Flow:
- Pre-trained models can be loaded from ckpts, and data preparation steps are partly defined via Git commands and scripts in README.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data & Configuration Layer
    subgraph "Data & Configuration"
        DI["Data Input & Preprocessing"]:::dataproc
        d1["dataset.py"]:::dataprocFile
        d2["extract_epic_kitchens.py"]:::dataprocFile
        DI --- d1
        DI --- d2

        CM["Configuration Manager"]:::config
        yml["YAML Files"]:::configFile
        c1["configs/utils.py"]:::configFile
        c2["configs/default.py"]:::configFile
        CM --- yml
        CM --- c1
        CM --- c2
    end

    %% Core Training and Model Definition Layer
    subgraph "Core Training and Model Definition"
        TP["Training Pipeline"]:::training
        t1["train.py"]:::trainingFile
        t2["train_tools.py"]:::trainingFile
        TP --- t1
        TP --- t2

        MC["Model Component"]:::model
        m1["basic.py"]:::modelFile
        m2["blocks.py"]:::modelFile
        m3["blocks_SepVerbNoun.py"]:::modelFile
        m4["loss.py"]:::modelFile
        MC --- m1
        MC --- m2
        MC --- m3
        MC --- m4

        TP -->|"calls_model"| MC
    end

    %% Evaluation & Logging Layer
    subgraph "Evaluation & Logging"
        EL["Evaluation & Logging"]:::evaluation
        e1["eval.py"]:::evaluationFile
        e2["evaluate.py"]:::evaluationFile
        EL --- e1
        EL --- e2

        wandb["WandB Logging"]:::external
        EL -->|"sends_metrics"| wandb
    end

    %% Data Flow Connections
    DI -->|"raw_frames"| TP
    CM -->|"YAML_config"| TP
    TP -->|"training_output"| EL
    EL -->|"feedback"| TP
    TP -->|"logs_metrics"| wandb

    %% Click Events for Data Input & Preprocessing
    click d1 "https://github.com/zijialewislu/cvpr2024-fact/blob/main/utils/dataset.py"
    click d2 "https://github.com/zijialewislu/cvpr2024-fact/blob/main/utils/extract_epic_kitchens.py"

    %% Click Events for Configuration Manager
    click yml "https://github.com/zijialewislu/cvpr2024-fact/tree/main/configs/"
    click c1 "https://github.com/zijialewislu/cvpr2024-fact/blob/main/configs/utils.py"
    click c2 "https://github.com/zijialewislu/cvpr2024-fact/blob/main/configs/default.py"

    %% Click Events for Model Component
    click MC "https://github.com/zijialewislu/cvpr2024-fact/tree/main/models/"
    click m1 "https://github.com/zijialewislu/cvpr2024-fact/blob/main/models/basic.py"
    click m2 "https://github.com/zijialewislu/cvpr2024-fact/blob/main/models/blocks.py"
    click m3 "https://github.com/zijialewislu/cvpr2024-fact/blob/main/models/blocks_SepVerbNoun.py"
    click m4 "https://github.com/zijialewislu/cvpr2024-fact/blob/main/models/loss.py"

    %% Click Events for Training Pipeline
    click t1 "https://github.com/zijialewislu/cvpr2024-fact/blob/main/train.py"
    click t2 "https://github.com/zijialewislu/cvpr2024-fact/blob/main/utils/train_tools.py"

    %% Click Events for Evaluation & Logging
    click e1 "https://github.com/zijialewislu/cvpr2024-fact/blob/main/eval.py"
    click e2 "https://github.com/zijialewislu/cvpr2024-fact/blob/main/utils/evaluate.py"

    %% Styles
    classDef dataproc fill:#a8d5e2,stroke:#333,stroke-width:2px;
    classDef dataprocFile fill:#cce5ff,stroke:#333,stroke-width:1px;
    classDef config fill:#ffcc99,stroke:#333,stroke-width:2px;
    classDef configFile fill:#ffe6cc,stroke:#333,stroke-width:1px;
    classDef training fill:#ffb3b3,stroke:#333,stroke-width:2px;
    classDef trainingFile fill:#ffd6d6,stroke:#333,stroke-width:1px;
    classDef model fill:#d4b3ff,stroke:#333,stroke-width:2px;
    classDef modelFile fill:#e0ccff,stroke:#333,stroke-width:1px;
    classDef evaluation fill:#c6e2b3,stroke:#333,stroke-width:2px;
    classDef evaluationFile fill:#e0f2d6,stroke:#333,stroke-width:1px;
    classDef external fill:#f9e79f,stroke:#333,stroke-width:2px;
```
