# ghassen-fatnassi/actia_research_internship architecture

> Architecture diagram of the GitHub repository ghassen-fatnassi/actia_research_internship, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ghassen-fatnassi/actia_research_internship
- Repository: https://github.com/ghassen-fatnassi/actia_research_internship

## Overview

##### Project Type & Purpose
- This is a machine learning research project that develops and tests methods for knowledge distillation in semantic segmentation.
- It integrates components for model training, loss function exploration, distributed training, and experiment monitoring using libraries like Accelerate and Weights & Biases.
- The README details experiments with teacher–student training paradigms and methods such as online distillation, representational KD, and response KD.

##### Analyzing the File Structure
- The top-level “workstation” directory contains a “src” folder that is clearly the heart of the system.
- Within “src” there are several key modules:
- Core training and inference scripts: train.py, teacher_training.py, response_distillation.py.
- Dataset handling: dataset.py.
- Loss and KPI modules: loss.py, benchmark.py.
- Utility functions: utils.py.
- A “config” folder holds YAML files that configure different experiments (e.g., SegFormer.yaml, Unet.yaml, config.yaml).
- The “main” directory under “src/main/Unet” contains submodules representing different distillation strategies (onlineKD, representational_KD, response_KD). This indicates a modular design where various distillation types are encapsulated separately.
- Additionally, the presence of GitHub workflow configs and requirements.txt shows standard software project practices.

a. Main Components:
- Data Layer: Represented by the dataset.py file which handles dataset loading and preparation.
- Configuration Management: Illustrate the config files (SegFormer.yaml, Unet.yaml, config.yaml) that parameterize experiments.
- Model Architecture:
- Base UNet model as the central segmentation architecture.
- Teacher model training (teacher_training.py) and Student model training (train.py) modules.
- Distillation Modules:
- Online Distillation: Modules in the onlineKD folder.
- Representational Distillation: Represented by the content under representational_KD.
- Response Distillation: Indicated by files like response_distillation.py and folders such as response_KD.
- Loss Functions: Encapsulated in loss.py, supporting customized losses including cross-entropy and dark knowledge-enhanced losses.
- Experiment Management & Monitoring:
- Distributed training support using Accelerate (as mentioned in README).
- Logging/Monitoring integrations (Weights & Biases).
- Benchmarking tools (benchmark.py) for performance comparisons.
- Utility Functions: Located in utils.py for shared operations across modules.

b. Relationships & Interactions:
- The configuration system feeds into training scripts (train.py, teacher_training.py) guiding model setup.
- The UNet-based architecture forms the core, with branching submodules applying different distillation paradigms.
- Loss functions interact with both teacher and student training modules.
- Experiment monitoring and evaluation (benchmark.py, logging integrations) occur alongside training.

c. Important Architectural Patterns:
- Modularity: The separation of distinct distillation strategies (online, representational, response) into individual modules.
- Configuration-Driven: Use of YAMLs for parameterizing experiments.
- Pipeline Structure: The project exhibits a clear ML pipeline starting with data preparation, model training (teacher and student), loss optimization, and finally distributed training and evaluation.

d. Technologies and Frameworks:
- PyTorch as the primary deep learning framework.
- Distributed training via the Accelerate library.
- Experiment monitoring via Weights & Biases.
- Standard Python components for utilities, configuration parsing, and workflow integration (GitHub actions for CodeQL scanning).

## Diagram (Mermaid)

```mermaid
graph TD
    %% Data and Configuration Layer
    subgraph "Data and Configuration Layer"
        dset["Dataset & Data Loader"]:::data
        config["Configuration Module"]:::config
    end

    %% Model and Training Layer
    subgraph "Model & Training Layer"
        unet["UNet Model"]:::core
        teach["Teacher Training Pipeline"]:::training
        stud["Student Training Pipeline"]:::training
        loss["Loss Functions"]:::aux
    end

    %% Distillation Modules
    subgraph "Distillation Modules"
        online["Online Distillation"]:::distill
        rep["Representational Distillation"]:::distill
        resp_script["Response Distillation Script"]:::distill
        resp_folder["Response KD Components"]:::distill
    end

    %% Experiment Monitoring & Utilities
    subgraph "Experiment Monitoring & Utilities"
        bench["Benchmarking & Monitoring"]:::monitor
        utils["Utility Functions"]:::aux
        disttrain["Distributed Training (Accelerate)"]:::external
    end

    %% Connections between layers
    dset --> stud
    dset --> teach
    config --> stud
    config --> teach
    unet --> teach
    unet --> stud
    loss --> teach
    loss --> stud
    teach --> resp_script
    teach --> resp_folder
    online --> stud
    rep --> stud
    resp_script --> stud
    resp_folder --> stud
    teach --> bench
    stud --> bench
    disttrain --> teach
    disttrain --> stud
    utils --> unet
    utils --> teach
    utils --> stud

    %% Styles
    classDef data fill:#a6cee3,stroke:#000,stroke-width:1px;
    classDef config fill:#fdbf6f,stroke:#000,stroke-width:1px;
    classDef core fill:#ffcc99,stroke:#000,stroke-width:1px;
    classDef training fill:#b2df8a,stroke:#000,stroke-width:1px;
    classDef aux fill:#c2a5cf,stroke:#000,stroke-width:1px;
    classDef distill fill:#fb9a99,stroke:#000,stroke-width:1px;
    classDef monitor fill:#e31a1c,stroke:#000,stroke-width:1px;
    classDef external fill:#33a02c,stroke:#000,stroke-width:1px;

    %% Click Events
    click dset "https://github.com/ghassen-fatnassi/actia_research_internship/blob/main/workstation/src/dataset.py"
    click config "https://github.com/ghassen-fatnassi/actia_research_internship/tree/main/workstation/src/config/"
    click unet "https://github.com/ghassen-fatnassi/actia_research_internship/blob/main/workstation/src/Unet.py"
    click teach "https://github.com/ghassen-fatnassi/actia_research_internship/blob/main/workstation/src/teacher_training.py"
    click stud "https://github.com/ghassen-fatnassi/actia_research_internship/blob/main/workstation/src/train.py"
    click loss "https://github.com/ghassen-fatnassi/actia_research_internship/blob/main/workstation/src/loss.py"
    click online "https://github.com/ghassen-fatnassi/actia_research_internship/tree/main/workstation/src/main/Unet/onlineKD/"
    click rep "https://github.com/ghassen-fatnassi/actia_research_internship/tree/main/workstation/src/main/Unet/representational_KD/"
    click resp_script "https://github.com/ghassen-fatnassi/actia_research_internship/blob/main/workstation/src/response_distillation.py"
    click resp_folder "https://github.com/ghassen-fatnassi/actia_research_internship/tree/main/workstation/src/main/Unet/response_KD/"
    click bench "https://github.com/ghassen-fatnassi/actia_research_internship/blob/main/workstation/src/benchmark.py"
    click utils "https://github.com/ghassen-fatnassi/actia_research_internship/blob/main/workstation/src/utils.py"
```
