# cchenyuchi/huggingface-art-transformer architecture

> Architecture diagram of the GitHub repository cchenyuchi/huggingface-art-transformer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/cchenyuchi/huggingface-art-transformer
- Repository: https://github.com/cchenyuchi/huggingface-art-transformer

## Overview

##### Project type and purpose
- This is an open-source research/project repository implementing an EEG-based ART (Adaptive Response Transformer) classification pipeline in TensorFlow, leveraging Huggingface Transformer conventions.
- Key goals: load EEG/SLT data, preprocess it, configure and train an ART-style transformer model, and evaluate/test it via notebooks.

##### File structure
- Top-level artifacts:
- ART_model_testing.ipynb, GitVisionModel_testing.ipynb, Trainer.ipynb, eval.ipynb → interactive evaluation & experiment tracking.
- modeling_git.py → glue or packaging script for model versions.
- DataLoader/ → data ingestion & preprocessing code + JSON group definitions.
- FirstMultiModel/EEGART/ → core model code, config folder, pretrained weights.
- Notice a clear pipeline structure: DataLoader → model definition & configuration → training scripts → evaluation notebooks.

##### README
- Brief: “Huggingface Transformer format for ART”
- Implies that the model code follows transformer conventions (config.json, safetensors), consistent with Huggingface’s layout.

##### Main components and interactions (
a. Components:
1. Data Ingestion Layer (DataLoader/SLT_dataloader.py + JSONs)
2. Data Processing & Batching (tf_data.py)
3. Model Configuration (artcls-config/config.json + model.safetensors)
4. Model Definition (tf_model.py)
5. Loss & Optimization (tf_loss.py, tf_opt.py)
6. Training Orchestration (tf_train2.py)
7. Utilities (tf_utils.py, tf_config.py)
8. Experiment Scripts/Notebooks (ART_model_testing.ipynb, GitVisionModel_testing.ipynb, Trainer.ipynb, eval.ipynb)
9. Packaging/Versioning (modeling_git.py)

b. Relationships & Dataflow:
- tf_model ← config.json/model.safetensors
- Model outputs → tf_loss → tf_opt → tf_train2 (loop)
- Trained checkpoints → artcls-config/ (persisted)
- Trained model & metrics → eval.ipynb / testing notebooks

c. Architectural patterns/design principles:
- Pipeline-oriented ML architecture (modular layers for data, model, training, evaluation).
- Huggingface-style decoupling of config, weights, and code.
- Separation of concerns: data preprocessing vs. model logic vs. training loop vs. evaluation.

d. Key technologies/frameworks:
- Python & Jupyter Notebooks for experimentation
- TensorFlow for model implementation
- Huggingface Transformer conventions (config.json, safetensors)
- Custom data loader parsing JSON subject groups

##### Detailed guidance
- Next: optimizer & loss modules feeding into the training orchestration block.
- Finally: outputs feeding into evaluation/test notebooks and the modeling_git packaging step.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Data Ingestion Layer"
        JSON1["common_subjects.json"]:::data
        JSON2["subject_groups.json"]:::data
        SLT["SLT_dataloader.py"]:::code
        JSON1 --> SLT
        JSON2 --> SLT
    end
    click SLT "https://github.com/cchenyuchi/huggingface-art-transformer/blob/master/DataLoader/SLT_dataloader.py"
    click JSON1 "https://github.com/cchenyuchi/huggingface-art-transformer/blob/master/DataLoader/common_subjects.json"
    click JSON2 "https://github.com/cchenyuchi/huggingface-art-transformer/blob/master/DataLoader/subject_groups.json"

    DLNB["dataloade_test.ipynb"]:::notebook
    SLT --> DLNB
    click DLNB "https://github.com/cchenyuchi/huggingface-art-transformer/blob/master/DataLoader/dataloade_test.ipynb"

    subgraph "Data Processing & Batching"
        tfdata["tf_data.py"]:::code
    end
    click tfdata "https://github.com/cchenyuchi/huggingface-art-transformer/blob/master/FirstMultiModel/EEGART/tf_data.py"
    SLT --> tfdata

    subgraph "Model Configuration"
        cfg["config.json"]:::data
        weights["model.safetensors"]:::data
    end
    click cfg "https://github.com/cchenyuchi/huggingface-art-transformer/blob/master/FirstMultiModel/EEGART/artcls-config/config.json"
    click weights "https://github.com/cchenyuchi/huggingface-art-transformer/blob/master/FirstMultiModel/EEGART/artcls-config/model.safetensors"

    subgraph "ART Transformer Model"
        model["tf_model.py"]:::code
        utils["tf_utils.py"]:::code
        configHelper["tf_config.py"]:::code
    end
    click model "https://github.com/cchenyuchi/huggingface-art-transformer/blob/master/FirstMultiModel/EEGART/tf_model.py"
    click utils "https://github.com/cchenyuchi/huggingface-art-transformer/blob/master/FirstMultiModel/EEGART/tf_utils.py"
    click configHelper "https://github.com/cchenyuchi/huggingface-art-transformer/blob/master/FirstMultiModel/EEGART/tf_config.py"

    tfdata --> model
    cfg --> model
    weights --> model

    subgraph "Loss & Optimization"
        loss["tf_loss.py"]:::code
        opt["tf_opt.py"]:::code
    end
    click loss "https://github.com/cchenyuchi/huggingface-art-transformer/blob/master/FirstMultiModel/EEGART/tf_loss.py"
    click opt "https://github.com/cchenyuchi/huggingface-art-transformer/blob/master/FirstMultiModel/EEGART/tf_opt.py"

    model --> loss --> opt

    subgraph "Training Orchestration"
        train["tf_train2.py"]:::code
    end
    click train "https://github.com/cchenyuchi/huggingface-art-transformer/blob/master/FirstMultiModel/EEGART/tf_train2.py"
    opt --> train
    train -->|"checkpoint"| cfg
    train -->|"metrics & model"| nb1

    subgraph "Evaluation & Experiment Notebooks"
        nb1["ART_model_testing.ipynb"]:::notebook
        nb2["GitVisionModel_testing.ipynb"]:::notebook
        nb3["Trainer.ipynb"]:::notebook
        nb4["eval.ipynb"]:::notebook
    end
    click nb1 "https://github.com/cchenyuchi/huggingface-art-transformer/blob/master/ART_model_testing.ipynb"
    click nb2 "https://github.com/cchenyuchi/huggingface-art-transformer/blob/master/GitVisionModel_testing.ipynb"
    click nb3 "https://github.com/cchenyuchi/huggingface-art-transformer/blob/master/Trainer.ipynb"
    click nb4 "https://github.com/cchenyuchi/huggingface-art-transformer/blob/master/eval.ipynb"
    train --> nb2
    train --> nb3
    train --> nb4

    pack["modeling_git.py"]:::code
    click pack "https://github.com/cchenyuchi/huggingface-art-transformer/blob/master/FirstMultiModel/modeling_git.py"
    train --> pack

    classDef code fill:#ADD8E6,stroke:#0000FF,stroke-width:1px
    classDef data fill:#90EE90,stroke:#008000,stroke-width:1px
    classDef notebook fill:#FFFFE0,stroke:#CCCC00,stroke-width:1px
```
