# samtitar/asterx architecture

> Architecture diagram of the GitHub repository samtitar/asterx, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/samtitar/asterx
- Repository: https://github.com/samtitar/asterx

## Overview

##### Key Relationships to Highlight

- Data flow from raw comics to processed HDF5 files
- Feature extraction pipeline using ViT
- Demonstrate the two-branch optimization process

##### Technical Details

- Input formats: JPG images, CSV annotations
- Intermediate formats: HDF5 files
- Model architecture details:
- ViT backbone
- Sequential encoder
- Sampling strategies (mix, comic_level)
- Optimization branches

##### Implementation Notes

- scripts/data/* for preprocessing
- src/vision_transformer.py for feature extraction
- src/models.py for the main ASTERX architecture
- src/engine.py for training logic
- src/data.py for data handling

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph Legend
        L1[Processing Component]
        L2[(Data Storage)]
        L3{{Model Component}}
        L4[/Input/Output/]
        style L1 fill:#a8d1f0
        style L2 fill:#d3d3d3
        style L3 fill:#90EE90
        style L4 fill:#FFE5B4
    end

    subgraph "Data Processing Pipeline"
        Input[/"Raw Comics (COMICS & TINTIN)"/]:::input
        
        PanelExt[Panel Extraction]:::process
        CharExt[Character Extraction]:::process
        NameNorm[Name Normalization]:::process
        DataSplit[Dataset Splitting]:::process
        HDF5Conv[HDF5 Conversion]:::process
        ClassExp[Class Export]:::process
        MetaExp[Metadata Export]:::process
        
        Storage[(HDF5 Files)]:::storage
    end

    subgraph "Feature Extraction Pipeline"
        ViT{{Vision Transformer}}:::model
        DINO{{DINO Pre-training}}:::model
        FeatExt[Feature Extraction]:::process
    end

    subgraph "ASTERX Architecture"
        SeqEnc{{Sequential Encoder}}:::model
        
        subgraph "Optimization Branches"
            CLS[Classification Branch]:::process
            TOK[Token Branch]:::process
        end
        
        Sample[Sampling Strategy]:::process
        Output[/Output Representations/]:::output
    end

    subgraph "Evaluation"
        ClassEval[Classification Evaluation]:::eval
        RetEval[Retrieval Evaluation]:::eval
        Infer[Model Inference]:::eval
    end

    %% Data Flow
    Input --> PanelExt
    Input --> CharExt
    PanelExt --> NameNorm
    CharExt --> NameNorm
    NameNorm --> DataSplit
    DataSplit --> HDF5Conv
    DataSplit --> ClassExp
    DataSplit --> MetaExp
    HDF5Conv --> Storage
    ClassExp --> Storage
    MetaExp --> Storage
    
    Storage --> ViT
    ViT --> DINO
    DINO --> FeatExt
    FeatExt --> SeqEnc
    
    SeqEnc --> CLS & TOK
    Sample --> SeqEnc
    CLS & TOK --> Output
    
    Output --> ClassEval
    Output --> RetEval
    Output --> Infer

    %% Styling
    classDef input fill:#FFE5B4
    classDef process fill:#a8d1f0
    classDef storage fill:#d3d3d3
    classDef model fill:#90EE90
    classDef output fill:#FFE5B4
    classDef eval fill:#FFA07A

    %% Click Events
    click PanelExt "https://github.com/samtitar/ASTERX/blob/main/scripts/data/export_panels.py"
    click CharExt "https://github.com/samtitar/ASTERX/blob/main/scripts/data/export_characters.py"
    click NameNorm "https://github.com/samtitar/ASTERX/blob/main/scripts/data/normalize_names.py"
    click DataSplit "https://github.com/samtitar/ASTERX/blob/main/scripts/data/split_dataset.py"
    click HDF5Conv "https://github.com/samtitar/ASTERX/blob/main/scripts/data/create_hdf5.py"
    click ViT "https://github.com/samtitar/ASTERX/blob/main/src/vision_transformer.py"
    click FeatExt "https://github.com/samtitar/ASTERX/blob/main/scripts/extract_features.py"
    click SeqEnc "https://github.com/samtitar/ASTERX/blob/main/src/models.py"
    click ClassEval "https://github.com/samtitar/ASTERX/blob/main/scripts/evaluate_classification.py"
    click RetEval "https://github.com/samtitar/ASTERX/blob/main/scripts/evaluate_retrieval.py"
    click Infer "https://github.com/samtitar/ASTERX/blob/main/scripts/inference.py"
    click ClassExp "https://github.com/samtitar/ASTERX/blob/main/scripts/data/export_classes.py"
    click MetaExp "https://github.com/samtitar/ASTERX/blob/main/scripts/data/export_metainfo.py"
```
