# octoberchang/gamlp architecture

> Architecture diagram of the GitHub repository octoberchang/gamlp, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/octoberchang/gamlp
- Repository: https://github.com/octoberchang/gamlp

## Overview

##### Main Components

a) Data Processing Pipeline
- Input data processing (papers100M dataset)
- GIANT-XRT feature extraction
- Preprocessed embeddings storage

b) Model Architecture
- GAMLP+RLU model components
- Multi-stage training process
- DGL (Deep Graph Library) integration

c) Training and Evaluation Flow
- Training stages (0-3)
- Validation/Test accuracy computation
- Model parameter management

**[Input Data] → [Preprocessing] → [GIANT-XRT] → [GAMLP+RLU] → [Evaluation]**

##### Important Elements to Highlight

- Clear labels for:
- File references (e.g., main.py, model.py)
- Stage transitions
- Accuracy metrics at each stage
- Data transformation steps

##### Technical Details

- DGL integration points
- CUDA/GPU utilization (reference to GPU_ID)
- Memory management for large-scale data
- Input/output formats for each component

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph Legend
        L1[Process]
        L2[(Database)]
        L3([Input/Output])
        L4{{Model Component}}
        style L1 fill:#a8d1ff
        style L2 fill:#ffb3b3
        style L3 fill:#c2f0c2
        style L4 fill:#ffdb99
    end

    subgraph "Data Processing Pipeline"
        Input([Papers100M Dataset]):::input
        Preprocess[Data Preprocessing]:::process
        GraphEmb[Graph Embedding Training]:::process
        DataLoad[Data Loading Interface]:::process
        HeteroData[Heterogeneous Data Handling]:::process
        Storage[(Node Embeddings Storage)]:::database
    end

    subgraph "Model Architecture"
        GIANXRT{{GIANT-XRT Feature Extractor}}:::model
        GAMLP{{GAMLP+RLU Model}}:::model
        ModelLayers{{Model Layers}}:::model
    end

    subgraph "Training and Evaluation"
        MainLoop[Main Training Loop]:::process
        TrainScript[Training Script]:::process
        Logger[Logging and Metrics]:::process
        Utils[Utility Functions]:::process
        Eval([Evaluation Metrics]):::output
    end

    Input --> Preprocess
    Preprocess --> GraphEmb
    GraphEmb --> DataLoad
    DataLoad --> HeteroData
    HeteroData --> Storage
    Storage --> GIANXRT
    GIANXRT --> GAMLP
    GAMLP --> ModelLayers
    ModelLayers --> MainLoop
    MainLoop --> TrainScript
    TrainScript --> Logger
    Logger --> Eval
    Utils --> MainLoop

    %% Component Mappings
    click Preprocess "https://github.com/OctoberChang/GAMLP/blob/main/data/preprocess_papers100m.py"
    click GraphEmb "https://github.com/OctoberChang/GAMLP/blob/main/data/train_graph_emb.sh"
    click DataLoad "https://github.com/OctoberChang/GAMLP/blob/main/load_dataset.py"
    click HeteroData "https://github.com/OctoberChang/GAMLP/blob/main/heteo_data.py"
    click GAMLP "https://github.com/OctoberChang/GAMLP/blob/main/model.py"
    click ModelLayers "https://github.com/OctoberChang/GAMLP/blob/main/layer.py"
    click MainLoop "https://github.com/OctoberChang/GAMLP/blob/main/main.py"
    click TrainScript "https://github.com/OctoberChang/GAMLP/blob/main/run_gamlp_xrt.sh"
    click Logger "https://github.com/OctoberChang/GAMLP/blob/main/logger.py"
    click Utils "https://github.com/OctoberChang/GAMLP/blob/main/utils.py"

    classDef process fill:#a8d1ff
    classDef database fill:#ffb3b3
    classDef input fill:#c2f0c2
    classDef output fill:#c2f0c2
    classDef model fill:#ffdb99
```
