# demix2023/demix architecture

> Architecture diagram of the GitHub repository demix2023/demix, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/demix2023/demix
- Repository: https://github.com/demix2023/demix

## Overview

##### Project type and purpose
- This is a research‐oriented machine learning codebase for Knowledge Graph Embedding (KGE) called “Demix.”
- It provides data preprocessing, model definitions (KGE, GNN, rule‐based), loss functions, PyTorch Lightning wrappers, training scripts, and evaluation tasks.
- The end goal is to train and evaluate KGE models with a novel negative sampling and mixup strategy.

##### File structure
- Top‐level modules:
- data: data preprocessing and DataModule classes (PyTorch Lightning)
- dataset: static KGE datasets (FB15K‑237, WN18RR)
- lit_model: LightningModule wrappers around core models
- model: core model definitions separated into subdirectories (KGEModel, GNNModel, RuleModel)
- loss: many loss‐function implementations
- eval_task: scripts for link‐prediction evaluation
- utils: parser setup, common tools
- scripts: shell scripts to launch training on various models/datasets
- main.py: orchestrates argument parsing, DataModule instantiation, model selection, Trainer invocation
- requirements.txt: dependency list (PyTorch Lightning, wandb, etc.)

##### README for additional insights
- Highlights use of Python 3.9, CUDA 10.1, PyTorch Lightning 1.6.5
- Describes training via scripts, logging to wandb, saving outputs under ./output

a. Main components:
- CLI layer: shell scripts under scripts/ invoke main.py with arguments
- Argument parser: utils/setup_parser.py
- Data layer:
– dataset/ (static .txt, .dict files)
– data/ modules: DataPreprocess, base_data_module.py, specific DataModule classes (KGDataModule, ConvEDataModule, etc.)
– data/Sampler.py
- Model layer:
– core models in model/KGEModel, model/GNNModel, model/RuleModel
- Lightning layer:
– lit_model/*.LitModel: wraps core models and defines training_step, validation_step, configure_optimizers
- Loss layer:
– loss/*.py: various loss functions (BCE_Loss, Margin_Loss, InfoNCE_Loss, etc.)
- Training orchestration:
– main.py: uses PyTorch Lightning Trainer to coordinate training, validation, testing
– External service: wandb for logging, GPU (CUDA)
- Evaluation layer:
– eval_task/link_prediction.py, CompGCN_prediction.py etc.
- Output layer:
– ./wandb (logs)
– ./output (model checkpoints)

b. Relationships and interactions:
1. scripts/*.sh → run main.py
2. main.py uses setup_parser → command‐line args
3. main.py instantiates a DataModule (from data/) which loads raw files (dataset/) → provides DataLoaders
4. main.py selects a model wrapper (lit_model) which internally creates a core model (from model/) and picks loss (from loss/)
5. Trainer (PyTorch Lightning) orchestrates training loops on DataLoaders using the LightningModule
6. During training, steps call core model forward(), compute loss, log metrics to wandb
7. After training, eval_task scripts can load saved checkpoints to run link prediction tests
8. Results are written to logs and output files

c. Architectural patterns and design principles:
- Layered architecture: clear separation of data, model definitions, training logic, CLI
- Dependency injection via argument parser and dynamic instantiation
- PyTorch Lightning pattern: DataModule for data, LightningModule for model + loss + optimizer + logging
- Modular design: each loss and model type lives in its own file, easy to extend

d. Technologies, frameworks, libraries:
- Python 3.9, PyTorch, PyTorch Lightning, CUDA, wandb, Conda environment
- Shell scripts for reproducible runs

##### Elements
- Blue for code modules (data, model, loss, lit_model)
- Green for frameworks (PyTorch Lightning, wandb)
- Gray for external/static data (dataset/)
- Rectangles for code components

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% CLI Layer
    subgraph "CLI Layer"
        Scripts["scripts/"]:::code
    end

    %% Parser
    Parser["utils/setup_parser.py"]:::code

    %% Main
    Main["main.py"]:::code

    %% Data Layer
    subgraph "Data Layer"
        Dataset["dataset/"]:::data
        DataPre["data/DataPreprocess.py"]:::code
        BaseDM["data/base_data_module.py"]:::code
        KGDM["data/KGDataModule.py"]:::code
        ConvEDM["data/ConvEDataModule.py"]:::code
        KGBCE["data/KGBCEDataModule.py"]:::code
        KGCacheDM["data/KGCacheDataModule.py"]:::code
        Sampler["data/Sampler.py"]:::code
    end

    %% Model Layer
    subgraph "Model Layer"
        CoreModels["model/"]:::code
    end

    %% Lightning Layer
    subgraph "Lightning Layer"
        LitBase["lit_model/BaseLitModel.py"]:::code
        LitModels["lit_model/"]:::code
    end

    %% Loss Layer
    subgraph "Loss Layer"
        Losses["loss/"]:::code
    end

    %% Training Orchestration
    subgraph "Training Orchestration"
        PL["PyTorch Lightning"]:::framework
        Trainer["Trainer"]:::framework
        GPU["GPU (CUDA)"]:::framework
        Wandb["wandb"]:::framework
    end

    %% Evaluation Layer
    subgraph "Evaluation Layer"
        LinkPred["eval_task/link_prediction.py"]:::code
        CompPred["eval_task/CompGCN_preditcion.py"]:::code
        RGCNPred["eval_task/RGCN_prediction.py"]:::code
    end

    %% Output Layer
    subgraph "Output Layer"
        Checkpoints["./output"]:::ext
        Logs["./wandb"]:::ext
    end

    %% Flow
    Scripts -->|runs| Main
    Parser -->|provides args| Main
    Main -->|instantiates| KGDM
    Main -->|uses preprocessing| DataPre
    Main -->|uses sampler| Sampler
    KGDM --> Dataset
    ConvEDM --> Dataset
    KGBCE --> Dataset
    KGCacheDM --> Dataset

    Main -->|selects core model| CoreModels
    Main -->|wraps model| LitBase
    LitBase -->|loads| CoreModels
    LitBase -->|uses loss| Losses

    Main -->|initializes| Trainer
    Trainer -->|data| KGDM
    Trainer -->|model| LitBase
    Trainer -->|executes on| GPU
    Trainer -->|logs metrics| Wandb
    Trainer -->|saves checkpoints| Checkpoints

    LinkPred -->|loads checkpoint| Checkpoints
    CompPred -->|loads checkpoint| Checkpoints
    RGCNPred -->|loads checkpoint| Checkpoints
    LinkPred -->|writes results| Logs
    CompPred -->|writes results| Logs
    RGCNPred -->|writes results| Logs

    %% Click Events
    click Scripts "https://github.com/demix2023/demix/tree/main/scripts/"
    click Parser "https://github.com/demix2023/demix/blob/main/utils/setup_parser.py"
    click DataPre "https://github.com/demix2023/demix/blob/main/data/DataPreprocess.py"
    click BaseDM "https://github.com/demix2023/demix/blob/main/data/base_data_module.py"
    click KGDM "https://github.com/demix2023/demix/blob/main/data/KGDataModule.py"
    click ConvEDM "https://github.com/demix2023/demix/blob/main/data/ConvEDataModule.py"
    click KGBCE "https://github.com/demix2023/demix/blob/main/data/KGBCEDataModule.py"
    click KGCacheDM "https://github.com/demix2023/demix/blob/main/data/KGCacheDataModule.py"
    click Sampler "https://github.com/demix2023/demix/blob/main/data/Sampler.py"
    click Main "https://github.com/demix2023/demix/blob/main/main.py"
    click LinkPred "https://github.com/demix2023/demix/blob/main/eval_task/link_prediction.py"
    click CompPred "https://github.com/demix2023/demix/blob/main/eval_task/CompGCN_preditcion.py"
    click RGCNPred "https://github.com/demix2023/demix/blob/main/eval_task/RGCN_prediction.py"

    %% Styles
    classDef code fill:#AED6F1,stroke:#1B4F72,color:#1B2631;
    classDef framework fill:#A9DFBF,stroke:#1D8348,color:#0B5345;
    classDef data fill:#E5E7E9,stroke:#7B7D7D,color:#1C2833;
    classDef ext fill:#F2D7D5,stroke:#922B21,color:#641E16;
```
