How demix2023/demix works
Project type and purpose
• This is a research‐oriented machine learning codebase for Knowledge Graph Embedding (KGE) called “Demix.”
• It provides data preprocessing, model definitions (KGE, GNN, rule‐based), loss functions, PyTorch Lightning wrappers, training scripts, and evaluation tasks.
• The end goal is to train and evaluate KGE models with a novel negative sampling and mixup strategy.
File structure
• Top‐level modules:
• data: data preprocessing and DataModule classes (PyTorch Lightning)
• dataset: static KGE datasets (FB15K‑237, WN18RR)
• lit_model: LightningModule wrappers around core models
• model: core model definitions separated into subdirectories (KGEModel, GNNModel, RuleModel)
• loss: many loss‐function implementations
• eval_task: scripts for link‐prediction evaluation
• utils: parser setup, common tools
• scripts: shell scripts to launch training on various models/datasets
• main.py: orchestrates argument parsing, DataModule instantiation, model selection, Trainer invocation
• requirements.txt: dependency list (PyTorch Lightning, wandb, etc.)
README for additional insights
• Highlights use of Python 3.9, CUDA 10.1, PyTorch Lightning 1.6.5
• Describes training via scripts, logging to wandb, saving outputs under ./output
a. Main components:
• CLI layer: shell scripts under scripts/ invoke main.py with arguments
• Argument parser: utils/setup_parser.py
• Data layer:
– dataset/ (static .txt, .dict files)
– data/ modules: DataPreprocess, base_data_module.py, specific DataModule classes (KGDataModule, ConvEDataModule, etc.)
– data/Sampler.py
• Model layer:
– core models in model/KGEModel, model/GNNModel, model/RuleModel
• Lightning layer:
– lit_model/*.LitModel: wraps core models and defines training_step, validation_step, configure_optimizers
• Loss layer:
– loss/*.py: various loss functions (BCE_Loss, Margin_Loss, InfoNCE_Loss, etc.)
• Training orchestration:
– main.py: uses PyTorch Lightning Trainer to coordinate training, validation, testing
– External service: wandb for logging, GPU (CUDA)
• Evaluation layer:
– eval_task/link_prediction.py, CompGCN_prediction.py etc.
• Output layer:
– ./wandb (logs)
– ./output (model checkpoints)
b. Relationships and interactions:
1. scripts/*.sh → run main.py
2. main.py uses setup_parser → command‐line args
3. main.py instantiates a DataModule (from data/) which loads raw files (dataset/) → provides DataLoaders
4. main.py selects a model wrapper (lit_model) which internally creates a core model (from model/) and picks loss (from loss/)
5. Trainer (PyTorch Lightning) orchestrates training loops on DataLoaders using the LightningModule
6. During training, steps call core model forward(), compute loss, log metrics to wandb
7. After training, eval_task scripts can load saved checkpoints to run link prediction tests
8. Results are written to logs and output files
c. Architectural patterns and design principles:
• Layered architecture: clear separation of data, model definitions, training logic, CLI
• Dependency injection via argument parser and dynamic instantiation
• PyTorch Lightning pattern: DataModule for data, LightningModule for model + loss + optimizer + logging
• Modular design: each loss and model type lives in its own file, easy to extend
d. Technologies, frameworks, libraries:
• Python 3.9, PyTorch, PyTorch Lightning, CUDA, wandb, Conda environment
• Shell scripts for reproducible runs
Elements
• Blue for code modules (data, model, loss, lit_model)
• Green for frameworks (PyTorch Lightning, wandb)
• Gray for external/static data (dataset/)
• Rectangles for code components