How yuxiawu/simpledyg works
Project type and purpose
• This is a research‐focused ML pipeline (“SimpleDyG”) implementing a Transformer for dynamic graph modeling.
• It includes data preprocessing, training, tokenization/vocabulary generation, and evaluation.
High-level components
a. Data Ingestion & Preprocessing
• all_data/…: raw datasets (dialog, hepth, movie, uci) and per‐dataset scripts (preprocess.py, data_trans.py, train_word2vec.py).
• csv2res.sh & csv2resources.py: transforms CSV+NPY into the final format under resources/.
b. Data Store (Preprocessed Artifacts)
• resources/: final train/val/test link‐prediction files for each dataset & timestamp.
• tokenizers/: HuggingFace tokenizer JSONs & configs, per dataset/timestamp.
• vocabs/: vocabulary JSONs matching each tokenizer.
c. Code Modules
• data/: Dataset and language_model wrapper (data ingestion API).
• models/: Transformer model definitions (configuration_gpt2.py, modeling_gpt2.py, utilities).
• utils/: argument parsers, evaluation routines (Evaluation.py), language_model helper, generic model helper.
d. Training & Orchestration
• main.py: entry point tying data, model, training loop, logging to runs/.
• train_seed_*.sh: shell wrappers invoking main.py with dataset‐specific arguments.
• requirements.txt: Python dependencies.
e. Output & Evaluation
• During training:
– tokenizers/, vocabs/, output/ (checkpoints), runs/ (TensorBoard).
• eval_single_step.py: runs one‐step evaluation, writing metrics to results/.
Relationships & Data Flow
1) Raw data in all_data/ → run each all_data/*/preprocess.py → CSV + NPY outputs (ml_*.csv/.npy).
2) csv2res.sh → csv2resources.py → write standardized link‐prediction files to resources/.
3) Training scripts (train_seed_*.sh → main.py):
– main.py loads resources/, instantiates a tokenizer (writes to tokenizers/), builds vocab (vocabs/).
– Data module loads examples, DataLoader → utils/model.py orchestrates training loop.
– models/modeling_gpt2.py defines the Transformer, configuration_utils.py sets hyperparams.
– Checkpoints saved to output/, logs to runs/.
4) Evaluation: eval_single_step.py loads checkpoint from output/, loads resources/ for test split, runs utils/Evaluation.py → results/.
Architectural patterns & design principles
• Pipeline architecture: clear segmentation into Preprocessing → Data Storage → Model Training → Evaluation.
• Modular codebase:
• data/ encapsulates dataset handling
• models/ encapsulates all modeling logic
• utils/ holds cross‐cutting concerns (parsing args, evaluation)
• Reproducibility & logging: shell scripts + main.py + tensorboard logs ensure experiment tracking.