How kilab-lyu/graphprobe works
Project Type and Purpose
• This is an open-source research toolkit for “Knowledge Probing for Graph Representation Learning.”
• It provides a model-agnostic pipeline that:
a. Trains a variety of graph embedding (GE) models on different graph datasets.
b. Stores learned embeddings.
c. Applies multiple probing tasks (CentProbe, DistProbe, etc.) to evaluate how well embeddings capture structural/semantic knowledge.
d. Produces evaluation results and visualizations.
High-Level File Structure Analysis
• Top-level orchestrators:
• main.py: entry point that reads configuration, loops over GE models and probing tasks.
• GE_train.py & GE_model.py: handle graph embedding training.
• DownstreamTask.py: dispatches and runs probe tasks.
• Data layer:
• Datasets.py + Utility/data/: dataset loaders, samplers.
• Cora&FlickrInfo.py, ProteinsDataset.py, MutagCos.py, ppi.py, yelp.py, amazon_book.py, etc.
• Model layer:
• Utility/model/GCL, Utility/model/base, Utility/model/general_recommender: implementations of GraphCL, GCN, GAT, GraphSAGE, Node2Vec, etc.
• Probing layer:
• probe/: various probing modules (CentProbe, DistProbe, MIProbe, etc.).
• Configuration & Utilities:
• Utility/constant.py: all hyperparameters & datasets/models lists.
• Utility/parser.py, Utility/helper.py, Utility/metrics.py, Utility/utils.py.
• Utility/reckit/: Cython-accelerated evaluation metrics, logging, plotting.
• Output:
• Embedding files under EMB_ROOT (e.g. './Embedding').
• results/: text files of probe evaluations per dataset/model.
• drawRadar.py, DrawLinePic.py, img/: scripts & images for visualization.
Key Insights from README
• The pipeline is driven entirely by constants in Utility/constant.py (model list, dataset list, downstream tasks).
• The code supports both transductive and inductive modes.
• Probing tasks selection is done via Dict_probe_emb mapping to List_permit_args.
Components
a. Main Components
1. Configuration Manager
• Reads Utility/constant.py
• Exposes: List_GE_Model, List_permit_args, Dict_probe_emb, device, modes
2. Data Ingestion & Preprocessing
• Datasets.py + Utility/data loaders & samplers
• Specialized dataset scripts (cora.py, flickr.py, etc.)
3. Graph Embedding Training
• GE_train.py orchestrates training loops
• Graph model implementations in Utility/model/… and GE_model.py
• Uses PyTorch Geometric, DGL, torch-cluster, Cython (Utility/reckit)
• Outputs serialized embeddings to EMB_ROOT
4. Knowledge Probing
• DownstreamTask.py calls specific probe modules under probe/
• Probes consume embeddings + dataset metadata
• Evaluation metrics via Utility/reckit (Cython-backed) and Utility/metrics
• Writes results under results/…
5. Visualization
• DrawLinePic.py, drawRadar.py
• img/ for static assets
6. Utilities & Shared Services
• Logging (Utility/reckit/logger.py)
• Common helper functions (Utility/helper.py, Utility/utils.py)
• Augmentors & losses for contrastive learning (Utility/model/GCL/Utility/…)
b. Relationships & Data Flow
1. Configuration Manager feeds parameters → Data Ingestion
2. Data Ingestion → Graph Embedding Trainer
3. Trainer → Serialized embeddings
4. Serialized embeddings + Configuration → Probing Tasks
5. Probing Tasks → Evaluation Results
6. Results → Visualization scripts
c. Architectural Patterns & Principles
• Modular, plugin-like design: new GE models in Utility/model, new probes under probe/.
• Separation of concerns: configuration, data, models, probes, evaluation, visualization all in distinct modules.
• Performance optimization: Cython modules under Utility/reckit/cython.
• Model-agnostic pipeline: same orchestration for any GE model.
d. Key Technologies & Frameworks
• Python 3.x, PyTorch 2.1+cu118, PyTorch-Geometric, DGL, Torch-Cluster/Scatter/Sparse, Torchaudio, Torchvision
• Cython for critical loops (Utility/reckit/cython)
• Numpy, Matplotlib or similar via DrawLinePic.py