How tail-unica/hopwise works
Project Type and Purpose
• hopwise is a Python library—an extension of the RecBole recommender-system framework—focused on knowledge-graph-enhanced recommendation and explainability. It exposes a CLI entry point (run_hopwise.py) and example scripts, but its core is a modular, plugin-based library for data loading, model definition, training, and evaluation.
Main Architectural Components
a. CLI & Configuration
• run_hopwise.py (and related scripts) invoke the system.
• hopwise/config/configurator.py reads YAML config files (from properties/) and merges CLI overrides.
b. Data Layer
• Abstract interfaces: hopwise/data/dataloader/abstract_dataloader.py.
• Concrete loaders: general, KG, user, knowledge loaders.
• Dataset classes in hopwise/data/dataset/ wrap raw data into PyTorch/DGL-friendly objects.
• Utility modules for transform, interaction, sampling.
c. Sampler
• hopwise/sampler/sampler.py produces batches or KG paths.
d. Model Registry & Implementations
• Base interface: hopwise/model/abstract_recommender.py.
• Subpackages for each family:
– context_aware_recommender
– general_recommender
– knowledge_aware_recommender
– knowledge_graph_embedding_recommender
– path_language_modeling_recommender
– sequential_recommender
• Each model class implements .forward(), .predict(), etc.
e. Trainer & Hyper-Tuning
• hopwise/trainer/trainer.py orchestrates the training loop (data → model → loss → optimizer → sampler).
• hopwise/trainer/hf_path_trainer.py for HuggingFace-style modeling.
• hyper_tuning.py for grid/random search over config space.
f. Evaluator
• hopwise/evaluator/evaluator.py, metrics (ranking, loss, top-K) and collector.
g. Utilities & Logging
• hopwise/utils/logger.py sets up logging.
• wandblogger.py for optional Weights & Biases integration.
h. Quick-Start & Examples
• Scripts in hopwise/quick_start/ and run_example/ illustrate common workflows.
Component Interactions & Data Flow
1) User calls CLI → Configurator loads CLI + YAML.
2) Configurator instantiates DataLoader → Dataset → Sampler.
3) Trainer receives model class from registry using config → initializes model, optimizer, loss.
4) Training loop: Sampler yields batches → Model forward pass → Loss → Backprop → Optimizer step.
5) At epoch end: Evaluator computes metrics → Logger/W&B logs.
6) Upon completion: artifacts (checkpoints, metrics) are saved.
Architectural Patterns & Design Principles
• Plugin architecture: abstract base classes for DataLoader, Recommender, Sampler, allowing easy extension.
• Layered design: separation of concerns between config, data ingestion, modeling, training, and evaluation.
• Configuration-driven: behavior entirely controlled by YAML + CLI args, with no hard-coded paths.
Technologies & Frameworks
• Python 3.9–3.11
• RecBole (as base)
• PyTorch, DGL (for GNN models)
• uv (dependency manager & CLI runner)
• YAML for config (hopwise/properties/)
• Optional: LightGBM, XGBoost backends; HuggingFace trainer; Weights & Biases.