How wzh9969/contrastive-htc works
Main Components
• Data / Preprocessing Layer:
• Three main dataset directories (data/WebOfScience, data/nyt, data/rcv1) each containing scripts to preprocess and load data.
• Includes specific preprocessing scripts (e.g., preprocess_wos.py, preprocess_rcv1.py) and data conversion scripts (e.g., data_wos.py, data_rcv1.py, data_nyt.py).
• Model / Training Layer:
• The training module (train.py) which accepts several hyperparameters and options to select contrastive learning mode and graph encoder usage.
• The “model” directory containing key modules:
· contrast.py: likely implements the contrastive learning loss or methodology.
· graph.py: handles graph- or tree-structured encoding to incorporate hierarchy.
· optim.py: manages optimization strategies.
• The main training routine handles data ingestion, batch processing, model training (with options for gradient accumulation, warmup, etc.), and checkpoint saving.
• Evaluation / Testing Layer:
• test.py and eval.py provide endpoints for model evaluation.
• They use checkpoints (stored in checkpoints/DATA-NAME) based on best macro-F1 and micro-F1 values.
• Utilities and Configuration:
• Global utilities in utils.py.
• Requirement file (requirement.txt) and additional configuration via command-line arguments (as described in the README).
Relationships and Interactions
• Data flows from the raw datasets into the preprocessing scripts. Preprocessed data is then fed into the training module.
• The training script calls the modules inside the “model” directory (contrast.py, graph.py, optim.py) that together implement the contrastive learning architecture and incorporate hierarchical relationships.
• Once trained, the resulting model checkpoints are passed to the testing script, which evaluates performance using test.py and/or eval.py.
• Utility functions (in utils.py) support common tasks across preprocessing, training, and evaluation.
Important Architectural Patterns and Design Principles
• Modular Design: The project separates the concerns of data handling, model architecture, training procedures, and evaluation.
• Layered Architecture: Separate “data” (loading/preprocessing), “model” (learning components), and “evaluation” layers enable clear responsibilities.
• Command-Line Configurability: The use of arguments in train.py and test.py allows flexibility and reproducibility, with different settings for various datasets.
• Extensibility: The modularity (especially with model components separated into contrast, graph, etc.) provides points where additional improvements or alternative encodings can be integrated.
Relevant Technologies and Frameworks
• Python (>=3.6)
• PyTorch (for neural network implementation)
• Transformers (for language models)
• Fairseq (for sequence modeling)
• PyTorch Geometric, torch-scatter, and torch-sparse (for graph operations)
Elements
• Clear labels for each component:
▪ For example: “Data Preprocessing – WebOfScience”, “Contrastive Module”, “Graph Encoder”, “Training Engine”, “Evaluation Module”.
• The data ingestion and preprocessing pipelines (for each dataset).
• The core model training pipeline which integrates contrastive learning and hierarchical graph encoding.
• The evaluation/testing procedures that access saved checkpoints.