How atanu2531/hats works
Project Type and Purpose
• This is a research-oriented machine learning codebase implementing HATS (Hierarchical Graph Attention Network) for two tasks:
a. Node classification (predicting stock movement at the node/company level)
High-Level File Structure Analysis
• Top-level
– README.md (project overview, usage, dependencies)
– node_classification/
– graph_classification/
• Each task directory
– Makefile & download.sh (data acquisition & execution entry points)
– src/
· base/ (base_model.py, base_train.py) – abstract classes for model structure & training loop
· config.py – hyperparameters, file paths, task-specific settings
· dataset.py – data loading + preprocessing from local files
· models/ (HATS.py) – concrete implementation of the hierarchical graph attention network
· trainer.py or graph_trainer.py – orchestration of forward/backward passes for node vs graph tasks
· evaluator.py or graph_evaluator.py – metric computation & result aggregation
· logger.py – logging utility
· main.py – ties together config → dataset → trainer → evaluator → logger
Key Components
A. External Data Sources (Yahoo Finance API, Wikidata)
B. Data Acquisition Layer
• download.sh (Bash script) → local raw data files
• Makefile target “download”
C. Configuration Module
• config.py defines paths, hyperparameters (learning rate, epochs, batch size)
D. Data Preprocessing & Dataset Module
• dataset.py reads raw CSV/JSON, applies normalization, constructs graph objects (adjacency, features)
E. Base Classes
• base_model.py – abstract graph neural net interface
• base_train.py – generic training loop implementation
F. Model Implementation
• HATS.py – hierarchical graph attention layers, aggregation from node to higher levels
G. Task-Specific Trainers
• node trainer vs graph trainer – batch creation, forward/backward, checkpointing
H. Evaluators
• evaluator.py / graph_evaluator.py – accuracy, F1, AUC calculations
I. Logger
• logger.py – metrics & hyperparameter logging to console or file
J. Orchestration (main.py)
• initializes config → instantiates dataset, model, trainer, evaluator → runs training/testing
Relationships & Data Flows
External Data → download.sh → Raw Data Storage → dataset.py → Processed Dataset
Processed Dataset + Config → Trainer → Model ← Base Classes
Trainer → Evaluator → Logger → Output Results
• For graph_classification, an extra “data_type” parameter flows from Makefile into config to select sub-graph sector.
Architectural Patterns & Design Principles
• Pipeline architecture: clear separation of concerns (data, config, model, training, evaluation)
• Template Method pattern in base_train.py / base_model.py providing hook methods overridden by HATS.py
• Modular design: identical module structure reused across two tasks for maintainability
• Script-driven orchestration via Makefile + main.py, enabling reproducibility
Technologies & Frameworks
• Python 3.x
• TensorFlow 1.11.0 for all neural network operations
• NumPy for numerical preprocessing
• Bash + Makefile for data download and experiment orchestration