How shenwzh3/rgat-absa works
Project type and purpose
• This is an open-source machine-learning research project: a PyTorch-based implementation of the “Relational Graph Attention Network for Aspect-based Sentiment Analysis.”
• Its primary purpose is to preprocess text corpora (SemEval and Twitter), build relational graph representations of sentences, train a GAT/GNN model on those graphs, and evaluate sentiment classification performance.
File structure
Top-level directories and scripts:
• data/ – raw data (XML, JSON) and preprocessed assets (PKL files, parsed JSONs) for SemEval and Twitter.
• data_preprocess_semeval.py & data_preprocess_twitter.py – extraction & dependency-parsing pipelines.
• datasets.py – PyTorch Dataset wrappers over preprocessed files.
• tree.py & tree_samples.json – constructs sentence dependency graphs.
• model*.py (model.py, model_gcn.py, model_utils.py) – definition of embedding layers (GloVe, BERT), graph attention layers, model forward logic.
• trainer.py – encapsulates training loop, loss, backprop, checkpointing.
• run.py & run.sh – CLI interface: parameter parsing, orchestrates preprocess (optional), dataset loading, model instantiation, training, evaluation.
• data/output-*/ – directories capturing model checkpoints, cached embeddings/vocabs, and evaluation results.
Key insights from README
• Uses external pre-trained embeddings: Stanford GloVe & HuggingFace BERT.
• Preprocessing may be skipped if using provided data/.
• Training entrypoint: run.sh → run.py, parameterizable.
• Core dependencies: Python 3.6, PyTorch 1.2.0, Transformers, CUDA.
Main components
a. Data Layer
– Raw Data Sources: SemEval XML, Twitter text
– Preprocessed Data: parsed JSON, PKLs (vocab, embeddings)
b. Preprocessing Pipeline
– data_preprocess_*.py → dependency parsing → JSON outputs
c. Dataset Module
– datasets.py wraps processed files into PyTorch Datasets & DataLoaders
d. Embedding Interfaces
– GloVeLoader (reads glove_dir)
– BERTLoader (loads pretrained bert-base-uncased)
e. Graph Construction
– tree.py builds adjacency matrices / edge lists / relation-type encodings
f. Model
– model_gcn.py: Relational Graph Attention layers, classifier head
– model_utils.py: utilities for masks, attention mechanism
g. Training Engine
– trainer.py: training loop, optimizer setup, checkpoint saving
h. Runner / Orchestrator
– run.py / run.sh: CLI parameter parser, component instantiation, invokes Trainer
i. Outputs
– Checkpoints, cached PKLs (vocab & embeddings), eval_results.txt
Component interactions and data flow
• DataLoader → Batch of (graph, features, labels) → Model → Predictions
• Predictions + Labels → Trainer computes Loss → Backprop → Model parameters updated
• Trainer → Checkpoint manager writes model weights & cached data to data/output-*
• Trainer after each epoch or at end → Evaluation module writes eval_results.txt
Architectural patterns & principles
– Pipeline pattern: distinct stages (preprocess → dataset → model → train → eval)
– Modular design: interchangeable embedding backends (GloVe vs. BERT)
– Separation of concerns: data handling vs. model vs. training logic
Technologies & frameworks
– Python 3.6.8, PyTorch 1.2.0 (autograd, Datasets, DataLoader)
– HuggingFace Transformers for BERT
– CUDA 9.0 for GPU acceleration
– Standard NLP tools for dependency parsing (e.g., spaCy or biaffine parser)