How zerorin/bertgcn works
Project type and purpose
• This is a research‐oriented machine learning project (“BertGCN”) for transductive text classification.
• It combines two main deep learning modules: a pretrained BERT encoder (for rich text embeddings) and a Graph Convolutional Network (GCN) (to propagate label information over a heterogeneous document‐word graph).
File structure
Top level:
• build_graph.py – constructs the document‐word graph from raw corpora in data/
• finetune_bert.py – fine‐tunes a pretrained BERT on each dataset’s training split
• train_bert_gcn.py – joint training script for the Bert+GCN model
• model/ – implementations of graph convolution modules (GCN, GAT, edge‐weighted GCN) and a unified models.py
• data/ – raw text files and cleaned corpora for multiple benchmarks (20NG, R8, R52, MR, Ohsumed)
• utils/ – helper functions (e.g., preprocessing, logging)
• requirements.txt – dependencies: Python, PyTorch, DGL, HuggingFace transformers, etc.
Key README insights
• Workflow steps:
1. Build the graph
2. Fine‐tune BERT on each dataset
3. Train BertGCN with a balancing factor λ between BERT and GCN outputs
• External frameworks: PyTorch for modeling, DGL (Deep Graph Library) for graph operations, HuggingFace Transformers for BERT.
• Model variants: BertGCN, RoBERTaGCN, BertGAT, RoBERTaGAT.
Essential elements
• Clear labels on each component and artifact
• Arrowheads showing direction of data/control flow
• Notes on important hyperparameters (e.g., λ balancing factor m)