# zerorin/bertgcn architecture

> Architecture diagram of the GitHub repository zerorin/bertgcn, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/zerorin/bertgcn
- Repository: https://github.com/zerorin/bertgcn

## Overview

##### Project type and purpose
- This is a research‐oriented machine learning project (“BertGCN”) for transductive text classification.
- It combines two main deep learning modules: a pretrained BERT encoder (for rich text embeddings) and a Graph Convolutional Network (GCN) (to propagate label information over a heterogeneous document‐word graph).

##### File structure
Top level:
- build_graph.py – constructs the document‐word graph from raw corpora in data/
- finetune_bert.py – fine‐tunes a pretrained BERT on each dataset’s training split
- train_bert_gcn.py – joint training script for the Bert+GCN model
- model/ – implementations of graph convolution modules (GCN, GAT, edge‐weighted GCN) and a unified models.py
- data/ – raw text files and cleaned corpora for multiple benchmarks (20NG, R8, R52, MR, Ohsumed)
- utils/ – helper functions (e.g., preprocessing, logging)
- requirements.txt – dependencies: Python, PyTorch, DGL, HuggingFace transformers, etc.

##### Key README insights
- Workflow steps:
1. Build the graph
2. Fine‐tune BERT on each dataset
3. Train BertGCN with a balancing factor λ between BERT and GCN outputs
- External frameworks: PyTorch for modeling, DGL (Deep Graph Library) for graph operations, HuggingFace Transformers for BERT.
- Model variants: BertGCN, RoBERTaGCN, BertGAT, RoBERTaGAT.

##### Essential elements
- Clear labels on each component and artifact
- Arrowheads showing direction of data/control flow
- Notes on important hyperparameters (e.g., λ balancing factor m)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data & Graph Phase
    subgraph "Data & Graph Construction"
        direction TB
        Raw["Raw Text Corpora"]:::dataPhase
        BuildGraph["build_graph.py"]:::dataPhase
        GraphObj["Heterogeneous Graph\ngraph.pkl"]:::dataPhase
    end

    Raw -->|"reads raw corpora"| BuildGraph
    BuildGraph -->|"outputs graph.pkl"| GraphObj

    %% BERT Fine-tuning Phase
    subgraph "BERT Fine-tuning"
        direction TB
        FineTune["finetune_bert.py"]:::bertPhase
        BERTCP["BERT Checkpoint\n*.pt"]:::bertPhase
    end

    Raw -->|"training text"| FineTune
    FineTune -->|"saves .pt checkpoint"| BERTCP

    %% GCN Training & Inference Phase
    subgraph "GCN Training & Inference"
        direction TB
        JointTrain["train_bert_gcn.py"]:::gcnPhase
        subgraph "Graph Neural Network Modules"
            direction TB
            GCNMod["GCN Module\ntorch_gcn.py"]:::gcnPhase
            GATMod["GAT Module\ntorch_gat.py"]:::gcnPhase
            EWGCN["Edge-Weighted GCN\ngraphconv_edge_weight.py"]:::gcnPhase
            UniModels["Unified Models\nmodels.py"]:::gcnPhase
        end
        ModelCP["BertGCN Model Checkpoint\n*.pt"]:::gcnPhase
        Eval["Classification Outputs\n& Metrics"]:::gcnPhase
    end

    GraphObj -->|"loads graph.pkl"| JointTrain
    BERTCP -->|"loads checkpoint"| JointTrain
    JointTrain -->|"saves .pt checkpoint"| ModelCP
    ModelCP -->|"evaluation"| Eval

    %% Infrastructure & Utilities
    subgraph "Infrastructure & Utilities"
        direction TB
        Utils["utils/utils.py"]:::infra
        Req["requirements.txt"]:::infra
        Libs["External Libraries"]:::infra
        PyTorch["PyTorch"]:::infra
        DGL["DGL"]:::infra
        HF["HuggingFace\nTransformers"]:::infra
    end

    Utils -.-> BuildGraph
    Utils -.-> FineTune
    Utils -.-> JointTrain
    Req -.-> Utils
    Libs --> PyTorch
    Libs --> DGL
    Libs --> HF

    %% Click Events
    click BuildGraph "https://github.com/zerorin/bertgcn/blob/main/build_graph.py"
    click Raw "https://github.com/zerorin/bertgcn/tree/main/data/"
    click FineTune "https://github.com/zerorin/bertgcn/blob/main/finetune_bert.py"
    click GCNMod "https://github.com/zerorin/bertgcn/blob/main/model/torch_gcn.py"
    click GATMod "https://github.com/zerorin/bertgcn/blob/main/model/torch_gat.py"
    click EWGCN "https://github.com/zerorin/bertgcn/blob/main/model/graphconv_edge_weight.py"
    click UniModels "https://github.com/zerorin/bertgcn/blob/main/model/models.py"
    click JointTrain "https://github.com/zerorin/bertgcn/blob/main/train_bert_gcn.py"
    click Utils "https://github.com/zerorin/bertgcn/blob/main/utils/utils.py"
    click Req "https://github.com/zerorin/bertgcn/blob/main/requirements.txt"

    %% Styles
    classDef dataPhase fill:#cce5ff,stroke:#004085,color:#004085
    classDef bertPhase fill:#d4edda,stroke:#155724,color:#155724
    classDef gcnPhase fill:#fff3cd,stroke:#856404,color:#856404
    classDef infra fill:#e2e3e5,stroke:#383d41,color:#383d41
```
