How dhavalpotdar/graph-convolution-on-structured-documents works
Project type and purpose
• This is a research-oriented Python codebase that converts structured document layouts into graph representations and applies a (work-in-progress) Graph Convolutional Neural Network (GCN) for node-level classification.
• Key use case: information extraction from invoices, receipts, tables, etc., by treating layout entities as graph nodes and running GCNs over them.
File structure
• Top-level scripts and modules:
• grapher.py – builds a graph (nodes+edges) from OCR object maps
• graph_operator.py – constructs adjacency matrices, feature matrices for GCN
• layers.py – defines custom GCN layers in TensorFlow
• models.py – composes those layers into a trainable model
• object_map_generation.py – (optional) helper to generate or preprocess object maps
• text_utils.py – extracts textual features from OCR’d text
• test_read.py – example of reading/parsing input
• Supporting directories:
• data/ – sample input object_map.csv
• outputs/ – generated graph image (object_tree.png/jpg) and connections.csv
• unit tests/ – module-specific tests for adjacency layers, graph operator, text utils
• Config/build files:
• requirements.txt – dependency list (TensorFlow 1.8, numpy, pandas, etc.)
Extract insights from the README
• Core pipeline:
2. grapher.py → link each object to nearest right/under neighbors → output graph image + CSV of connections
3. Build adjacency & feature matrices (graph_operator.py + text_utils.py)
4. Feed into GCN layers (layers.py) combined in models.py for node classification
• Model is TensorFlow 1.8, under active development.
Components
a. Input & Preprocessing
• External OCR Tool → object_map.csv (data folder)
• (Optional) object_map_generation.py for formatting/preprocessing
b. Graph Generation
• grapher.py
• Reads object_map.csv
• Computes nearest-neighbor links
• Outputs connections.csv + object_tree.png in outputs/
c. Feature & Adjacency Preparation
• text_utils.py – textual feature extraction (embeddings, counts)
• graph_operator.py – builds adjacency matrices and node feature matrices
d. Graph Convolutional Model
• layers.py – defines GraphConvLayer, ReLU, dropout, etc.
• models.py – composes layers into a forward pass, loss, training ops
e. Training & Inference
• TensorFlow session code (inside models.py or external script)
• Outputs node classification results
f. Testing & Validation
• unit tests/ covering adjacency, graph operator, text utils
g. Supporting artifacts
• requirements.txt
• Example reader: test_read.py
Relationships & data flow
OCR Tool → object_map_generation (optional) → Grapher → Outputs (graph CSV & image) → Feature Extraction + Adjacency Builder → GCN Model → Predictions
• Side branch: unit tests target each module individually.
Architectural patterns & design principles
• Pipeline/ETL style: clear separation between data ingestion, transformation, and modeling
• Modular design: each major step is a separate Python module/script
• Test‐driven verification: each module has its own unit tests
Relevant technologies & dependencies
• Python 3.x, Pandas, NumPy
• TensorFlow 1.8 for model layers & training
• CSV and image output for graph visualization