How dianbowork/spn4re works
Project type and purpose
• This is an open-source NLP research tool that implements and trains a joint entity and relation extraction model using Set Prediction Networks (SPN). It is a machine-learning pipeline, not a web app or microservice suite. The goal is to load benchmark datasets (NYT, WebNLG), preprocess them, instantiate a PyTorch/Transformers-based model, train it, and evaluate performance.
File structure
• Top-level:
• main.py — entry point, parses CLI args, kicks off training/evaluation
• data/ — raw datasets and preprocessing scripts for NYT and WebNLG
• models/ — model components: sequence encoder, set decoder, matcher, loss criterion, high-level SetPred4RE module
• trainer/ — Trainer class encapsulating the training loop, optimization, checkpointing, logging
• utils/ — helpers for data loading, metrics, logging (average meters), alphabet handling, generic functions
• Patterns:
• Modular ML pipeline with clear separation: data prep → model definition → trainer → evaluation
• Use of external resources: BERT checkpoint directory, PyTorch, Transformers
README for additional insights
• Confirms use of PyTorch ≥1.5.0, Transformers 2.6.0, Python 3.7
• Shows CLI commands that pass BERT directory, training hyperparameters, dataset choices (NYT partial/exact, WebNLG)
• Mentions that trained model parameters are downloadable separately