How m3rg-iitd/matscibert works
Project type and purpose
• This is an open-source NLP toolkit centering on MatSciBERT, a domain-specific BERT model for materials-science text mining.
• It provides:
• A pretraining pipeline to train/extend MatSciBERT
• A text-normalization utility
• Three fine-tuning pipelines:
– Abstract/label classification (“cls”)
– Named-entity recognition (“ner”)
– Relation classification (“relation_classification”)
High-level components
A. Core model & utilities
• normalize_text.py – text-normalization
• HuggingFace Transformers (AutoTokenizer, AutoModel)
• The MatSciBERT checkpoints on HuggingFace
B. Pretraining
• pretraining/ – scripts to prepare corpus (normalize_corpus.py) and launch matscibert_pre_train.py
C. Fine-tuning modules
1. cls/
– cls_baseline/ – PyTorch baseline code (dataloader, models, train.py)
– custom datasets (glass_non_glass)
– cls/model/ – any production scripts or artifacts
2. ner/
– ner_datasets.py, ner.py – code for NER experiments
– conlleval.py, run.sh for evaluation
– ner/models/ – sub-folders for each dataset/model variant (matscholar, sofc, sofc_slot)
3. relation_classification/
– rel_baseline/ – baseline code similarly structured to cls_baseline
– datasets/ – annotated corpora with .txt/.ann files
– model/ – run.sh to launch experiments
– relation_classification.py, run.sh
Relationships and data flow
1. Data sources (raw text, CSV, .ann, .txt)
→ normalization utility
→ tokenization (HuggingFace tokenizer)
→ MatSciBERT embeddings
→ task-specific data loader
→ task-specific model head (classification, sequence labeling, relation scorer)
→ evaluation scripts (conlleval, custom metrics)
2. Pretraining pipeline runs on a large materials corpus, produces the MatSciBERT weights that feed all fine-tuning modules.
Key architectural patterns & principles
• Modular separation: one folder per task, each with baseline implementation, data loader, model definitions, training script
• Reuse of core utilities: normalization + HuggingFace APIs shared by all tasks
• Clear “data → preprocess → model → evaluate” pipeline structure
• Configuration via environment.yaml or requirements.txt and run.sh scripts
Essential elements
• Shared utilities explicitly to illustrate code reuse
• Per-task baseline code and custom models to highlight extensibility
• Run.sh / environment.yaml / requirements.txt as the configuration/deployment layer