How nslabteam/unilm works
This repository is a research suite of pretrained-model projects and downstream NLP, vision, speech, and document applications—not one integrated service. The README describes pretraining and adaptation capabilities, while the tree shows separate model implementations, training or inference entrypoints, and shared toolkit code. The graph preserves those documented product branches without implying a shared runtime pipeline. Relationships between projects are included only where the README explicitly describes reuse; several subsystems are represented at their available entrypoint or model-file granularity.
Main entry points: run_seq2seq.py LayoutReader · pic_inference.py TrOCR Inference · run_class_finetuning.py BEiT Finetuning · layoutlm.py LayoutLM
Components 16
Language Models
- AdaLM Adaptation
adalm/finetune/run_pico.py - InfoXLM
infoxlm/src-infoxlm/infoxlm/models/infoxlm.py - DeltaLM
deltalm/deltalm/models/deltalm.py - UniLM Seq2Seq
unilm-v1/src/biunilm/run_seq2seq.py
Vision And Speech
- BEiT Pretraining
beit/run_beit_pretraining.py - BEiT Finetuning
beit/run_class_finetuning.py - WavLM Speech
wavlm/WavLM.py
Document AI
- LayoutLM
layoutlm/deprecated/layoutlm/modeling/layoutlm.py - LayoutLMv2
layoutlmft/layoutlmft/models/layoutlmv2/modeling_layoutlmv2.py - LayoutXLM
layoutlmft/layoutlmft/models/layoutxlm/modeling_layoutxlm.py - MarkupLM
markuplm/markuplmft/models/markuplm/modeling_markuplm.py
Training Toolkit
- XTune Toolkit
xtune/src/pequod/training/trainer.py - Seq2Seq Fine-tuning
s2s-ft/run_seq2seq.py
Applications
- LayoutReader
layoutreader/run_seq2seq.py - TrOCR Inference
trocr/pic_inference.py
Other components
- Researcher
Connections 16
- Researcher → AdaLM Adaptation: adapts models
- Researcher → InfoXLM: pretrains language
- Researcher → DeltaLM: pretrains generation
- Researcher → UniLM Seq2Seq: uses language models
- Researcher → BEiT Pretraining: pretrains vision
- Researcher → BEiT Finetuning: finetunes vision
- Researcher → WavLM Speech: pretrains speech
- Researcher → LayoutLM: uses document AI
- Researcher → LayoutLMv2: uses document AI
- Researcher → LayoutXLM: uses multilingual documents
- Researcher → MarkupLM: pretrains markup
- Researcher → Seq2Seq Fine-tuning: fine-tunes seq2seq
- Researcher → LayoutReader: detects reading order
- Researcher → TrOCR Inference: recognizes text
- LayoutReader → LayoutLM: built with
- TrOCR Inference → BEiT Finetuning: uses pretrained vision