How nustm/vlp-mabsa works
VLP-MABSA is a research system for multimodal aspect-based sentiment analysis. Its documented workflow prepares text spans, sentiment labels, image-region features and ANP annotations; pretrains on multimodal objectives; then fine-tunes on Twitter data for aspect/opinion extraction and aspect sentiment classification. The graph separates data preparation, modeling, and task-specific training. Some source excerpts are partial or absent, so links are limited to documented capabilities and visible orchestration; uncertain internal calls are omitted.
Main entry points: MAESC_model.py Multimodal task model · pretrain.py Pre-training entry · twitter_sc_training.py Twitter SC training · twitter_ae_training.py Twitter AE training
Components 20
Data and encoding
- MVSA dataset
src/data/dataset.py - Twitter datasets
src/data/dataset.py - Condition tokenizer
src/data/tokenization_new.py - Task collation
src/data/collation.py - Dataset metadata
src/data/jsons
Pre-training
- Pre-training entry
pretrain.py - Pre-training loop
src/training.py - Training state
src/utils.py
Model and generation
- Model configuration
src/model/config.py - Multimodal task model
src/model/MAESC_model.py - BART backbone
src/model/modeling_bart.py - Task modules
src/model/modules.py - Sequence generation
src/model/generater.py
Downstream tasks
- Twitter SC training
twitter_sc_training.py - Twitter AE training
twitter_ae_training.py - MAESC training
MAESC_training.py - Fine-tuning and evaluation
src/training.py - Metrics and evaluation
src/eval_utils.py
Other components
- Researcher
- Prepared datasets and features
Connections 35
- Researcher → Pre-training entry: starts pretraining
- Researcher → Twitter SC training: starts sentiment task
- Researcher → Twitter AE training: starts extraction task
- Researcher → MAESC training: starts MAESC task
- Prepared datasets and features → MVSA dataset: provides data
- Prepared datasets and features → Twitter datasets: provides data
- Dataset metadata → MVSA dataset: configures inputs
- Dataset metadata → Twitter datasets: configures inputs
- Pre-training entry → MVSA dataset: loads samples
- Pre-training entry → Condition tokenizer: constructs
- Pre-training entry → Task collation: constructs
- Pre-training entry → Model configuration: loads configuration
- Pre-training entry → Multimodal task model: creates model
- Pre-training entry → Pre-training loop: runs epochs
- Pre-training loop → Multimodal task model: forwards task batches
- Pre-training loop → Training state: uses training state
- Twitter SC training → Twitter datasets: loads samples
- Twitter AE training → Twitter datasets: loads samples
- MAESC training → Twitter datasets: loads samples
- Twitter SC training → Condition tokenizer: constructs
- Twitter AE training → Condition tokenizer: constructs
- MAESC training → Condition tokenizer: constructs
- Twitter SC training → Multimodal task model: creates model
- Twitter AE training → Multimodal task model: creates model
- MAESC training → Multimodal task model: creates model
- Twitter SC training → Fine-tuning and evaluation: runs fine-tuning
- Twitter AE training → Fine-tuning and evaluation: runs fine-tuning
- MAESC training → Fine-tuning and evaluation: runs fine-tuning
- Fine-tuning and evaluation → Multimodal task model: forwards batches
- Fine-tuning and evaluation → Metrics and evaluation: evaluates predictions
- Multimodal task model → BART backbone: uses backbone
- Multimodal task model → Task modules: uses task modules
- Multimodal task model → Sequence generation: uses generation
- Multimodal task model → Model configuration: uses configuration
- Condition tokenizer → Task collation: encodes batch inputs