How azyoung233/mse-adapter works
Project type and purpose
• This is a research‐oriented machine-learning toolkit (“MSE-Adapter”) that plugs into large-language models (LLMs) to perform multimodal sentiment analysis and emotion recognition.
• It provides implementations for three backbones: ChatGLM3-6B, Llama2-7B, and Qwen-1.8B, each in a separate subfolder.
File structure
Top level:
• MSE-ChatGLM3-6B/, MSE-Llama2-7B/, MSE-Qwen-1.8B/ — per-backbone codebases
• README.md, requirements.txt
Inside each backbone folder:
• config/ — training-mode and task-mode configurations (classification vs. regression)
• data/ — data loaders and preprocessors (DataPre.py, TextPre.py, getFeature, getLengths, load_data)
• models/
– AMIO.py — multimodal input/output adapter
– ChatGLM3 or core-model subpackage — LLM integration, tokenization, quantization
– multiTask/CMCM.py — multitask head (sentiment & emotion)
– subNets/Textmodel.py — auxiliary unimodal text encoder
• trains/ — training routines (ATIO.py for single-task, multiTask/CMCM.py)
• utils/ — shared helpers (functions.py, metricsTop.py)
• logs/ — run logs
• run.py — entrypoint for training and evaluation
README for additional insights
• Core workflow:
a. Specify dataset directory and backbone model path in run.py’s arguments
b. Data is loaded and preprocessed into features and lengths
c. MSE-Adapter (AMIO + subNets + multiTask) is wired into the frozen or quantized LLM
d. Training with 5 seeds; results persist under results/result
• External dependencies: Hugging Face datasets, virtual environment with fixed transformers version
Details
• MultiTask head coexisting with the adapter
• If space allows, note “config_classification.py” vs. “config_regression.py” as two branches of the Config Manager
• That only adapter parameters are trainable, while the backbone LLM remains frozen or quantized