How tearcake/video-sentiment-model works
This repository trains and serves a tri-modal video emotion and sentiment classifier. MELD examples are tokenized and converted into text, frame, and audio features; the training loop optimizes emotion and sentiment heads and saves a model. At inference, a video request is transcribed and split into utterances, processed, classified, and serialized. SageMaker scripts support cloud training and endpoint deployment. The main training and inference paths are sampled; some inference details and the deployment model’s runtime wiring are not, so uncertain links are omitted.
Main entry points: inference.py Inference handler · train.py Training loop · deploy_endpoint.py Model repackaging · train_sagemaker.py Cloud training launcher
Components 17
MELD Training
- Dataset loader
training/meld_dataset.py - Training loop
training/train.py - Trainer and metrics
training/models.py
Classification Models
- Multimodal model
training/models.py - Serving model
deployment/models.py
Video Inference
- Inference handler
deployment/inference.py - Speech transcription
deployment/inference.py - Utterance segmentation
deployment/inference.py - Audio/video processing
deployment/inference.py - JSON predictions
deployment/inference.py
SageMaker Cloud
- Model artifacts
- Cloud training launcher
train_sagemaker.py - Model repackaging
deployment/deploy_endpoint.py - Endpoint provisioning
deployment/deploy_endpoint.py
Other components
- Video requester
- MELD dataset
- SageMaker
Connections 19
- MELD dataset → Dataset loader: provides examples
README.md - Training loop → Dataset loader: loads batches
training/train.py - Training loop → Trainer and metrics: creates trainer
training/train.py - Training loop → Multimodal model: creates model
training/train.py - Trainer and metrics → Multimodal model: trains and evaluates
training/models.py - Training loop → Model artifacts: saves weights
training/train.py - Cloud training launcher → SageMaker: launches training
train_sagemaker.py - Cloud training launcher → Training loop: configures entry script
train_sagemaker.py - Model artifacts → Model repackaging: downloads artifact
deployment/deploy_endpoint.py - Model repackaging → Model artifacts: uploads package
deployment/deploy_endpoint.py - Model repackaging → Endpoint provisioning: provides model package
deployment/deploy_endpoint.py - Endpoint provisioning → SageMaker: creates endpoint
deployment/deploy_endpoint.py - Video requester → Inference handler: submits video
deployment/inference.py - Inference handler → Speech transcription: transcribes video
deployment/inference.py - Inference handler → Utterance segmentation: extracts segments
deployment/inference.py - Utterance segmentation → Audio/video processing: uses processors
deployment/inference.py - Inference handler → Serving model: runs predictions
- Inference handler → JSON predictions: serializes result
deployment/inference.py - Endpoint provisioning → Inference handler: runs inference code
deployment/deploy_endpoint.py