How lefteris12/multitask-schema-guided-dst works
Project type and purpose
• This is an open-source Python toolkit implementing a multi-task BERT model for Schema-Guided Dialogue State Tracking (DST).
• It provides data-processing pipelines, PyTorch datasets, training/evaluation scripts, and a baseline BERT implementation under mtsgdst/schema_guided_dst/baseline.
File structure
• Top level:
– download_data.sh – helper to clone the SGD dataset.
– mtsgdst/ – the main package.
– train.py, dst.py – CLI entrypoints for training and inference.
– config.py – global hyperparameter/config definitions.
• mtsgdst/data/ – raw data ingestion, EDA, feature extraction, dialogue processing, input-sequence construction, and two dataset classes (pytorch_dataset.py for SGD and sgd_dataset.py for low-level.
• mtsgdst/schema_guided_dst/ – implements the schema-guided logic. Within it:
– baseline/bert/ – a self-contained BERT modeling, tokenization, optimization, data utilities, and train/predict script.
– multiwoz/ – schema conversion for the MultiWOZ dataset.
– metrics/ – code to compute DST metrics.
– evaluate.py – script to measure performance against references.
README
• Describes the three sub-tasks (intent prediction, requested‐slot prediction, slot filling) all handled by one BERT.
• Provides instructions for environment setup, dataset creation, training, checkpointing, and evaluation.
1. External Data Sources
– Schema-Guided Dialogue (SGD) dataset repo
– MultiWOZ schema files (optional)
2. Data Ingestion & Preprocessing
– download_data.sh
– mtsgdst/data/*: JSON readers, EDA, feature extraction, input sequence builder
3. Dataset Serialization
– mtsgdst/data/pytorch_dataset.py → pickle files per split (train/dev/test)
4. Model Module
– BERT encoder + three classification heads (intent, requested slots, span‐extraction) in mtsgdst/models.py
– Schema encoding logic in mtsgdst/schema_guided_dst/schema.py
5. Training Orchestrator
– mtsgdst/train.py: loads config, DataLoader, model, optimizer, checkpoint manager
6. Checkpoint Storage
– checkpoints/latest.pt and best.pt
7. Evaluation & Metrics
– mtsgdst/dst.py for inference on test set
– mtsgdst/schema_guided_dst/metrics.py for joint goal accuracy, slot‐fill F1, etc.
8. Baseline Submodule (optional)
– mtsgdst/schema_guided_dst/baseline – standalone BERT baseline for comparison
• External Data → Data Ingestion & Preprocessing
• Preprocessing → Dataset Serialization
• Serialized datasets → Training Orchestrator (via PyTorch DataLoaders)
• Training → Checkpoint Storage
• Checkpoint Storage → Evaluation & Metrics
• Baseline Submodule can feed into Evaluation & Metrics as a parallel branch.
c. Architectural Patterns & Principles
• Modular layering: Data layer, Model layer, Orchestration layer, Evaluation layer.
• Single-model multi-task learning pattern: one shared encoder + multiple heads.
• Schema-guided abstraction: loose coupling between dialogue history and service schema representations.
• CLI façade: train.py and dst.py decouple implementation from run commands.
d. Key Technologies & Frameworks
• Python 3.10, PyTorch (Dataset, DataLoader, nn.Module)
• transformers-style BERT tokenizer/model (custom in baseline/bert)
• Pickle for dataset caching
• Standard JSON for dialogues and schema