How alvi-uiu/multihaludet works
MultiHaluDet is a research pipeline for detecting hallucinations in multilingual question-answer pairs by probing frozen LLM hidden states, training a deep detector, and stacking its out-of-fold features in a calibrated ensemble. The main workflow runs through run_pipeline.py: load a dataset, extract and persist features, train across folds and persist embeddings, then predict, evaluate, and plot results. The graph includes the documented model, loss, and augmentation capabilities; unsampled subsystem wiring is limited to what the README documents.
Main entry points: run_pipeline.py Pipeline stages · trainer.py Fold training · feature_extractor.py Hidden-state probing · loader.py Dataset loader
Components 17
Data and probing
- Pipeline stages
run_pipeline.py - Model registry
src/config.py - Dataset loader
src/data/loader.py - Hidden-state probing
src/data/feature_extractor.py - Feature artifacts
Deep detection
- Fold training
src/training/trainer.py - MultiHaluDet
src/models/multihaludet.py - Training losses
src/models/losses.py - MixUp and CutMix
src/training/augmentations.py - OOF embeddings
Ensemble and evaluation
- Log-odds stacker
src/ensemble/meta_learner.py - Evaluation metrics
src/utils/metrics.py - Result plots
src/utils/visualization.py - Result artifacts
Other components
- Pipeline user
- Dataset sources
- Frozen LLM
Connections 18
- Pipeline user → Pipeline stages: selects stages
- Pipeline stages → Model registry: gets configuration
- Pipeline stages → Dataset loader: loads samples
- Dataset loader → Dataset sources: loads dataset
- Pipeline stages → Frozen LLM: loads model
- Pipeline stages → Hidden-state probing: extracts features
- Hidden-state probing → Frozen LLM: probes hidden states
- Pipeline stages → Feature artifacts: writes and reads
- Pipeline stages → Fold training: trains folds
- Fold training → MultiHaluDet: optimizes and evaluates
- Fold training → Training losses: computes loss
- Fold training → MixUp and CutMix: augments batches
- Fold training → Hidden-state probing: uses hybrid dataset
- Pipeline stages → OOF embeddings: writes and reads
- Pipeline stages → Log-odds stacker: fits and predicts
- Pipeline stages → Evaluation metrics: evaluates probabilities
- Pipeline stages → Result plots: plots results
- Pipeline stages → Result artifacts: writes plots