How siddhavi/detecting-llm-hallucinations-on-cpu-based-systems--a-cloud-assisted-approach works
This checkout contains the VM3 judge and model-preparation/training scripts; the README describes VM1 and VM2 as separately deployed services, but their source files are not in this tree. The principal workflow is a judge request, a deterministic fact-check override or five weighted signals, and a thresholded verdict. Training is a separate data-preparation branch. Runtime links among the five signal functions are documented in the judge source; external deployment links rely on the README.
Main entry points: app.py Weighted ensemble · data_prep.py Data preparation · train_distilbert_v1.py DistilBERT V1 training · train_distilbert_v2.py DistilBERT V2 training
Components 16
Judge runtime
- Judge API
app.py - Fact override
app.py - Weighted ensemble
app.py
Judging signals
- SelfCheckGPT
app.py - DistilBERT V2
app.py - Flan-T5 CoT
app.py - DistilBERT V1
app.py - Qwen judge
app.py
Data and training
- Data preparation
data_prep.py - DistilBERT V2 training
train_distilbert_v2.py - Flan-T5 training
train_flant5.py - DistilBERT V1 training
train_distilbert_v1.py
Other components
- Evaluator
- Streamlit dashboard
- VM2 LLM service
- HaluBench dataset
Connections 14
- Evaluator → Streamlit dashboard: enters passage
README.md - Streamlit dashboard → VM2 LLM service: requests responses
README.md - Streamlit dashboard → Judge API: submits evaluation
README.md - Judge API → Fact override: checks facts
app.py - Judge API → Weighted ensemble: scores request
app.py - Weighted ensemble → SelfCheckGPT: scores consistency
app.py - Weighted ensemble → DistilBERT V2: scores answer
app.py - Weighted ensemble → Flan-T5 CoT: scores answer
app.py - Weighted ensemble → DistilBERT V1: scores answer
app.py - Weighted ensemble → Qwen judge: scores answer
app.py - HaluBench dataset → Data preparation: loads dataset
data_prep.py - Data preparation → DistilBERT V2 training: supplies training data
README.md - Data preparation → Flan-T5 training: supplies training data
README.md - DistilBERT V1 training → DistilBERT V1: trains baseline