How ivanvmoreno/correctness-model-internals works
Project type and purpose
• This is a research‐oriented data‐processing pipeline (“full stack” in the data‐science sense) for probing large language model (LLM) internals.
• It ingests raw question/answer datasets, runs questions through LLMs, captures intermediate activations, trains linear probes to predict correctness, and evaluates probe performance.
High‐level components
• Orchestration & environment
– Docker container (docker/Dockerfile + scripts) provides reproducible compute.
– DVC (dvc.yaml, dvc.lock) and Taskfile.yaml manage the multi‐stage pipeline and data versioning.
• Data stores
– Raw input datasets (datasets/raw/*).
– Custom/derived CSVs (datasets/custom/*).
– Captured activations and embeddings (artifacts produced by pipeline stages).
– Experiment outputs (experiment_results/*, evaluations_revisited/*).
• Pipeline stages (src/stages/*.py)
1. download_hf: fetch HF datasets/models
2. format_datasets: normalize raw data into CSV/parquet
3. generate_answers: run LLM on questions
4. capture_activations: extract hidden‐layer tensors mid‐inference
5. postprocess_activations: aggregate or reshape tensors
6. generate_embeddings: convert activations into fixed‐size embeddings
7. train_classifiers & train_embedding_assessors: fit linear probes
8. collect_classification_stats: compute predictions vs. ground truth
9. evaluate_answers: compare against baselines
• Core libraries (src/*.py)
– classifying/: activation handlers, direction calculators, classifier utilities
– llm_judge.py & model.py: LLM interface and model wrapper
– data.py, utils/*: configuration, logging, data I/O, metrics
– visualisations/: functions for embedding‐probe result plots
• Analysis & reporting
– Jupyter notebooks (notebooks/*) for exploratory analysis, best‐layer search, plotting.
– Custom scripts for “truth_analysis_arnau” and “verbalized_confidence” experiments.
Interactions & data flow
1. User invokes Taskfile or DVC:
‣ Docker container spins up; DVC fetches raw data.
2. format_datasets reads datasets/raw → writes standardized CSV/parquet.
3. generate_answers sends questions to HF or RunPod pods → writes model outputs.
4. capture_activations hooks into model runs → stores layer activations.
5. postprocess_activations & generate_embeddings convert activations to vectors.
6. train_classifiers & train_embedding_assessors read embeddings + labels → produce probe weights and prediction scores.
7. collect_classification_stats & evaluate_answers compute accuracy, ROC AUC, directional scores → saved under experiment_results and evaluations_revisited.
Architectural patterns & principles
• Pipeline as directed acyclic graph (DAG) managed by DVC.
• Modular stages: each “src/stages” script is a self‐contained step that reads inputs and writes outputs.
• Separation of concerns:
– Data ingestion vs. preprocessing vs. model inference vs. feature extraction vs. modeling vs. evaluation vs. visualization.
• Configuration in params.yaml and src/utils/config.py drives reproducibility and parameter sweeps.
• Containerization (Docker) ensures environment parity.
Technologies & external services
• Python 3.x, pyproject.toml, requirements-cpu.txt for dependencies.
• DVC for data pipeline orchestration and artifact versioning.
• Docker for containerized execution.
• Hugging Face and/or RunPod for LLM inference.
• Jupyter notebooks for ad‐hoc analysis.
• Linear classifiers (scikit-learn or similar) for probes.