How emilfabel04/dvx works
dvx is a pre-alpha design for measuring a dataset’s impact on a target language model. The intended workflow loads and samples data, scans for novelty, duplicates, and model exposure, then fine-tunes and evaluates the model against benchmarks and a generic-data control to produce an impact report. The README also describes scaling analysis, contamination and cluster contributions, and a future local corpus. The latter evaluation and reporting stages are documented capabilities, not verified implementations in the sampled source; no runtime orchestrator was shown.
Main entry points: data.py Dataset Loading · dvx Benchmark Evaluation · losses.py Model Scoring · calibration.py Exposure Calibration
Components 18
Inputs and Models
- Dataset Loading
src/dvx/data.py - Model Loading
src/dvx/models.py - Run Profiles
src/dvx/config.py - Prompt Encoding
src/dvx/prompting.py
Dataset Scans
- Model Scoring
src/dvx/scan/losses.py - Duplicate Detection
src/dvx/scan/duplicates.py - Novelty Buckets
src/dvx/scan/novelty.py - Exposure Calibration
src/dvx/scan/calibration.py - Exposure Estimate
src/dvx/scan/exposure.py
Impact Evaluation
- LoRA Fine-tuning
src/dvx - Benchmark Evaluation
src/dvx - Data Scaling Curve
src/dvx - Contamination Analysis
src/dvx - Cluster Contributions
src/dvx
Reports and Records
- Impact Report
src/dvx - Impact Corpus
src/dvx
Other components
- Dataset Evaluator
- Wikipedia
Connections 22
- Dataset Evaluator → Dataset Loading: provides data
README.md - Dataset Evaluator → Model Loading: selects model
README.md - Dataset Loading → Duplicate Detection: supplies examples
src/dvx/scan/duplicates.py - Dataset Loading → Novelty Buckets: supplies examples
src/dvx/scan/novelty.py - Dataset Loading → Prompt Encoding: provides examples
src/dvx/prompting.py - Dataset Loading → Model Scoring: provides examples
src/dvx/scan/losses.py - Prompt Encoding → Model Scoring: encodes examples
src/dvx/scan/losses.py - Model Loading → Model Scoring: scores with model
- Exposure Calibration → Dataset Evaluator: fetches from
- Exposure Calibration → Wikipedia: fetches texts
src/dvx/scan/calibration.py - Model Scoring → Exposure Estimate: provides log-probs
src/dvx/scan/exposure.py - Exposure Calibration → Exposure Estimate: provides calibration texts
- Dataset Evaluator → LoRA Fine-tuning: requests impact run
README.md - LoRA Fine-tuning → Benchmark Evaluation: scores before and after
README.md - Benchmark Evaluation → Data Scaling Curve: measures across data sizes
README.md - Benchmark Evaluation → Contamination Analysis: compares raw and clean gain
README.md - Benchmark Evaluation → Cluster Contributions: reports cluster contributions
README.md - Benchmark Evaluation → Impact Report: contributes results
README.md - Data Scaling Curve → Impact Report: contributes curve
README.md - Contamination Analysis → Impact Report: contributes analysis
README.md - Cluster Contributions → Impact Report: contributes analysis
README.md - Impact Report → Impact Corpus: writes run record
README.md