How rohannx29/dataforge-dcai works
This research pipeline tests whether data-quality interventions improve manufacturing-defect classification under a fixed CNN. Its intended workflow runs dataset preparation and label-quality interventions, trains and evaluates comparable conditions, then compares experiment results. The README documents acquisition, MVTec processing, targeted augmentation, and active learning as product capabilities. Evidence is limited to the README and indexed or excerpted files; some documented stages have no verified implementation links, and the active-learning loop currently uses random selection rather than its imported sampling strategies.
Main entry points: train_baseline.py Baseline training CLI · noise_detection.py Label-noise detection · train.py Model training · evaluate.py Model evaluation
Components 28
Data preparation
- Validation CLI
scripts/run_validation.py - Dataset acquisition
src/data/acquisition.py - Dataset validation
src/data/validation.py - Sample cleaning
src/data/cleaning.py - Image preprocessing
src/data/preprocessing.py - MVTec preprocessing
src/data/mvtec_preprocessing.py - Manifest dataset
src/data/dataset.py
Label and data quality
- Label quality audit
src/labels/quality_audit.py - Label-noise detection
src/labels/noise_detection.py - Label correction
src/labels/noise_correction.py - Targeted augmentation
src/dcai/augmentation.py - Class balancing
src/dcai/class_balance.py - Outlier detection
src/dcai/outlier_detection.py
Model and learning
- Baseline training CLI
scripts/train_baseline.py - Active-learning loop
src/active_learning/loop.py - Query sampling
src/active_learning/sampling.py - CNN architecture
src/models/architectures.py - Model training
src/models/train.py - Model evaluation
src/models/evaluate.py
Experiment results
- Experiment tracking
src/experiment/tracker.py - Condition comparison
src/experiment/comparison.py - Exploratory plots
src/visualization/eda_plots.py - Results plots
src/visualization/results_plots.py
Shared utilities
- Configuration
src/utils/config.py - Logging
src/utils/logger.py - Reproducible seeding
src/utils/seed.py
Other components
- Researcher
- MLflow tracking store
Connections 27
- Researcher → Validation CLI: runs
README.md - Researcher → Baseline training CLI: runs
README.md - Baseline training CLI → Manifest dataset: loads
scripts/train_baseline.py - Baseline training CLI → CNN architecture: builds
scripts/train_baseline.py - Baseline training CLI → Model training: trains
scripts/train_baseline.py - Baseline training CLI → Model evaluation: evaluates
scripts/train_baseline.py - Baseline training CLI → Experiment tracking: logs runs
scripts/train_baseline.py - Baseline training CLI → Configuration: loads config
scripts/train_baseline.py - Baseline training CLI → Reproducible seeding: sets seed
scripts/train_baseline.py - Model training → Model evaluation: scores epochs
src/models/train.py - Label-noise detection → Manifest dataset: loads folds
src/labels/noise_detection.py - Label-noise detection → CNN architecture: builds model
src/labels/noise_detection.py - Label-noise detection → Model training: trains folds
src/labels/noise_detection.py - Label-noise detection → Model evaluation: predicts probabilities
src/labels/noise_detection.py - Active-learning loop → Query sampling: imports strategies
src/active_learning/loop.py - Condition comparison → MLflow tracking store: fetches runs
src/experiment/comparison.py - Condition comparison → Results plots: summarizes results
- Researcher → Dataset acquisition: acquires data
- Dataset validation → Sample cleaning: validates samples
src/data/cleaning.py - MVTec preprocessing → Image preprocessing: reuses preprocessing
src/data/mvtec_preprocessing.py - Label quality audit → Label-noise detection: informs correction
- Label-noise detection → Label correction: flags for correction
- Targeted augmentation → Class balancing: supports rare classes
- Active-learning loop → Targeted augmentation: supports improved condition
- Researcher → Outlier detection: uses quality tools
- Researcher → Exploratory plots: explores data
- Researcher → Results plots: reviews figures