# rohannx29/dataforge-dcai architecture

> Architecture diagram of the GitHub repository rohannx29/dataforge-dcai, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rohannx29/dataforge-dcai
- Repository: https://github.com/rohannx29/dataforge-dcai

## Overview

This research pipeline tests whether data-quality interventions improve manufacturing-defect classification under a fixed CNN. Its intended workflow runs dataset preparation and label-quality interventions, trains and evaluates comparable conditions, then compares experiment results. The README documents acquisition, MVTec processing, targeted augmentation, and active learning as product capabilities. Evidence is limited to the README and indexed or excerpted files; some documented stages have no verified implementation links, and the active-learning loop currently uses random selection rather than its imported sampling strategies.

Main entry points: [`scripts/train_baseline.py`](https://github.com/rohannx29/dataforge-dcai/blob/main/scripts/train_baseline.py) (Baseline training CLI), [`src/labels/noise_detection.py`](https://github.com/rohannx29/dataforge-dcai/blob/main/src/labels/noise_detection.py) (Label-noise detection), [`src/models/train.py`](https://github.com/rohannx29/dataforge-dcai/blob/main/src/models/train.py) (Model training), [`src/models/evaluate.py`](https://github.com/rohannx29/dataforge-dcai/blob/main/src/models/evaluate.py) (Model evaluation)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Data preparation"]
  node_validate_cli["Validation CLI<br/>[run_validation.py]"]
  node_acquisition["Dataset acquisition<br/>[acquisition.py]"]
  node_validation["Dataset validation<br/>[validation.py]"]
  node_cleaning["Sample cleaning<br/>[cleaning.py]"]
  node_preprocessing["Image preprocessing<br/>[preprocessing.py]"]
  node_mvtec["MVTec preprocessing"]
  node_dataset["Manifest dataset<br/>[dataset.py]"]
end

subgraph group_quality["Label and data quality"]
  node_audit["Label quality audit<br/>[quality_audit.py]"]
  node_noise_detection["Label-noise detection<br/>[noise_detection.py]"]
  node_noise_correction["Label correction"]
  node_augmentation["Targeted augmentation<br/>[augmentation.py]"]
  node_class_balance["Class balancing<br/>[class_balance.py]"]
  node_outliers["Outlier detection"]
end

subgraph group_learning["Model and learning"]
  node_baseline_cli["Baseline training CLI<br/>[train_baseline.py]"]
  node_al_loop["Active-learning loop<br/>[loop.py]"]
  node_sampling["Query sampling<br/>[sampling.py]"]
  node_architecture["CNN architecture<br/>[architectures.py]"]
  node_training["Model training<br/>[train.py]"]
  node_evaluation["Model evaluation<br/>[evaluate.py]"]
end

subgraph group_research["Experiment results"]
  node_tracker["Experiment tracking<br/>[tracker.py]"]
  node_comparison["Condition comparison<br/>[comparison.py]"]
  node_eda["Exploratory plots<br/>[eda_plots.py]"]
  node_result_plots["Results plots<br/>[results_plots.py]"]
end

subgraph group_support["Shared utilities"]
  node_config["Configuration<br/>[config.py]"]
  node_logger["Logging<br/>[logger.py]"]
  node_seed["Reproducible seeding<br/>[seed.py]"]
end

node_researcher(("Researcher"))
node_mlflow[("MLflow tracking store")]

node_researcher -->|"runs"| node_validate_cli
node_researcher -->|"runs"| node_baseline_cli
node_baseline_cli -->|"loads"| node_dataset
node_baseline_cli -->|"builds"| node_architecture
node_baseline_cli -->|"trains"| node_training
node_baseline_cli -->|"evaluates"| node_evaluation
node_baseline_cli -->|"logs runs"| node_tracker
node_baseline_cli -->|"loads config"| node_config
node_baseline_cli -->|"sets seed"| node_seed
node_training -->|"scores epochs"| node_evaluation
node_noise_detection -->|"loads folds"| node_dataset
node_noise_detection -->|"builds model"| node_architecture
node_noise_detection -->|"trains folds"| node_training
node_noise_detection -->|"predicts probabilities"| node_evaluation
node_al_loop -->|"imports strategies"| node_sampling
node_comparison -->|"fetches runs"| node_mlflow
node_comparison -.->|"summarizes results"| node_result_plots
node_researcher -.->|"acquires data"| node_acquisition
node_validation -->|"validates samples"| node_cleaning
node_mvtec -->|"reuses preprocessing"| node_preprocessing
node_audit -.->|"informs correction"| node_noise_detection
node_noise_detection -.->|"flags for correction"| node_noise_correction
node_augmentation -.->|"supports rare classes"| node_class_balance
node_al_loop -.->|"supports improved condition"| node_augmentation
node_researcher -.->|"uses quality tools"| node_outliers
node_researcher -.->|"explores data"| node_eda
node_researcher -.->|"reviews figures"| node_result_plots

click node_validate_cli "https://github.com/rohannx29/dataforge-dcai/blob/main/scripts/run_validation.py"
click node_baseline_cli "https://github.com/rohannx29/dataforge-dcai/blob/main/scripts/train_baseline.py"
click node_acquisition "https://github.com/rohannx29/dataforge-dcai/blob/main/src/data/acquisition.py"
click node_validation "https://github.com/rohannx29/dataforge-dcai/blob/main/src/data/validation.py"
click node_cleaning "https://github.com/rohannx29/dataforge-dcai/blob/main/src/data/cleaning.py"
click node_preprocessing "https://github.com/rohannx29/dataforge-dcai/blob/main/src/data/preprocessing.py"
click node_mvtec "https://github.com/rohannx29/dataforge-dcai/blob/main/src/data/mvtec_preprocessing.py"
click node_dataset "https://github.com/rohannx29/dataforge-dcai/blob/main/src/data/dataset.py"
click node_audit "https://github.com/rohannx29/dataforge-dcai/blob/main/src/labels/quality_audit.py"
click node_noise_detection "https://github.com/rohannx29/dataforge-dcai/blob/main/src/labels/noise_detection.py"
click node_noise_correction "https://github.com/rohannx29/dataforge-dcai/blob/main/src/labels/noise_correction.py"
click node_augmentation "https://github.com/rohannx29/dataforge-dcai/blob/main/src/dcai/augmentation.py"
click node_class_balance "https://github.com/rohannx29/dataforge-dcai/blob/main/src/dcai/class_balance.py"
click node_outliers "https://github.com/rohannx29/dataforge-dcai/blob/main/src/dcai/outlier_detection.py"
click node_al_loop "https://github.com/rohannx29/dataforge-dcai/blob/main/src/active_learning/loop.py"
click node_sampling "https://github.com/rohannx29/dataforge-dcai/blob/main/src/active_learning/sampling.py"
click node_architecture "https://github.com/rohannx29/dataforge-dcai/blob/main/src/models/architectures.py"
click node_training "https://github.com/rohannx29/dataforge-dcai/blob/main/src/models/train.py"
click node_evaluation "https://github.com/rohannx29/dataforge-dcai/blob/main/src/models/evaluate.py"
click node_tracker "https://github.com/rohannx29/dataforge-dcai/blob/main/src/experiment/tracker.py"
click node_comparison "https://github.com/rohannx29/dataforge-dcai/blob/main/src/experiment/comparison.py"
click node_eda "https://github.com/rohannx29/dataforge-dcai/blob/main/src/visualization/eda_plots.py"
click node_result_plots "https://github.com/rohannx29/dataforge-dcai/blob/main/src/visualization/results_plots.py"
click node_config "https://github.com/rohannx29/dataforge-dcai/blob/main/src/utils/config.py"
click node_logger "https://github.com/rohannx29/dataforge-dcai/blob/main/src/utils/logger.py"
click node_seed "https://github.com/rohannx29/dataforge-dcai/blob/main/src/utils/seed.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_validate_cli,node_acquisition,node_validation,node_cleaning,node_preprocessing,node_mvtec,node_dataset toneBlue
class node_audit,node_noise_detection,node_noise_correction,node_augmentation,node_class_balance,node_outliers,node_mlflow toneAmber
class node_baseline_cli,node_al_loop,node_sampling,node_architecture,node_training,node_evaluation toneMint
class node_tracker,node_comparison,node_eda,node_result_plots toneRose
class node_config,node_logger,node_seed,node_researcher toneIndigo
```

## Components

### Data preparation

- [Validation CLI](https://github.com/rohannx29/dataforge-dcai/blob/main/scripts/run_validation.py): `scripts/run_validation.py`
- [Dataset acquisition](https://github.com/rohannx29/dataforge-dcai/blob/main/src/data/acquisition.py): `src/data/acquisition.py`
- [Dataset validation](https://github.com/rohannx29/dataforge-dcai/blob/main/src/data/validation.py): `src/data/validation.py`
- [Sample cleaning](https://github.com/rohannx29/dataforge-dcai/blob/main/src/data/cleaning.py): `src/data/cleaning.py`
- [Image preprocessing](https://github.com/rohannx29/dataforge-dcai/blob/main/src/data/preprocessing.py): `src/data/preprocessing.py`
- [MVTec preprocessing](https://github.com/rohannx29/dataforge-dcai/blob/main/src/data/mvtec_preprocessing.py): `src/data/mvtec_preprocessing.py`
- [Manifest dataset](https://github.com/rohannx29/dataforge-dcai/blob/main/src/data/dataset.py): `src/data/dataset.py`

### Label and data quality

- [Label quality audit](https://github.com/rohannx29/dataforge-dcai/blob/main/src/labels/quality_audit.py): `src/labels/quality_audit.py`
- [Label-noise detection](https://github.com/rohannx29/dataforge-dcai/blob/main/src/labels/noise_detection.py): `src/labels/noise_detection.py`
- [Label correction](https://github.com/rohannx29/dataforge-dcai/blob/main/src/labels/noise_correction.py): `src/labels/noise_correction.py`
- [Targeted augmentation](https://github.com/rohannx29/dataforge-dcai/blob/main/src/dcai/augmentation.py): `src/dcai/augmentation.py`
- [Class balancing](https://github.com/rohannx29/dataforge-dcai/blob/main/src/dcai/class_balance.py): `src/dcai/class_balance.py`
- [Outlier detection](https://github.com/rohannx29/dataforge-dcai/blob/main/src/dcai/outlier_detection.py): `src/dcai/outlier_detection.py`

### Model and learning

- [Baseline training CLI](https://github.com/rohannx29/dataforge-dcai/blob/main/scripts/train_baseline.py): `scripts/train_baseline.py`
- [Active-learning loop](https://github.com/rohannx29/dataforge-dcai/blob/main/src/active_learning/loop.py): `src/active_learning/loop.py`
- [Query sampling](https://github.com/rohannx29/dataforge-dcai/blob/main/src/active_learning/sampling.py): `src/active_learning/sampling.py`
- [CNN architecture](https://github.com/rohannx29/dataforge-dcai/blob/main/src/models/architectures.py): `src/models/architectures.py`
- [Model training](https://github.com/rohannx29/dataforge-dcai/blob/main/src/models/train.py): `src/models/train.py`
- [Model evaluation](https://github.com/rohannx29/dataforge-dcai/blob/main/src/models/evaluate.py): `src/models/evaluate.py`

### Experiment results

- [Experiment tracking](https://github.com/rohannx29/dataforge-dcai/blob/main/src/experiment/tracker.py): `src/experiment/tracker.py`
- [Condition comparison](https://github.com/rohannx29/dataforge-dcai/blob/main/src/experiment/comparison.py): `src/experiment/comparison.py`
- [Exploratory plots](https://github.com/rohannx29/dataforge-dcai/blob/main/src/visualization/eda_plots.py): `src/visualization/eda_plots.py`
- [Results plots](https://github.com/rohannx29/dataforge-dcai/blob/main/src/visualization/results_plots.py): `src/visualization/results_plots.py`

### Shared utilities

- [Configuration](https://github.com/rohannx29/dataforge-dcai/blob/main/src/utils/config.py): `src/utils/config.py`
- [Logging](https://github.com/rohannx29/dataforge-dcai/blob/main/src/utils/logger.py): `src/utils/logger.py`
- [Reproducible seeding](https://github.com/rohannx29/dataforge-dcai/blob/main/src/utils/seed.py): `src/utils/seed.py`

### Other components

- **Researcher**
- **MLflow tracking store**

## Connections

- Researcher → Validation CLI: runs (evidence: `README.md`)
- Researcher → Baseline training CLI: runs (evidence: `README.md`)
- Baseline training CLI → Manifest dataset: loads (evidence: `scripts/train_baseline.py`)
- Baseline training CLI → CNN architecture: builds (evidence: `scripts/train_baseline.py`)
- Baseline training CLI → Model training: trains (evidence: `scripts/train_baseline.py`)
- Baseline training CLI → Model evaluation: evaluates (evidence: `scripts/train_baseline.py`)
- Baseline training CLI → Experiment tracking: logs runs (evidence: `scripts/train_baseline.py`)
- Baseline training CLI → Configuration: loads config (evidence: `scripts/train_baseline.py`)
- Baseline training CLI → Reproducible seeding: sets seed (evidence: `scripts/train_baseline.py`)
- Model training → Model evaluation: scores epochs (evidence: `src/models/train.py`)
- Label-noise detection → Manifest dataset: loads folds (evidence: `src/labels/noise_detection.py`)
- Label-noise detection → CNN architecture: builds model (evidence: `src/labels/noise_detection.py`)
- Label-noise detection → Model training: trains folds (evidence: `src/labels/noise_detection.py`)
- Label-noise detection → Model evaluation: predicts probabilities (evidence: `src/labels/noise_detection.py`)
- Active-learning loop → Query sampling: imports strategies (evidence: `src/active_learning/loop.py`)
- Condition comparison → MLflow tracking store: fetches runs (evidence: `src/experiment/comparison.py`)
- Condition comparison → Results plots: summarizes results
- Researcher → Dataset acquisition: acquires data
- Dataset validation → Sample cleaning: validates samples (evidence: `src/data/cleaning.py`)
- MVTec preprocessing → Image preprocessing: reuses preprocessing (evidence: `src/data/mvtec_preprocessing.py`)
- Label quality audit → Label-noise detection: informs correction
- Label-noise detection → Label correction: flags for correction
- Targeted augmentation → Class balancing: supports rare classes
- Active-learning loop → Targeted augmentation: supports improved condition
- Researcher → Outlier detection: uses quality tools
- Researcher → Exploratory plots: explores data
- Researcher → Results plots: reviews figures
