# haddybhaiya/sem-i-con architecture

> Architecture diagram of the GitHub repository haddybhaiya/sem-i-con, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-03-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/haddybhaiya/sem-i-con
- Repository: https://github.com/haddybhaiya/sem-i-con

## Overview

This repository is an end-to-end computer vision pipeline for semiconductor defect classification from grayscale SEM images, with a clear separation between model development, edge deployment, and evaluation.

At a high level, the architecture has five major subsystems:

1. Data preparation and test-set construction
2. Model training and validation
3. Compression and export for edge deployment
4. ONNX-based edge inference and auto-edge logic
5. Evaluation, benchmarking, and reporting

The project is centered on a two-model strategy:
- a larger teacher model for higher-accuracy learning
- a lightweight student model for edge inference

Main runtime and technology boundaries:
- Training and evaluation are implemented in Python with PyTorch
- Edge deployment uses ONNX Runtime on CPU
- The models operate on grayscale SEM images resized to 224x224
- Quantization and export utilities exist to reduce model size and improve edge efficiency

Repository structure by subsystem

1. Training subsystem
Located under `training/`

Files:
- `training/model.py`
- `training/train.py`
- `training/trainer.py`
- `training/validate.py`

Role:
- Defines the model architectures and training workflow.
- Supports the two core model families described in the README:
  - ConvNeXt-like teacher model
  - MobileNetV3-based student model
- Handles optimization, validation, and likely experiment iteration.

Architecture boundary:
- Inputs are labeled grayscale SEM images from a dataset directory.
- Outputs are trained PyTorch models saved as `.pth` checkpoints.
- This subsystem is the source of the model artifacts later exported to ONNX.

2. Edge export and inference subsystem
Located under `edge/`

Files:
- `edge/export_onnx.py`
- `edge/export_onnx_int8.py`
- `edge/torch_quantize.py`
- `edge/edge_infer.py`
- `edge/auto_edge.py`
- `edge/auto_quantize.py`
- `edge/batch_test.py`
- `edge/check.py`

Role:
- Converts trained PyTorch models into deployable ONNX models.
- Supports quantization paths, including integer quantization.
- Runs single-image inference and batch testing in an edge-compatible way.
- Contains the “auto edge” logic mentioned in the README, which appears to adapt the inference/deployment path depending on the model or constraints.

Key architectural flow:
- PyTorch checkpoint in `models/` or training outputs
  -> export scripts
  -> ONNX model artifacts in `models/`
  -> ONNX Runtime inference in `edge_infer.py` or `auto_edge.py`

Important boundary:
- This subsystem is the deployment boundary between research/training code and runtime inference code.
- It isolates PyTorch-only training logic from ONNX-based execution.

Inference pipeline described in the README:
- grayscale loading with PIL
- resize to 224x224
- normalize
- reshape to `[1, 1, 224, 224]`
- ONNX Runtime inference
- softmax and confidence thresholding
- semantic guard for ambiguous outputs

3. Evaluation subsystem
Located under `evaluation/`

Files:
- `evaluation/create_test_dataset.py`
- `evaluation/create_test_set.py`
- `evaluation/edge_benchmark.py`
- `evaluation/metrics.py`
- `evaluation/phase1_eval.py`
- `evaluation/pytorch_acc.py`
- `evaluation/report.json`

Role:
- Builds or curates test sets.
- Measures classification accuracy and other metrics.
- Benchmarks edge inference performance.
- Produces report artifacts for phase-1 analysis.

Data flow:
- Test images and labels are assembled into evaluation datasets.
- Models are run through evaluation scripts either in PyTorch or ONNX form.
- Metrics are written into JSON report files, such as `report.json`.
- The README also references evaluation outputs like confusion matrices and class-wise accuracy.

Architectural boundary:
- This subsystem sits between training/export and deployment verification.
- It validates both accuracy and edge performance, so it is the key checkpoint before using a model in the field.

4. Learning and self-learning subsystem
Located under `learning/`

Files:
- `learning/self_learn.py`
- `learning/buffer/logs.csv`

Role:
- Implements the repository’s adaptive or iterative learning loop.
- The README mentions a “self learn pipeline with integrated auto edge system,” and this folder likely contains the logic for collecting inference outputs or feedback for future retraining.
- The `buffer/logs.csv` file suggests a log-based memory or replay mechanism.

Architectural role:
- This is the feedback loop boundary.
- It connects deployed inference back to future training or adaptation, rather than being part of the static training pipeline.

5. Artifacts and model storage
Located under `models/`

Files:
- `models/edge_model.onnx.data`
- `models/edge_model_int8.onnx.data`
- `models/mobilenetv3_sem.onnx.data`

Role:
- Stores model data files associated with ONNX exports.
- The repository README also references `.pth` and `.onnx` files that may be external or not checked in.

Architectural meaning:
- This is the artifact boundary between code and deployable model binaries.
- The ONNX files are the deployment target for edge runtime.
- The `.onnx.data` files indicate externalized weights or large model data bundled with ONNX export.

Supporting / experiment files

- `phs1/` and `phs2/` contain phase-specific results, logs, and datasets/prediction scripts.
- `ipynb checks/` and root notebooks are experiment and inspection tools, not production flow.
- `semIcon.pptx.pdf` is documentation rather than runtime architecture.
- `requirements.txt` defines the Python environment.

End-to-end data flow

A realistic system flow in this repository is:

1. Input data acquisition
- Raw grayscale SEM defect images are collected, labeled, and possibly synthesized for balance and robustness.

2. Dataset preparation
- Training and evaluation scripts assemble train/validation/test splits.
- Synthetic augmentation helps compensate for limited real data.

3. Model training
- The teacher model learns higher-capacity features.
- The student model is trained, likely using distillation from the teacher.

4. Model export and compression
- Trained models are exported from PyTorch to ONNX.
- Quantization scripts produce smaller or faster variants, including INT8 models.

5. Edge deployment
- ONNX Runtime executes the model on CPU.
- Preprocessing is grayscale-safe and fixed to the expected input geometry.

6. Post-processing / reliability logic
- Softmax confidence is computed.
- A semantic guard handles ambiguous outputs, reducing unsafe or uncertain classifications.

7. Evaluation and benchmarking
- Accuracy and edge performance are measured.
- Confusion matrices and JSON reports summarize performance.

8. Optional feedback loop
- Inference or evaluation results can be logged into the learning subsystem for future iteration.

What to emphasize in an architecture diagram

If you are drawing this repo, the most important boxes and boundaries are:

- Data source / synthetic data generation
- Training pipeline with teacher and student models
- Export/quantization pipeline
- ONNX Runtime edge inference service
- Evaluation and benchmarking pipeline
- Self-learning feedback loop
- Model artifact store

The central design pattern is a train-export-deploy loop:
- PyTorch for model development
- ONNX for deployment
- Evaluation as a gate between them
- Optional self-learning to feed operational data back into model improvement

In short, this is not just a model-training repo. It is an edge-AI product pipeline for semiconductor defect detection, with explicit support for model compression, deployment, evaluation, and iterative improvement.

Main entry points: [`training/train.py`](https://github.com/haddybhaiya/sem-i-con/blob/HEAD/training/train.py) (Train loop), [`edge/edge_infer.py`](https://github.com/haddybhaiya/sem-i-con/blob/HEAD/edge/edge_infer.py) (Edge infer), [`models`](https://github.com/haddybhaiya/sem-i-con/tree/HEAD/models) (Model store), [`evaluation/create_test_dataset.py`](https://github.com/haddybhaiya/sem-i-con/blob/HEAD/evaluation/create_test_dataset.py) (Test set build)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph "Data prep"
  raw_sem["SEM images<br/>data source"]
  test_build["Test set build<br/>dataset prep"]
end

subgraph "Training"
  train_model["Model defs<br/>PyTorch models<br/>[model.py]"]
  train_loop["Train loop<br/>training entrypoint<br/>[train.py]"]
  trainer_core["Trainer<br/>training engine<br/>[trainer.py]"]
  validate_run["Validate<br/>validation<br/>[validate.py]"]
  teacher_model(("Teacher<br/>high-capacity model"))
  student_model(("Student<br/>edge model"))
end

subgraph "Edge deployment"
  export_onnx["Export ONNX<br/>[export_onnx.py]"]
  export_int8["INT8 export<br/>quantized export"]
  quantize_torch["Torch quantize<br/>quantization<br/>[torch_quantize.py]"]
  edge_infer["Edge infer<br/>ONNX Runtime<br/>[edge_infer.py]"]
  auto_edge["Auto edge<br/>deployment logic<br/>[auto_edge.py]"]
  auto_quantize["Auto quantize<br/>compression logic<br/>[auto_quantize.py]"]
  batch_test["Batch test<br/>edge testing<br/>[batch_test.py]"]
  edge_check["Check<br/>model sanity<br/>[check.py]"]
end

subgraph "Evaluation"
  metrics["Metrics<br/>evaluation core<br/>[metrics.py]"]
  phase_eval["Phase eval<br/>evaluation run<br/>[phase1_eval.py]"]
  edge_bench["Edge bench<br/>benchmarking<br/>[edge_benchmark.py]"]
  pytorch_acc["PyTorch acc<br/>accuracy eval<br/>[pytorch_acc.py]"]
end

subgraph "Feedback loop"
  self_learn["Self learn<br/>feedback loop<br/>[self_learn.py]"]
  logs["Logs<br/>buffer<br/>[logs.csv]"]
end

edge_artifacts[("Model store<br/>artifact store")]

raw_sem -->|"split"| test_build
raw_sem -->|"train on"| train_loop
train_model -->|"define"| train_loop
train_loop -->|"orchestrate"| trainer_core
trainer_core -->|"validate"| validate_run
teacher_model -->|"distill"| student_model
train_loop -->|"fit"| teacher_model
train_loop -->|"fit"| student_model
teacher_model -->|"checkpoint"| export_onnx
student_model -->|"checkpoint"| export_onnx
quantize_torch -->|"prepare"| export_int8
export_onnx -->|"write"| edge_artifacts
export_int8 -->|"write"| edge_artifacts
edge_artifacts -->|"load"| edge_infer
auto_quantize -->|"choose"| quantize_torch
auto_edge -->|"route"| edge_infer
edge_infer -->|"batch"| batch_test
edge_infer -->|"score"| metrics
batch_test -->|"benchmark"| edge_bench
pytorch_acc -->|"report"| phase_eval
edge_bench -->|"report"| phase_eval
test_build -->|"evaluate"| pytorch_acc
test_build -->|"evaluate"| batch_test
self_learn -->|"record"| logs
logs -->|"feedback"| train_loop
edge_check -->|"inspect"| edge_artifacts

click test_build "https://github.com/haddybhaiya/sem-i-con/blob/main/evaluation/create_test_dataset.py"
click train_model "https://github.com/haddybhaiya/sem-i-con/blob/main/training/model.py"
click train_loop "https://github.com/haddybhaiya/sem-i-con/blob/main/training/train.py"
click trainer_core "https://github.com/haddybhaiya/sem-i-con/blob/main/training/trainer.py"
click validate_run "https://github.com/haddybhaiya/sem-i-con/blob/main/training/validate.py"
click export_onnx "https://github.com/haddybhaiya/sem-i-con/blob/main/edge/export_onnx.py"
click export_int8 "https://github.com/haddybhaiya/sem-i-con/blob/main/edge/export_onnx_int8.py"
click quantize_torch "https://github.com/haddybhaiya/sem-i-con/blob/main/edge/torch_quantize.py"
click edge_infer "https://github.com/haddybhaiya/sem-i-con/blob/main/edge/edge_infer.py"
click auto_edge "https://github.com/haddybhaiya/sem-i-con/blob/main/edge/auto_edge.py"
click auto_quantize "https://github.com/haddybhaiya/sem-i-con/blob/main/edge/auto_quantize.py"
click batch_test "https://github.com/haddybhaiya/sem-i-con/blob/main/edge/batch_test.py"
click edge_check "https://github.com/haddybhaiya/sem-i-con/blob/main/edge/check.py"
click metrics "https://github.com/haddybhaiya/sem-i-con/blob/main/evaluation/metrics.py"
click phase_eval "https://github.com/haddybhaiya/sem-i-con/blob/main/evaluation/phase1_eval.py"
click edge_bench "https://github.com/haddybhaiya/sem-i-con/blob/main/evaluation/edge_benchmark.py"
click pytorch_acc "https://github.com/haddybhaiya/sem-i-con/blob/main/evaluation/pytorch_acc.py"
click self_learn "https://github.com/haddybhaiya/sem-i-con/blob/main/learning/self_learn.py"
click logs "https://github.com/haddybhaiya/sem-i-con/blob/main/learning/buffer/logs.csv"
click edge_artifacts "https://github.com/haddybhaiya/sem-i-con/tree/main/models"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class raw_sem,test_build toneBlue
class train_model,train_loop,trainer_core,validate_run,teacher_model,student_model toneAmber
class export_onnx,export_int8,quantize_torch,edge_infer,auto_edge,auto_quantize,batch_test,edge_check toneMint
class metrics,phase_eval,edge_bench,pytorch_acc toneRose
class self_learn,logs toneIndigo
class edge_artifacts toneNeutral
```

## Components

### Data prep

Dataset construction and split preparation for training and evaluation.

- **SEM images**: data source · Grayscale semiconductor defect images used as the core input domain.
- [Test set build](https://github.com/haddybhaiya/sem-i-con/blob/HEAD/evaluation/create_test_dataset.py): dataset prep · Creates curated test sets and evaluation splits from image data. · `evaluation/create_test_dataset.py`

### Training

PyTorch model development, optimization, and validation.

- [Model defs](https://github.com/haddybhaiya/sem-i-con/blob/HEAD/training/model.py): PyTorch models · Defines the teacher and student architectures used in training. · `training/model.py`
- [Train loop](https://github.com/haddybhaiya/sem-i-con/blob/HEAD/training/train.py): training entrypoint · Coordinates optimization and experiment runs. · `training/train.py`
- [Trainer](https://github.com/haddybhaiya/sem-i-con/blob/HEAD/training/trainer.py): training engine · Encapsulates the training workflow, iteration, and checkpoint handling. · `training/trainer.py`
- [Validate](https://github.com/haddybhaiya/sem-i-con/blob/HEAD/training/validate.py): validation · Runs model validation and reports performance during training. · `training/validate.py`
- **Teacher**: high-capacity model · ConvNeXt-like model used for higher-accuracy learning.
- **Student**: edge model · MobileNetV3-based lightweight model intended for deployment.

### Edge deployment

ONNX export, quantization, and CPU inference flow.

- [Export ONNX](https://github.com/haddybhaiya/sem-i-con/blob/HEAD/edge/export_onnx.py): export · Converts trained PyTorch checkpoints into ONNX artifacts. · `edge/export_onnx.py`
- [INT8 export](https://github.com/haddybhaiya/sem-i-con/blob/HEAD/edge/export_onnx_int8.py): quantized export · Produces integer-quantized ONNX variants for smaller edge models. · `edge/export_onnx_int8.py`
- [Torch quantize](https://github.com/haddybhaiya/sem-i-con/blob/HEAD/edge/torch_quantize.py): quantization · Applies PyTorch-side quantization utilities before export. · `edge/torch_quantize.py`
- [Edge infer](https://github.com/haddybhaiya/sem-i-con/blob/HEAD/edge/edge_infer.py): ONNX Runtime · Runs single-image ONNX inference on CPU with preprocessing and softmax post-processing. · `edge/edge_infer.py`
- [Auto edge](https://github.com/haddybhaiya/sem-i-con/blob/HEAD/edge/auto_edge.py): deployment logic · Selects or adapts the edge inference path based on model or constraints. · `edge/auto_edge.py`
- [Auto quantize](https://github.com/haddybhaiya/sem-i-con/blob/HEAD/edge/auto_quantize.py): compression logic · Automates choosing and applying quantization settings. · `edge/auto_quantize.py`
- [Batch test](https://github.com/haddybhaiya/sem-i-con/blob/HEAD/edge/batch_test.py): edge testing · Runs bulk edge inference checks over a dataset. · `edge/batch_test.py`
- [Check](https://github.com/haddybhaiya/sem-i-con/blob/HEAD/edge/check.py): model sanity · Performs consistency or readiness checks on exported edge models. · `edge/check.py`

### Evaluation

Accuracy, benchmarking, and reporting for phase analysis.

- [Metrics](https://github.com/haddybhaiya/sem-i-con/blob/HEAD/evaluation/metrics.py): evaluation core · Computes classification metrics and summary statistics. · `evaluation/metrics.py`
- [Phase eval](https://github.com/haddybhaiya/sem-i-con/blob/HEAD/evaluation/phase1_eval.py): evaluation run · Produces phase-level evaluation outputs and analysis artifacts. · `evaluation/phase1_eval.py`
- [Edge bench](https://github.com/haddybhaiya/sem-i-con/blob/HEAD/evaluation/edge_benchmark.py): benchmarking · Measures edge runtime performance and throughput. · `evaluation/edge_benchmark.py`
- [PyTorch acc](https://github.com/haddybhaiya/sem-i-con/blob/HEAD/evaluation/pytorch_acc.py): accuracy eval · Evaluates model accuracy in the native PyTorch runtime. · `evaluation/pytorch_acc.py`

### Feedback loop

Logs and self-learning path that can feed future retraining.

- [Self learn](https://github.com/haddybhaiya/sem-i-con/blob/HEAD/learning/self_learn.py): feedback loop · Collects operational feedback for iterative adaptation or retraining. · `learning/self_learn.py`
- [Logs](https://github.com/haddybhaiya/sem-i-con/blob/HEAD/learning/buffer/logs.csv): buffer · CSV log buffer used to retain learning or inference history. · `learning/buffer/logs.csv`

### Other components

- [Model store](https://github.com/haddybhaiya/sem-i-con/tree/HEAD/models): artifact store · Holds deployable ONNX weight data for edge models. · `models`

## Connections

- SEM images → Test set build: split · Raw SEM images are organized into curated test and evaluation sets.
- SEM images → Train loop: train on · Training consumes labeled grayscale SEM images.
- Model defs → Train loop: define · Model definitions are instantiated by the training entrypoint.
- Train loop → Trainer: orchestrate · The training entrypoint delegates execution to the trainer.
- Trainer → Validate: validate · Training periodically invokes validation.
- Teacher → Student: distill · The teacher guides the smaller student model.
- Train loop → Teacher: fit · Teacher model training produces high-capacity checkpoints.
- Train loop → Student: fit · Student model training produces compact checkpoints.
- Teacher → Export ONNX: checkpoint · Trained checkpoints are exported for deployment.
- Student → Export ONNX: checkpoint · Student checkpoints are exported for edge use.
- Torch quantize → INT8 export: prepare · Torch quantization feeds the INT8 export path.
- Export ONNX → Model store: write · Exported ONNX artifacts are stored in the model directory.
- INT8 export → Model store: write · Quantized ONNX artifacts are stored in the model directory.
- Model store → Edge infer: load · Edge inference loads the deployable ONNX model artifacts.
- Auto quantize → Torch quantize: choose · Automated quantization selects a compression strategy.
- Auto edge → Edge infer: route · Auto-edge logic routes requests to the runtime path.
- Edge infer → Batch test: batch · The inference path can be exercised in batch mode.
- Edge infer → Metrics: score · Inference outputs are summarized into metrics.
- Batch test → Edge bench: benchmark · Batch tests feed edge benchmarking.
- PyTorch acc → Phase eval: report · Native-runtime accuracy contributes to phase evaluation outputs.
- Edge bench → Phase eval: report · Edge performance results contribute to phase evaluation outputs.
- Test set build → PyTorch acc: evaluate · Curated test sets are used for PyTorch accuracy runs.
- Test set build → Batch test: evaluate · Curated test sets are used for batch edge testing.
- Self learn → Logs: record · Operational feedback is persisted to the learning buffer.
- Logs → Train loop: feedback · Logged feedback can inform future training iterations.
- Check → Model store: inspect · Checks verify exported model artifacts before deployment.
