How arezkisaba/french-id-detection-training works
Main components
• Data‑sources
• Document templates (src/dataset/templates)
• OpenImages backgrounds (src/openimages via backgrounds‑download scripts)
• Dataset generation pipeline
• Python scripts:
– backgrounds-download(.py) & backgrounds-download-openimages.py
– dataset-generate.py (combines templates + backgrounds + augmentations)
• Model training pipeline
• dataset-train.py (Ultralytics YOLO11 in TensorFlow, albumentations, tensorboard)
• args.yaml for hyperparameters (src/results/args.yaml)
• Model artifacts storage
• Local filesystem (src/results/weights: .pt, web_model_original, web_model_quantized)
• Model conversion & testing
• model-convert.py (convert .pt → TF SavedModel / web model)
• model-test.py (validation/inference on samples)
• Auxiliary
• utils.py (shared routines)
• default.ipynb (exploration / visualization)
• AWS integration via boto3 (optional for uploading/downloading)
Relationships & data‑flows
b. “Dataset Generator” → output folder (src/dataset/… or results/…) → input into “Model Trainer”
c. “Model Trainer” writes to “Results/Weights” and “Results/args.yaml”
d. “Model Converter” consumes weights → produces TF/web artifacts
e. “Model Tester” consumes converted model + test samples (src/tests/samples)
f. (Optional) upload/download from S3 via boto3 at each stage
Architectural patterns
• Pipeline pattern: clear stages (ETL → training → post‑processing → evaluation)
• Modularity: each stage is a standalone Python script, enabling ad‑hoc execution and replacement
• Config‑driven: args.yaml, metadata.yaml in web models
• Data versioning: results folder holds multiple weight variants (best.pt, last.pt, quantized, original, etc.)
Technologies/frameworks to components
• Dataset Generator: Python3.11, albumentations
• Model Trainer: Ultralytics YOLO (TF backend), tensorboard
• Conversion: TensorFlow SavedModel, Web‑model JSON/BIN
• Testing: boto3 (if retrieving data/models), pure Python inference
Final checklist
• Clear component labels (script name + responsibility)
• File system storage illustrated for inputs/outputs