How arig23498/gemma3-object-detection works
Project type and purpose
• It provides scripts for dataset creation, model training/fine-tuning (with optional tokenizer extension), and inference/prediction.
File structure
• Top-level scripts:
• create_dataset.py – pulls/modifies the Hugging Face dataset into the text-token format needed.
• train.py – runs fine-tuning, with a --include_loc_tokens flag to extend the tokenizer.
• predict.py – loads a fine-tuned model, processes input images, and emits annotated outputs.
• Support files:
• config.py – centralizes hyperparameters, paths, and flags.
• utils.py – shared helper functions (e.g., token↔coordinate conversion, image I/O).
• Artifacts:
• outputs/ – folder for generated images and possibly checkpoints.
• requirements.txt – lists Python dependencies.
• Git‐level files: .gitignore, README.md.
Extract architectural insights from the README
• Two tuning modes: with and without explicitly extending the tokenizer.
• External dependencies: Hugging Face Model Hub (Gemma 3), Hugging Face Datasets, Transformers, PyTorch.
• Workflow steps: dataset prep → training (stage 1: embeddings or full model) → inference.
a. Main components
• Data Acquisition & Preprocessing
– External: Hugging Face Dataset Hub (ariG23498/license-detection-paligemma)
– Script: create_dataset.py
– Output: tokenized text examples (images + <locXXXX> sequences)
• Configuration Layer
– config.py (hyperparameters, model path, flags)
• Training & Fine-Tuning
– Script: train.py
– Sub-components:
– Tokenizer extension module (optional)
– Hugging Face Trainer (backed by PyTorch)
– External: Hugging Face Model Hub (Gemma 3 base model)
– Output: fine-tuned model checkpoints
• Inference / Prediction
– Script: predict.py
– Output: annotated images in outputs/
• Utilities & Shared Libraries
– utils.py (image I/O, token↔coordinate logic)
b. Relationships & data flow
1. create_dataset.py pulls raw images/annotations → produces text-token dataset
2. train.py reads from dataset + config.py → (optionally extends tokenizer) → fine-tunes Gemma 3 → writes checkpoint
4. Both training and inference depend on shared utilities and configuration
c. Architectural patterns & design principles
• Pipeline pattern: clear linear stages (Data Prep → Training → Inference)
• Flag-driven extensibility: --include_loc_tokens toggles tokenizer extension
• Modularity: each script has a single responsibility; shared logic is factored into utils.py and config.py
• External integration: Hugging Face ecosystems for datasets, models, and tokenizers
d. Key technologies
• Python 3.10
• Hugging Face Transformers & Datasets
• PyTorch (training backend)
• Tokenizer library (extended vocabulary)
• Standard Python imaging libs (e.g., PIL or OpenCV via utils.py)