How paulinamoskwa/detect-pikachu works
Project type and purpose
• This is a data-generation and model-training pipeline for object detection, specifically fine-tuning a YOLOv8 model to detect Pikachu using synthetic data.
• It combines 3D asset creation and cleanup, synthetic image/annotation generation in Unity, data conversion, model training (Jupyter notebook), and real-time inference.
File structure
• Top-level folders:
• 3d_modeling: contains raw and processed 3D assets (GLB, FBX).
• code: Python scripts (.py) and a Jupyter notebook (.ipynb) for dataset extraction, format conversion, model training, and inference.
• tutorial.md: a guide linking the pipeline steps.
• No dedicated “frontend/backend” but rather a linear pipeline of stages.
• Key files:
• extract_frame_and_data.py: converts Unity output to YOLOv8 format.
• train_yolov8_model.ipynb: notebook for model training.
• run_realtime_pikachu_detection.py: script for inference using the trained model.
• No build or deployment scripts, but Unity Perception is implied as an external project.
README for additional insights
• The README outlines a 5-stage workflow:
1. 3D scanning (MagiScan)
2. Cleanup in Blender
3. Import into Unity + Perception
4. Generate synthetic images + annotations
5. Convert annotations → YOLOv8 format → train model
6. (Bonus) Real-time inference using the trained model
• Technologies: Unity Perception, Blender, Python, YOLOv8, glTF/FBX.
Elements
• Clear labels on every block (e.g., “3D Cleanup (Blender)”, “Data Conversion Script”).
• Blue blocks for 3D/modeling tools
• Green blocks for data pipelines
• Orange blocks for training/inference
• Gray for external tools