How fritingo/yolo_on_browser works
Main Components
a. Web Frontend:
• A set of HTML pages (e.g., index.html in root, pokedex/index.html, voc/index.html) that serve as the user interface.
• Styling provided by directories such as css and pokedex/src/styles.
• Dynamic behaviors and interactions via JavaScript files found in js, iottalk, pokedex/src/scripts, and voc/js.
• Integration of additional libraries (e.g., within the “iottalk” directory which contains csmapi.js, dai.js, dan.js, ida.js, and jquery.js) that may be used for real-time data handling or device communication.
b. YOLO Model Processing & Inference Backend:
• Python-based modules for YOLOv4 (found under the “yolov4” folder) and YOLOv4_tiny (under the “yolov4_tiny” folder) that handle tasks such as model conversion (to ONNX format) and web inference.
• Model files such as yolov4_tiny.onnx and animal.pth that indicate pre-trained model artifacts are part of the workflow.
• Scripts like to_onnx.py, inf_web.py, and web_yolo.py that orchestrate the conversion and inference pipelines.
c. Demo and Example Applications:
• The “pokedex” demonstration provides a themed example (likely showcasing object detection integrated with a familiar interface such as for Pokémon).
• The “voc” example appears to serve as another demo instance—perhaps tailored to a dataset or a specific object detection use-case.
Relationships and Interactions
• The Web Frontend interacts with the underlying YOLO inference logic either by integrating ONNX runtime/wrappers directly in the browser (if using JavaScript-based inference libraries) or by making asynchronous API calls to server-side Python endpoints (e.g., provided by scripts in yolov4 or yolov4_tiny).
• The JavaScript libraries (including those in iottalk) can be used to communicate real-time data (e.g., image frames from a webcam) to the inference modules.
• Different demo apps (pokedex and voc) serve as alternate frontends that utilize the underlying detection capabilities, demonstrating the model’s performance in different contexts.
Architectural Patterns and Design Principles
• Separation of concerns is evident: the client-side is exclusively handled by HTML/CSS/JS assets, while model processing and conversion are dealt with in Python.
• There is a modular design that supports multiple demonstration applications (each in its own directory) while reusing common assets (such as the iottalk and shared JavaScript libraries).
• The use of ONNX models suggests a design decision aimed at portability, allowing for potential client-side inference using ONNX Runtime on WebAssembly or by interfacing with a backend service.
Technologies, Frameworks, and Libraries
• Frontend: Standard web technologies (HTML, CSS, JavaScript, jQuery).
• Model Inference/Conversion: Python with specialized scripts (e.g., CSPDarknet53, upsampling utilities) and ONNX models.
• Optional IoT/Realtime Communication: The “iottalk” JavaScript components imply integration with external or IoT services.