How ocean-data-factory-sweden/kso works
Project Type & Purpose
• This is an open-source, citizen science and machine learning system for analyzing subsea movies. It provides an end-to-end pipeline that moves and processes underwater footage (with associated metadata), integrates with the Zooniverse citizen science platform, and provides facilities for training and evaluating ML models (using YOLO variants) via modular Jupyter Notebooks.
• The system is used by marine scientists and citizen scientists to upload, annotate, process, analyze, and eventually publish analysis of underwater footage.
File Structure – Key Observations
• Top-level files and directories (e.g., README.md, Dockerfile, conda_env.yaml) indicate that the project supports containerized and conda-dependent environments.
• The “notebooks” directory is organized into four main functional groups: setup, classify, analyse, and publish. Each group represents a stage in the workflow.
• The “kso_utils” package contains key utility modules for operations such as database initialization (“db_starter”), frame and video processing, interfacing with external platforms (e.g., Zooniverse, Zenodo), and specialized tools for machine learning (e.g., YOLO, tracking, etc.).
• The “src” directory contains executable scripts that appear to be used for running real-time processing tasks such as detection, tracking, and data preparation.
• There are also examples and tests that illustrate usage and validate functionality.
a. Main Components
• User Interface & Workflow: Represented by the modular Jupyter Notebooks. This interface supports different tasks:
– Setup (e.g., metadata checking)
– Classification (upload footage, process classifications via Zooniverse)
– Analysis (training and evaluating models)
– Publishing (publishing models and exporting observations)
• Processing & Utilities Module (kso_utils): Contains libraries for data manipulation, video processing, interfacing with databases (via CSV templates in “db_starter”), and integrating with ML workflows.
• Core Processing Scripts (src): Scripts for detection, tracking, dataset preparation, and model execution. These perform the “heavy lifting” in video and image analysis using models like YOLO and other computer vision/ML techniques.
• External Integrations:
– Zooniverse: For citizen science annotations and classifications.
– Cloudina: A hosted JupyterHub-based environment supporting storage, notebooks (album), and MLFlow (vendor).
– Possibly other external data repositories (e.g., GBIF via “Publish_observations”).
• Supporting Infrastructure:
– Docker & Conda for environment reproducibility.
– GitHub workflows for CI/CD (linting, build/test pipelines).
– CSV “database” files and schema in the db_starter folder that initialize or seed projects.
b. Relationships & Data Flow
• Data Input: Underwater footage (videos, images) and associated metadata in CSV files flow into the system.
– The “db_starter” component sets up initial project details.
• Notebook Frontend: Users execute notebooks for data preparation, classification, analysis, and publishing.
– These tasks trigger utilities in kso_utils and processing scripts in src.
• ML & Processing Pipeline: Functions within kso_utils (e.g., for video reading, YOLO integrations, frame tracking) are used by scripts to process the media.
• External Services: Processed data is sent to Zooniverse (citizen science platform) for annotation and may also be published via Cloudina endpoints.
• CI/CD & Environment: Dockerfile, conda_env.yaml, and GitHub workflows support building, testing, and deploying the application.
Key Architectural Patterns & Design Principles
• Extensibility: The system is structured to allow researchers to plug in custom notebooks or extend utilities (e.g., support for new detectors).
• Integration with External Services: Clear concerns for data ingestion and output when interacting with external platforms (Zooniverse, Cloudina).
• Environment Agnosticism: Use of Docker and Conda indicates cross-platform deployments.