# jubjones/spoton_ml architecture

> Architecture diagram of the GitHub repository jubjones/spoton_ml, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jubjones/spoton_ml
- Repository: https://github.com/jubjones/spoton_ml

## Overview

##### Project type and purpose
- This is an open-source, Python-based machine-learning experimentation framework (“spoton_ml”) that provides configurable pipelines for exploratory data analysis (EDA), object detection (using Faster R-CNN), and tracking/re-identification.
- You drive experiments via small CLI scripts (e.g. run_eda.py, run_training_fasterrcnn.py), backed by a YAML-driven configuration system and integrated with PyTorch and MLflow.

##### File structure
- Top-level directories:
- configs/ – all YAML configs for different experiments
- scripts/ – helper scripts (e.g. exporting weights)
- src/
– core/runner.py – orchestrates pipeline execution
– data/ – data loaders & dataset definitions
– detection/ – detection strategy implementations
– evaluation/ – metric calculations for detection & tracking
– pipelines/ – EDA, detection, and tracking/reid pipeline definitions
– training/ – PyTorch training engine & runner
– utils/ – config loading, device selection, logging, MLflow hooks, reproducibility helpers
- tests/ – pytest suite for each module
- Config files (comparison_run_config.yaml, fasterrcnn_training_config.yaml, etc.) control which pipeline, parameters, and resources get used.

##### README
- Bare-bones README (“# spoton_ml”) but naming and directory layout imply a modular, pipeline-based ML experimentation tool.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% CLI & Config Layer
    subgraph "CLI Layer" 
        direction TB
        EDA_Script["run_eda.py"]:::cli_config
        Train_Script["run_training_fasterrcnn.py"]:::cli_config
        Exp_Script["run_experiment.py"]:::cli_config
        Comp_Script["run_comparison.py"]:::cli_config
        Track_Script["run_tracking_reid_comparison.py"]:::cli_config
        Export_Script["export_reid_weights.py"]:::cli_config
    end

    subgraph "Configuration" 
        direction TB
        ConfigLoader["Config Loader"]:::cli_config
        ConfigFiles["configs/"]:::cli_config
    end

    %% Core & Pipelines Layer
    CoreRunner["Core Runner"]:::core_pipeline

    subgraph "Pipelines" 
        direction TB
        EDA_Pipeline["EDA Pipeline"]:::core_pipeline
        Detection_Pipeline["Detection Pipeline"]:::core_pipeline
        Tracking_Pipeline["Tracking/ReID Pipeline"]:::core_pipeline
    end

    %% Data, Training, Detection, Evaluation
    subgraph "Data Module" 
        direction TB
        EDA_Loader["EDA Loader"]:::data_ops
        Generic_Loader["Generic Loader"]:::data_ops
        TrainingDataset["TrainingDataset"]:::data_ops
    end

    Training_Engine["PyTorch Engine & Runner"]:::data_ops

    Detection_Strategies["Detection Strategies"]:::data_ops

    subgraph "Evaluation Module"
        direction TB
        Det_Metrics["Detection Metrics"]:::data_ops
        Track_Metrics["Tracking Metrics"]:::data_ops
    end

    %% Utilities
    subgraph "Utilities" 
        direction TB
        Device_Utils["Device Utils"]:::utils
        Logging_Utils["Logging Utils"]:::utils
        MLflow_Utils["MLflow Utils"]:::utils
        Repro_Utils["Reproducibility"]:::utils
        Torch_Utils["Torch Utils"]:::utils
    end

    %% External Services
    MLflow_Server((MLflow Server)):::utils
    Data_Storage[(Filesystem / Data Storage)]:::utils

    %% Connections
    EDA_Script -->|"invoke"| CoreRunner
    Train_Script -->|"invoke"| CoreRunner
    Exp_Script -->|"invoke"| CoreRunner
    Comp_Script -->|"invoke"| CoreRunner
    Track_Script -->|"invoke"| CoreRunner
    Export_Script -->|"export_weights"| Data_Storage

    CoreRunner -->|"load_config"| ConfigLoader
    ConfigLoader -->|"reads YAML"| ConfigFiles
    ConfigLoader -->|"config object"| CoreRunner

    CoreRunner -->|"select_pipeline"| EDA_Pipeline
    CoreRunner -->|"select_pipeline"| Detection_Pipeline
    CoreRunner -->|"select_pipeline"| Tracking_Pipeline

    EDA_Pipeline -->|"load_data"| EDA_Loader
    Detection_Pipeline -->|"load_data"| Generic_Loader
    Tracking_Pipeline -->|"load_data"| Generic_Loader

    Generic_Loader -->|"prepare_dataset"| TrainingDataset

    Detection_Pipeline -->|"infer_batch"| Detection_Strategies
    Tracking_Pipeline -->|"infer_batch"| Detection_Strategies
    EDA_Pipeline -->|"analyze_data"| TrainingDataset

    TrainingDataset -->|"train_batch"| Training_Engine
    Detection_Strategies -->|"detect_batch"| Training_Engine

    Training_Engine -->|"compute_metrics"| Det_Metrics
    Detection_Strategies -->|"compute_metrics"| Det_Metrics
    Tracking_Pipeline -->|"compute_metrics"| Track_Metrics

    Det_Metrics -->|"log_to_mlflow"| MLflow_Utils
    Track_Metrics -->|"log_to_mlflow"| MLflow_Utils
    MLflow_Utils -->|"send_runs"| MLflow_Server

    CoreRunner -->|"use_utils"| Device_Utils
    CoreRunner -->|"use_utils"| Logging_Utils
    CoreRunner -->|"use_utils"| Repro_Utils
    CoreRunner -->|"use_utils"| Torch_Utils

    CoreRunner -->|"save_artifacts"| Data_Storage
    Training_Engine -->|"save_model"| Data_Storage

    %% Click Events
    click EDA_Script "https://github.com/jubjones/spoton_ml/blob/main/src/run_eda.py"
    click Train_Script "https://github.com/jubjones/spoton_ml/blob/main/src/run_training_fasterrcnn.py"
    click Exp_Script "https://github.com/jubjones/spoton_ml/blob/main/src/run_experiment.py"
    click Comp_Script "https://github.com/jubjones/spoton_ml/blob/main/src/run_comparison.py"
    click Track_Script "https://github.com/jubjones/spoton_ml/blob/main/src/run_tracking_reid_comparison.py"
    click Export_Script "https://github.com/jubjones/spoton_ml/blob/main/scripts/export_reid_weights.py"

    click ConfigLoader "https://github.com/jubjones/spoton_ml/blob/main/src/utils/config_loader.py"
    click ConfigFiles "https://github.com/jubjones/spoton_ml/tree/main/configs/"

    click CoreRunner "https://github.com/jubjones/spoton_ml/blob/main/src/core/runner.py"

    click EDA_Pipeline "https://github.com/jubjones/spoton_ml/blob/main/src/pipelines/eda_pipeline.py"
    click Detection_Pipeline "https://github.com/jubjones/spoton_ml/blob/main/src/pipelines/detection_pipeline.py"
    click Tracking_Pipeline "https://github.com/jubjones/spoton_ml/blob/main/src/pipelines/tracking_reid_pipeline.py"

    click EDA_Loader "https://github.com/jubjones/spoton_ml/blob/main/src/data/eda_loader.py"
    click Generic_Loader "https://github.com/jubjones/spoton_ml/blob/main/src/data/loader.py"
    click TrainingDataset "https://github.com/jubjones/spoton_ml/blob/main/src/data/training_dataset.py"

    click Training_Engine "https://github.com/jubjones/spoton_ml/blob/main/src/training/pytorch_engine.py"
    click Training_Engine "https://github.com/jubjones/spoton_ml/blob/main/src/training/runner.py"

    click Detection_Strategies "https://github.com/jubjones/spoton_ml/blob/main/src/detection/strategies.py"

    click Det_Metrics "https://github.com/jubjones/spoton_ml/blob/main/src/evaluation/metrics.py"
    click Track_Metrics "https://github.com/jubjones/spoton_ml/blob/main/src/evaluation/tracking_metrics.py"

    click Device_Utils "https://github.com/jubjones/spoton_ml/blob/main/src/utils/device_utils.py"
    click Logging_Utils "https://github.com/jubjones/spoton_ml/blob/main/src/utils/logging_utils.py"
    click MLflow_Utils "https://github.com/jubjones/spoton_ml/blob/main/src/utils/mlflow_utils.py"
    click Repro_Utils "https://github.com/jubjones/spoton_ml/blob/main/src/utils/reproducibility.py"
    click Torch_Utils "https://github.com/jubjones/spoton_ml/blob/main/src/utils/torch_utils.py"

    %% Styles
    classDef cli_config fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef core_pipeline fill:#90EE90,stroke:#333,stroke-width:1px
    classDef data_ops fill:#FFA500,stroke:#333,stroke-width:1px
    classDef utils fill:#D3D3D3,stroke:#333,stroke-width:1px
```
