How bladeszasza/sowlv2 works
Project Type and Purpose
• This is a Python‐based, open‐source command‐line tool (CLI application) named SOWLv2.
• It performs text‐prompted object segmentation on images, sequences of frames, or videos.
File Structure Highlights
• Top‐level:
• CLI scaffolding and project metadata (setup.py, pyproject.toml, requirements.txt).
• config/config_example.yaml for YAML‐based configuration.
• assets/ (logos, example images).
• docs/architecture.md (detailed architecture write‐up).
• notebooks/ (Colab & Jupyter demos).
• sowlv2/ package:
• cli.py: entrypoint, argument parsing, config loading.
• pipeline.py: orchestrator that routes to image or video pipeline.
• image_pipeline.py, video_pipeline.py: per‐input‐type processing.
• models/: wrappers for OWLv2 and SAM 2 (owl.py, sam2_wrapper.py).
• data/config.py & utils/path_config.py: configuration handling.
• utils/: filesystem IO, frame extraction, video utilities, pipeline helpers.
• tests/: unit and integration tests covering CLI, utils, end‐to‐end behavior.
Extracted Architectural Pattern
• Pipeline pattern with two sequential stages:
b. Segmentation stage (SAM 2) → produces masks & overlays.
• Configuration layer (CLI args or YAML) decoupled from pipeline logic.
• Modularized wrappers for each external model, making it extensible.
• Utilities for I/O operations and output assembly (images, masks, videos).
Main Components
A. Input Layer
• “User” via CLI or “Config File” (YAML).
• Supported inputs: single image, directory of frames, video file.
B. Configuration Loader
• Module: sowlv2/cli.py & data/config.py
• Parses args, loads YAML, resolves model names, thresholds, paths.
C. Pipeline Orchestrator
• Module: sowlv2/pipeline.py
• Routes to Image Pipeline or Video Pipeline based on input type.
D. Image Pipeline
• Module: image_pipeline.py
• Loop over frames/images: invokes Detection → Segmentation → I/O.
E. Video Pipeline
• Module: video_pipeline.py
• Extracts frames (via utils/frame_utils), runs image pipeline logic, reassembles videos (via utils/video_utils).
F. OWLv2 Detection Wrapper
• Module: sowlv2/models/owl.py
G. SAM 2 Segmentation Wrapper
• Module: sowlv2/models/sam2_wrapper.py
H. Output Generator / Filesystem
• Modules: utils/filesystem_utils.py, utils/pipeline_utils.py
• Writes binary masks, overlay images, merged outputs, and final videos to disk under structured directories.
I. External Dependencies / Services
• Transformers library (OWLv2), sam2 library (SAM 2), OpenCV + NumPy + Pillow for image/video I/O, PyTorch for model execution.
J. Storage
• Local filesystem (no external databases) with structured directories: binary/, overlay/, video/.
Relationships and Data Flow
• User ➔ CLI Parser ➔ Configuration Loader ➔ Pipeline Orchestrator
• Pipeline Orchestrator ➔ [Image Pipeline or Video Pipeline]
• For each frame/image:
└─➔ Output Generator writes mask files and overlay images
• Video Pipeline additionally: Frame extraction ➔ per‐frame pipeline ➔ video assembly.
Architectural Patterns and Principles
• Separation of concerns: parsing/configuration vs. orchestration vs. model invocation vs. I/O.
• Modular wrappers around external systems (OWL, SAM) for ease of swapping models.
• Pipeline abstraction supporting both image and video flows.
• High cohesion: each module has a clear single responsibility.
• Extensibility: new detectors or segmenters can be plugged in by adding wrappers and updating orchestrator.
Essential Elements to Capture
• Component labels exactly matching module names.