How weecology/doublecounting works
Project Type and Purpose
• This is a Python-based data-processing pipeline / command-line tool (not a web app) for airborne ecological object detection and de-duplication (“double counting”) in overlapping imagery.
• Its workflows include: building a structure-from-motion (SfM) model, running an object detector on images, aligning detections via homographies, removing duplicates, and visualizing results.
File Structure Analysis
• Top level:
• pipeline.py & pipeline.sh – the main orchestration script (CLI entry point and shell wrapper)
• create_image_pairs_for_annotation.py – utility to generate overlapping image pairs for manual annotation (LabelStudio)
• public/ – example outputs and images
• requirements.txt – Python dependencies
• tests/ – unit tests covering each sub-module
• scripts/ (Python package):
• stitching.py – core homography alignment and duplicate removal logic (align_and_delete)
• reconstruction.py – builds a SfM model using hloc & pycolmap
• labelstudio.py – helper to interface with LabelStudio for manual annotation
• reupload_to_dropbox.sh – uploads results to Dropbox
• Tests mirror scripts/ to validate predict, stitching, reconstruction, etc.
README Insights
• Describes three de-duplication strategies: “Stitch and Predict,” “Predict and Delete,” “Predict and Verify.”
• Emphasizes external dependencies: hloc, pycolmap, DeepForest, OpenCV, Matplotlib.
• Shows how each pipeline stage is invoked via Python API.
• Illustrates both 2D homography examples and optional 3D reconstruction outputs.
Core Components & Interactions (
a. Data Ingestion
• Input: Raw overlapping images (folder)
• Optional: Annotated image pairs from LabelStudio (for training/evaluation)
b. SfM Reconstruction Module
• scripts/reconstruction.py uses hloc & pycolmap → outputs matches.h5 and 3D point cloud
c. Object Detection Module
d. De-duplication Module
• scripts/stitching.py align_and_delete:
– Loads matches from SfM
– Computes homographies between image pairs
– Applies chosen strategy (stitch & predict, etc.)
– Outputs unique detections
e. Visualization & Output
• pipeline.py or user script
• Optionally re-uploads results via reupload_to_dropbox.sh
f. CLI Orchestration
• pipeline.sh calls pipeline.py or individual scripts in sequence
g. Testing
• pytest on tests/test_*.py ensures each component behaves as expected
Architectural Patterns & Design Principles
• Modular scripts with single responsibility: one script per major task
• File-based data passing ensures reproducibility and traceability
• Strategy pattern for duplicate removal (stitch-predict, predict-delete, predict-verify)
• Test-driven components allowing incremental validation
• Blue for input/data
• Green for core processing modules
• Orange for optional/third-party services
• Purple for output/visualization