conservationtechlab/cougarvision

1. What this repo is

CougarVision is a set of command-line pipelines for wildlife camera-trap analysis, not a single app. The code centers on one repeated pattern: ingest media, run Megadetector-based animal detection, crop detections, classify cropped animals with an Imagenet-style model, then optionally emit alerts or annotated outputs.

2. Top-level execution entrypoints

The main workflows are separate scripts:

- fetch_and_alert.py for retrieving images from Strikeforce and email attachments, then alerting on detections

- stream_detect.py for live stream inference and optional recording

- run_batch_images.py for folder-based batch processing and annotation

- annotate_video.py for video annotation

- test_rtsp.py as a stream connectivity/test utility

This is a pipeline-style repo where each script is an orchestrator over shared utilities.

3. Shared processing library

Most reusable logic lives in cougarvision_utils/. The notable responsibilities are:

- detection inference helpers in detect_img.py

- crop generation in cropping.py and ImageCropGenerator.py

- image acquisition in get_images.py

- alert formatting/sending in alert.py

- email ingestion in fetch_emails.py

- logging helpers in log_utils.py

- generic utilities in more_utils.py

The entrypoint scripts are thin coordinators around these modules.

4. Core vision pipeline

The system’s main data flow is: input image/video frame → Megadetector animal detection → bounding-box crop generation → secondary classifier on crops → final label decision.

The README makes clear that the detector is Megadetector v5/PyTorch-compatible, while the classifier is interchangeable and model-specific. The repo is designed so detector and classifier model paths can be configured per workflow.

5. Input sources and acquisition boundaries

CougarVision supports several upstream media sources:

- local image folders for batch processing

- video files for annotation

- live RTSP-like streams

- Strikeforce camera trap downloads

- email inboxes with image attachments

These are distinct ingestion boundaries, but they all feed the same downstream detection/classification/alerting pipeline.

6. Strikeforce integration

fetch_and_alert.py depends on Strikeforce credentials and camera metadata. The helper scripts strikeforceget.py and strikeforcegetcameras.py are used to obtain the auth token and camera IDs. This creates a specific operational dependency on Strikeforce’s API and camera naming conventions, including the repo’s requirement that camera names be 4 characters long for downstream mapping.

7. Alerting and external response

fetch_and_alert.py and stream_detect.py can trigger alerts when target species are detected. The README mentions email/text alerts and optional EarthRanger event creation. That means the post-inference boundary is not just output generation; it can also push events into external operational systems. EarthRanger integration depends on matching camera names between the local config and the EarthRanger instance.

8. Batch output and artifacts

run_batch_images.py produces annotated images and JSON outputs. The README describes two important artifacts:

- an intermediate JSON for crop detections

- a final output.json for crop-level classifications

This implies a two-stage output model: first detection/cropping metadata, then final classification results. Batch runs also write rendered images and supporting data under configured output directories.

9. Streaming-specific behavior

stream_detect.py is the live-analysis path. It not only runs inference on frames, but can trigger recording when a cougar or lizard remains present. That makes it a stateful, continuous pipeline with temporal behavior, unlike the stateless batch scripts.

10. Configuration as the orchestration layer

The config/ directory contains separate YAML files for each workflow: fetch_and_alert.yml, stream_detect.yml, run_batch_images.yml, annotate_video.yml, and cameratraps.yml. These configs define model paths, credentials, camera mappings, output directories, and integration toggles. In practice, the architecture is “same code, different config-driven deployment modes.”

11. Model and label assets

The labels/ directory stores class lists and mappings such as megadetector.names, labels_map.txt, and region-specific label sets. This indicates the classifier layer is label-file driven and likely supports multiple deployment geographies/species sets without code changes.

12. Runtime and dependency environment

The repo is built around Conda environments (cougarvision_env.yml, mlgpuenv.yml) and an external CameraTraps checkout that is added to the Python path at runtime. That matters architecturally because inference is coupled to external ML tooling rather than being fully self-contained in this repo.

13. Operational outputs and logs

The README calls for separate images and logs directories, with local storage for downloaded media and per-image detection dataframes/logs. stream.log also exists at the repo root, suggesting live runs produce persistent operational logs alongside media artifacts.

14. Architectural summary

The cleanest diagram for this repo is: multiple media sources and integrations on the left, a shared detection/crop/classification core in the middle, and multiple sinks on the right: annotated files, JSON results, logs, alerts, email/text messages, and EarthRanger events. The same core pipeline is reused across batch, stream, and fetch/alert workflows, with config files determining which edges are active.