How shravansuthar2/smart-crowd-safety works
1. Overall shape
- This repository is a two-part system: a FastAPI backend that performs AI inference and alerting, and a static dashboard frontend that consumes the backend APIs.
- A separate training workspace produces or fine-tunes the models used at runtime.
2. Runtime request flow
- User actions in the dashboard trigger HTTP requests to the backend.
- The backend receives image/video payloads, runs one or more detectors, stores or updates alert/person state, and returns annotated results plus metadata for display.
3. Backend entry point and API boundary
- backend/main.py is the application entry point and API host.
- The API is organized into routers under backend/routers/:
- detection.py for crowd, fire, heatmap, and video processing endpoints
- persons.py for missing-person registration and lookup
- alerts.py for alert listing and clearing
- This is the main boundary between external clients and internal ML/business logic.
4. Detection subsystem
- backend/modules/crowd_density.py handles people counting and overcrowding detection using YOLOv8.
- backend/modules/emergency.py handles fire/emergency detection using a custom YOLOv8 fire model.
- backend/modules/face_finder.py handles missing-person search using InsightFace embeddings and similarity matching.
- These modules are the inference layer and likely operate frame-by-frame on uploaded media.
5. Video processing pipeline
- The system is designed for real-time or batch video processing rather than just single-image inference.
- OpenCV is used for reading frames, running detections, and producing annotated output video.
- The backend exposes progress/status and download endpoints for long-running video jobs, indicating asynchronous or staged processing within the server.
6. Alerting and event state
- backend/modules/alert_manager.py centralizes alert creation, acknowledgment, resolution, and clearing.
- Alerts are generated from detection outcomes, such as overcrowding, fire, or a matched missing person.
- This subsystem forms the operational event layer above raw model predictions.
7. Persistence and Firebase boundary
- backend/firebase_config.py and dashboard/firebase-config.js show an optional Firebase integration on both backend and frontend sides.
- The README states the database is Firebase Firestore optional and works locally, so persistence is likely abstracted to allow either remote Firebase storage or local fallback behavior.
- This is the main data persistence boundary for alerts and registered persons.
8. Frontend/dashboard
- dashboard/index.html, dashboard/style.css, and dashboard/app.js implement a static, dark-themed UI.
- The dashboard is not a separate application server; it is a client that talks to the backend APIs and may also use Firebase configuration for remote data access or auth-related behavior.
- The README notes it is accessible at /dashboard when served by FastAPI, which means the backend also serves the static frontend.
9. Model assets and runtime dependencies
- The fire model weights live in backend/fire_model.pt and are tracked via Git LFS because of size.
- YOLOv8 person detection and InsightFace face models are downloaded automatically at runtime, while the fire detector is bundled with the repo.
- This creates a split between runtime-supplied models and repo-bundled custom weights.
10. Training subsystem
- training/train_crowd_yolo.py and training/train_fire.py are offline model-training scripts.
- training/download_dataset.py and the included backend/datasets/coco128/ directory provide training data support.
- The training subsystem is separate from production inference; it exists to regenerate weights rather than serve requests.
11. Crowd-safety domain logic
- The system combines three safety signals into one operational workflow: crowd density, missing persons, and fire/emergency detection.
- These are not isolated features; they converge in the same alert system and video-processing pipeline, which is the core domain composition of the repo.
12. Configuration and thresholds
- backend/config.py contains operational thresholds such as crowd density, confidence, image size, and face-similarity settings.
- These values control detection sensitivity and directly affect when alerts are emitted, so they are part of the runtime policy layer.
13. Deployment/container boundary
- The repo includes a Dockerfile and a Hugging Face Space-style README header indicating containerized deployment.
- The documented deployment path is a single backend service serving APIs and the dashboard, with optional Firebase and model artifacts bundled or downloaded at startup.
14. What an architecture diagram should emphasize
- Show the dashboard as the client surface.
- Show FastAPI as the central orchestrator.
- Show three ML inference modules feeding a shared alert manager.
- Show persistence as optional Firebase/local storage for persons and alerts.
- Show training as a separate offline lane that outputs model artifacts into the backend runtime.
Main entry points: detection.py Detection API · app.js Dashboard app · main.py FastAPI app · persons.py Persons API
Components 21
Backend
FastAPI server, routers, inference modules, and runtime config.
- FastAPI app · Application entry point and server for APIs and static dashboard assets.
backend/main.py - Detection API · Endpoints for crowd, fire, heatmap, and video processing.
backend/routers/detection.py - Persons API · Endpoints for missing-person registration and lookup.
backend/routers/persons.py - Alerts API · Endpoints for listing, acknowledging, and clearing alerts.
backend/routers/alerts.py - Crowd density · People counting and overcrowding detection using YOLOv8.
backend/modules/crowd_density.py - Fire detect · Fire and emergency detection using the bundled custom model.
backend/modules/emergency.py - Face finder · Missing-person search using embeddings and similarity matching.
backend/modules/face_finder.py - Pickpocket · Additional safety detector for suspicious activity.
backend/modules/pickpocket.py - Alert manager · Centralizes alert creation, acknowledgement, resolution, and clearing.
backend/modules/alert_manager.py - Schemas · Shared request and response models for API and runtime data.
backend/models/schemas.py - Config · Runtime thresholds and sensitivity settings that control detection behavior.
backend/config.py - Firebase config · Backend-side Firebase/Firestore integration and local fallback boundary.
backend/firebase_config.py - Fire module · Bundled fire-model loading and detection implementation used at runtime.
backend/fire_module/fire_detection.py - Fire weights · Custom fire detector weights bundled with the repository.
backend/fire_model.pt
Dashboard
Static client served by the backend.
- Dashboard UI · Dark-themed dashboard shell and controls for operators.
dashboard/index.html - Dashboard app · Frontend logic that calls backend APIs and renders results.
dashboard/app.js - Firebase client · Frontend-side Firebase configuration for optional remote persistence.
dashboard/firebase-config.js
Training
Offline model-training and dataset preparation lane.
- Crowd train · Offline training script for the crowd-density detector.
training/train_crowd_yolo.py - Fire train · Offline training script for the fire detector.
training/train_fire.py - Dataset prep · Dataset download and preparation support for model training.
training/download_dataset.py
Other components
- Firebase · Optional Firestore persistence used by backend and dashboard.
Connections 29
- FastAPI app → Detection API: mounts · Registers detection routes.
- FastAPI app → Persons API: mounts · Registers person-management routes.
- FastAPI app → Alerts API: mounts · Registers alert routes.
- FastAPI app → Dashboard UI: serves · Hosts the static dashboard.
- Dashboard UI → Dashboard app: loads · Browser loads client logic.
- Dashboard app → Detection API: calls · Sends media and detection requests.
- Dashboard app → Persons API: calls · Submits person registration and lookup requests.
- Dashboard app → Alerts API: calls · Fetches and clears alert state.
- Dashboard app → Firebase client: uses · Optional frontend Firebase integration.
- Detection API → Crowd density: runs · Crowd and overcrowding inference.
- Detection API → Fire detect: runs · Fire/emergency inference.
- Detection API → Face finder: runs · Missing-person image matching.
- Detection API → Pickpocket: runs · Additional safety detection path.
- Detection API → Schemas: uses · Request and response payloads.
- Persons API → Face finder: uses · Registers and searches faces.
- Persons API → Firebase config: persists · Stores registered-person data when enabled.
- Alerts API → Alert manager: uses · Queries and mutates alert state.
- Crowd density → Config: reads · Uses crowd thresholds and sensitivity settings.
- Fire detect → Config: reads · Uses runtime confidence and image settings.
- Face finder → Config: reads · Uses face-similarity thresholds.
- Alert manager → Firebase config: stores · Writes alert state through the persistence boundary.
- Firebase config → Firebase: connects · Optional Firestore backend.
- Firebase client → Firebase: connects · Optional client-side Firebase access.
- Fire module → Fire weights: loads · Uses bundled fire weights at runtime.
- Fire module → Fire detect: implements · Backend emergency detection wraps the custom fire stack.
- Crowd train → Crowd density: produces · Retrains the crowd detector used in runtime.
- Fire train → Fire module: produces · Trains the custom fire detector stack.
- Dataset prep → Crowd train: feeds · Provides dataset setup for training.
- Dataset prep → Fire train: feeds · Provides dataset setup for training.