# abishek-kk/railmind-ai architecture

> Architecture diagram of the GitHub repository abishek-kk/railmind-ai, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/abishek-kk/railmind-ai
- Repository: https://github.com/abishek-kk/railmind-ai

## Overview

1. **Repository shape**
   - This is a **monorepo** with two primary deployable applications:
     - `backend/`: a Python 3.11 FastAPI service that owns CV inference, risk scoring, alerting, persistence, and training APIs.
     - `frontend/`: a React 19 single-page dashboard for live monitoring, alerts, and analytics.
   - The repo also includes training notebooks/scripts, test suites, and a checked-in SQLite database for local/dev use.

2. **End-to-end system boundary**
   - The architecture is split between an **edge processing pipeline** and a **cloud/API layer**.
   - Raw CCTV video is processed close to the camera in the backend CV pipeline, converted into compact behavioral signals, then handed to the API/business layer for alerting, storage, and dashboard delivery.
   - The frontend never touches video directly; it consumes API data and WebSocket events from the backend.

3. **Backend entrypoints and runtime**
   - `backend/app/main.py` is the FastAPI application entrypoint; `backend/run.py` and `uvicorn app.main:app` are the main launch paths.
   - Configuration, DB setup, scheduler, and WebSocket management live under `backend/app/core/`.
   - The backend is organized as a layered service: API routes → service layer → models/database → analytics / agent / CV subsystems.

4. **Computer vision and edge inference pipeline**
   - The CV path is under `backend/app/cv/` and `backend/app/features/`.
   - It processes video frames through OpenCV, YOLOv8 person detection, ByteTrack-style tracking, and YOLOv8 pose estimation.
   - Feature modules such as `edge_proximity`, `loitering_detector`, `pacing_detector`, `following_detector`, and `movement_analyzer` generate the behavioral inputs used downstream.

5. **Temporal ML pipeline**
   - The LSTM subsystem under `backend/app/lstm/` builds, preprocesses, trains, and predicts on behavioral sequences.
   - The model is designed around sliding temporal windows and is treated as a separate capability from the rule-based feature detectors.
   - Training utilities, CLI, and model generation scripts suggest the repo supports both offline training and runtime inference with saved artifacts.

6. **Agentic reasoning layer**
   - `backend/app/agents/` contains a three-stage LangGraph-style flow: `perception_agent`, `reasoning_agent`, and `intervention_agent`, composed in `agent_graph.py`.
   - This layer turns model outputs and incident context into risk decisions, escalation levels, and recommended actions.
   - It is the main boundary between pure detection and operational response.

7. **Business logic and alert lifecycle**
   - The service layer under `backend/app/services/` owns alert creation, escalation, notification, incident handling, training orchestration, and risk scoring.
   - `risk_scoring.py`, `alert_service.py`, `incident_service.py`, and `escalation_service.py` appear to be the core orchestration points for converting detections into actionable incidents.
   - Notification delivery is separated from core logic, with support for email/SMS style escalation flows.

8. **API surface**
   - FastAPI routes are grouped by domain in `backend/app/api/routes/`: alerts, analytics, dashboard, feeds, health, incidents, staff, and training.
   - There is a clear split between synchronous REST endpoints for querying/managing state and WebSocket endpoints for live event delivery.
   - `backend/app/api/deps.py` likely provides shared dependencies such as DB sessions, auth/context, or service injection.

9. **Persistence and domain model**
   - Domain entities live under `backend/app/models/` with matching request/response schemas under `backend/app/schemas/`.
   - The README indicates core persisted concepts: incidents, alerts, tracks, analytics, staff, platforms, and feedback.
   - SQLite is used locally (`data/railmind.db`), while PostgreSQL is intended for production.

10. **Realtime transport**
    - `backend/app/core/websocket_manager.py` manages live push delivery to the UI.
    - The frontend subscribes to alert and live-feed streams, so the backend acts as the realtime hub for operational monitoring.
    - This is important architecturally: the dashboard is not polling for everything; some state changes are event-driven.

11. **Analytics and dashboard preparation**
    - `backend/app/analytics/` contains precomputed metrics for heatmaps, incident stats, and dashboard summaries.
    - These modules sit between raw incident data and frontend visualization, which implies the dashboard consumes curated aggregates rather than reconstructing analytics client-side.
    - This helps keep the frontend thin and the backend authoritative for business metrics.

12. **Frontend structure**
    - `frontend/src/routes/` contains the top-level app views: dashboard, alerts, live, and the landing page.
    - `frontend/src/components/` holds reusable UI pieces such as feed cards, risk badges, stats cards, sidebar/top bar, and a large shadcn/ui component set.
    - The frontend is a standard client app with route-based views, shared API clients in `frontend/src/lib/api/`, and a WebSocket hook in `frontend/src/hooks/useWebSocket.ts`.

13. **Frontend data flow**
    - The frontend fetches domain data through typed API helper modules (`alerts.ts`, `dashboard.ts`, `feeds.ts`) and subscribes to live updates over WebSocket.
    - It is primarily a presentation layer for alerts, feed status, trends, and live risk classification.
    - E2E tests under `frontend/tests/e2e/` indicate browser-level validation of the live feed and dashboard experience.

14. **Training and retraining pipeline**
    - Training assets live under `backend/training/` and `backend/app/lstm/`.
    - There are notebooks for analysis, plus scripts for feature extraction, dataset preparation, evaluation, and LSTM training.
    - The backend also exposes training-related API routes, so model training is operationalized rather than being a separate offline-only workflow.

15. **Operational tooling and validation**
    - The repo includes scripts for demo mode, video processing, and stream URL backfilling under `backend/scripts/`.
    - Tests cover agents, CV, LSTM, risk scoring, analytics, alerts, and escalation behavior, which reflects the main architectural seams.
    - CI is present in `.github/workflows/ci.yml`, so backend and frontend behaviors are likely validated automatically.

16. **External integrations and boundaries**
    - Optional integrations include LLM providers for reasoning, SMTP for email escalation, and Twilio for SMS alerts.
    - The system is designed to be privacy-preserving: no face recognition, no biometric identity persistence, and track/session IDs are used instead.
    - For an architecture diagram, the critical boundaries are: **camera/video sources → edge CV/ML → backend reasoning/services/storage → WebSocket/API → React dashboard**, with optional notification services branching off the alerting layer.

Main entry points: [`backend/app/services`](https://github.com/abishek-kk/railmind-ai/tree/main/backend/app/services) (Services), [`backend/app/api/routes`](https://github.com/abishek-kk/railmind-ai/tree/main/backend/app/api/routes) (API routes), [`backend/app/core`](https://github.com/abishek-kk/railmind-ai/tree/main/backend/app/core) (Core runtime), [`frontend/src/routes`](https://github.com/abishek-kk/railmind-ai/tree/main/frontend/src/routes) (Dashboard views)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_backend["Backend"]
  node_backend_entry["FastAPI app<br/>api entrypoint<br/>[main.py]"]
  node_backend_core["Core runtime<br/>infrastructure"]
  node_processor_manager["Processor mgr<br/>video orchestration"]
  node_websocket_manager["Realtime hub<br/>websocket delivery"]
  node_cv_pipeline["CV pipeline<br/>edge vision"]
  node_feature_detectors["Feature detectors<br/>behavior features"]
  node_lstm_runtime["LSTM runtime<br/>temporal ML"]
  node_agent_graph["Agent graph<br/>reasoning flow<br/>[agent_graph.py]"]
  node_service_layer["Services<br/>business logic"]
  node_alert_service["Alerting<br/>alert lifecycle<br/>[alert_service.py]"]
  node_risk_scoring["Risk scoring<br/>[risk_scoring.py]"]
  node_incident_service["Incidents<br/>incident orchestration"]
  node_notification_service["Notifications<br/>delivery"]
  node_analytics_layer["Analytics<br/>dashboard metrics"]
  node_api_routes["API routes<br/>REST surface"]
  node_models_schemas["Domain model<br/>persistence model"]
end

subgraph group_frontend["Frontend"]
  node_frontend_app["React app<br/>SPA entrypoint<br/>[main.tsx]"]
  node_routes_views["Dashboard views<br/>route views"]
  node_ui_components["UI components<br/>presentation"]
  node_api_client["API client<br/>frontend data"]
  node_ws_hook["WebSocket hook<br/>live updates<br/>[useWebSocket.ts]"]
end

subgraph group_training["Training"]
  node_training_assets["Training assets<br/>model training"]
end

subgraph group_external["External"]
  node_camera_sources(("Camera feeds<br/>video input"))
  node_database[("SQLite DB<br/>local persistence<br/>[railmind.db]")]
end

node_camera_sources -->|"video frames"| node_cv_pipeline
node_backend_entry -->|"bootstraps"| node_backend_core
node_backend_core -->|"manages"| node_processor_manager
node_backend_core -->|"hosts"| node_websocket_manager
node_cv_pipeline -->|"signals"| node_feature_detectors
node_cv_pipeline -->|"sequences"| node_lstm_runtime
node_feature_detectors -->|"features"| node_lstm_runtime
node_lstm_runtime -->|"predictions"| node_agent_graph
node_feature_detectors -->|"context"| node_agent_graph
node_agent_graph -->|"decides"| node_risk_scoring
node_risk_scoring -->|"raises"| node_alert_service
node_risk_scoring -->|"opens"| node_incident_service
node_alert_service -->|"notifies"| node_notification_service
node_incident_service -->|"escalates"| node_notification_service
node_service_layer -->|"orchestrates"| node_alert_service
node_service_layer -->|"orchestrates"| node_risk_scoring
node_service_layer -->|"orchestrates"| node_incident_service
node_service_layer -->|"orchestrates"| node_notification_service
node_service_layer -->|"feeds"| node_analytics_layer
node_api_routes -->|"calls"| node_service_layer
node_api_routes -->|"serves"| node_analytics_layer
node_api_routes -->|"reads/writes"| node_models_schemas
node_models_schemas -->|"persists"| node_database
node_backend_core -->|"connects"| node_database
node_websocket_manager -->|"pushes events"| node_frontend_app
node_api_routes -->|"serves data"| node_frontend_app
node_api_client -->|"fetches"| node_api_routes
node_ws_hook -->|"subscribes"| node_websocket_manager
node_frontend_app -->|"renders"| node_routes_views
node_routes_views -->|"composes"| node_ui_components
node_routes_views -->|"loads data"| node_api_client
node_routes_views -->|"listens live"| node_ws_hook
node_training_assets -->|"trains"| node_lstm_runtime
node_api_routes -->|"exposes training"| node_training_assets

click node_backend_entry "https://github.com/abishek-kk/railmind-ai/blob/main/backend/app/main.py"
click node_backend_core "https://github.com/abishek-kk/railmind-ai/tree/main/backend/app/core"
click node_processor_manager "https://github.com/abishek-kk/railmind-ai/blob/main/backend/app/core/processor_manager.py"
click node_websocket_manager "https://github.com/abishek-kk/railmind-ai/blob/main/backend/app/core/websocket_manager.py"
click node_cv_pipeline "https://github.com/abishek-kk/railmind-ai/tree/main/backend/app/cv"
click node_feature_detectors "https://github.com/abishek-kk/railmind-ai/tree/main/backend/app/features"
click node_lstm_runtime "https://github.com/abishek-kk/railmind-ai/tree/main/backend/app/lstm"
click node_agent_graph "https://github.com/abishek-kk/railmind-ai/blob/main/backend/app/agents/agent_graph.py"
click node_service_layer "https://github.com/abishek-kk/railmind-ai/tree/main/backend/app/services"
click node_alert_service "https://github.com/abishek-kk/railmind-ai/blob/main/backend/app/services/alert_service.py"
click node_risk_scoring "https://github.com/abishek-kk/railmind-ai/blob/main/backend/app/services/risk_scoring.py"
click node_incident_service "https://github.com/abishek-kk/railmind-ai/blob/main/backend/app/services/incident_service.py"
click node_notification_service "https://github.com/abishek-kk/railmind-ai/blob/main/backend/app/services/notification_service.py"
click node_analytics_layer "https://github.com/abishek-kk/railmind-ai/tree/main/backend/app/analytics"
click node_api_routes "https://github.com/abishek-kk/railmind-ai/tree/main/backend/app/api/routes"
click node_models_schemas "https://github.com/abishek-kk/railmind-ai/tree/main/backend/app/models"
click node_database "https://github.com/abishek-kk/railmind-ai/blob/main/data/railmind.db"
click node_frontend_app "https://github.com/abishek-kk/railmind-ai/blob/main/frontend/src/main.tsx"
click node_routes_views "https://github.com/abishek-kk/railmind-ai/tree/main/frontend/src/routes"
click node_ui_components "https://github.com/abishek-kk/railmind-ai/tree/main/frontend/src/components"
click node_api_client "https://github.com/abishek-kk/railmind-ai/tree/main/frontend/src/lib/api"
click node_ws_hook "https://github.com/abishek-kk/railmind-ai/blob/main/frontend/src/hooks/useWebSocket.ts"
click node_training_assets "https://github.com/abishek-kk/railmind-ai/tree/main/backend/training"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_backend_entry,node_backend_core,node_processor_manager,node_websocket_manager,node_cv_pipeline,node_feature_detectors,node_lstm_runtime,node_agent_graph,node_service_layer,node_alert_service,node_risk_scoring,node_incident_service,node_notification_service,node_analytics_layer,node_api_routes,node_models_schemas toneBlue
class node_frontend_app,node_routes_views,node_ui_components,node_api_client,node_ws_hook toneAmber
class node_training_assets toneMint
class node_camera_sources,node_database toneRose
```

## Components

### Backend

FastAPI service for CV ingestion, reasoning, persistence, and live delivery.

- [FastAPI app](https://github.com/abishek-kk/railmind-ai/blob/main/backend/app/main.py): api entrypoint · Backend application entrypoint and main runtime bootstrap. · `backend/app/main.py`
- [Core runtime](https://github.com/abishek-kk/railmind-ai/tree/main/backend/app/core): infrastructure · Configuration, database setup, scheduler, and websocket coordination. · `backend/app/core`
- [Processor mgr](https://github.com/abishek-kk/railmind-ai/blob/main/backend/app/core/processor_manager.py): video orchestration · Coordinates video processors and runtime processing workers. · `backend/app/core/processor_manager.py`
- [Realtime hub](https://github.com/abishek-kk/railmind-ai/blob/main/backend/app/core/websocket_manager.py): websocket delivery · Pushes live events and state changes to connected clients. · `backend/app/core/websocket_manager.py`
- [CV pipeline](https://github.com/abishek-kk/railmind-ai/tree/main/backend/app/cv): edge vision · Video processing and pose/detection pipeline that extracts behavior signals. · `backend/app/cv`
- [Feature detectors](https://github.com/abishek-kk/railmind-ai/tree/main/backend/app/features): behavior features · Rule-based behavioral detectors such as loitering, pacing, following, and proximity. · `backend/app/features`
- [LSTM runtime](https://github.com/abishek-kk/railmind-ai/tree/main/backend/app/lstm): temporal ML · Sliding-window sequence model for behavior prediction and scoring. · `backend/app/lstm`
- [Agent graph](https://github.com/abishek-kk/railmind-ai/blob/main/backend/app/agents/agent_graph.py): reasoning flow · Three-stage perception, reasoning, and intervention flow for operational decisions. · `backend/app/agents/agent_graph.py`
- [Services](https://github.com/abishek-kk/railmind-ai/tree/main/backend/app/services): business logic · Alerting, escalation, incident handling, risk scoring, notification, and training orchestration. · `backend/app/services`
- [Alerting](https://github.com/abishek-kk/railmind-ai/blob/main/backend/app/services/alert_service.py): alert lifecycle · Creates and manages alerts and their state transitions. · `backend/app/services/alert_service.py`
- [Risk scoring](https://github.com/abishek-kk/railmind-ai/blob/main/backend/app/services/risk_scoring.py): scoring · Converts detections and context into risk levels and priorities. · `backend/app/services/risk_scoring.py`
- [Incidents](https://github.com/abishek-kk/railmind-ai/blob/main/backend/app/services/incident_service.py): incident orchestration · Builds incidents from detections and manages incident lifecycle. · `backend/app/services/incident_service.py`
- [Notifications](https://github.com/abishek-kk/railmind-ai/blob/main/backend/app/services/notification_service.py): delivery · Sends outbound notifications such as email/SMS escalation messages. · `backend/app/services/notification_service.py`
- [Analytics](https://github.com/abishek-kk/railmind-ai/tree/main/backend/app/analytics): dashboard metrics · Precomputed metrics and summaries for dashboards and trends. · `backend/app/analytics`
- [API routes](https://github.com/abishek-kk/railmind-ai/tree/main/backend/app/api/routes): REST surface · Domain routes for alerts, analytics, dashboard, feeds, incidents, staff, health, and training. · `backend/app/api/routes`
- [Domain model](https://github.com/abishek-kk/railmind-ai/tree/main/backend/app/models): persistence model · Persisted entities and request/response schemas for alerts, feeds, incidents, staff, and training runs. · `backend/app/models`

### Frontend

React dashboard for monitoring alerts, feeds, and analytics.

- [React app](https://github.com/abishek-kk/railmind-ai/blob/main/frontend/src/main.tsx): SPA entrypoint · Frontend application bootstrap and routing setup. · `frontend/src/main.tsx`
- [Dashboard views](https://github.com/abishek-kk/railmind-ai/tree/main/frontend/src/routes): route views · Top-level pages for dashboard, alerts, live monitoring, and landing. · `frontend/src/routes`
- [UI components](https://github.com/abishek-kk/railmind-ai/tree/main/frontend/src/components): presentation · Reusable cards, navigation, badges, feed widgets, and shared UI primitives. · `frontend/src/components`
- [API client](https://github.com/abishek-kk/railmind-ai/tree/main/frontend/src/lib/api): frontend data · Typed REST helpers for alerts, dashboard, and feeds. · `frontend/src/lib/api`
- [WebSocket hook](https://github.com/abishek-kk/railmind-ai/blob/main/frontend/src/hooks/useWebSocket.ts): live updates · Client-side subscription hook for realtime backend events. · `frontend/src/hooks/useWebSocket.ts`

### Training

Offline and operational model training pipeline for temporal behavior models.

- [Training assets](https://github.com/abishek-kk/railmind-ai/tree/main/backend/training): model training · Scripts and notebooks for feature extraction, dataset prep, evaluation, and LSTM training. · `backend/training`

### External

Non-repo systems and data sources the application integrates with.

- **Camera feeds**: video input · CCTV/video streams that enter the edge processing pipeline.
- [SQLite DB](https://github.com/abishek-kk/railmind-ai/blob/main/data/railmind.db): local persistence · Checked-in local database used for development and testing. · `data/railmind.db`

## Connections

- Camera feeds → CV pipeline: video frames · Raw CCTV/video streams are processed by the backend CV pipeline.
- FastAPI app → Core runtime: bootstraps · The FastAPI app initializes core runtime services and infrastructure.
- Core runtime → Processor mgr: manages · Core runtime coordinates video processors and workers.
- Core runtime → Realtime hub: hosts · Core runtime exposes websocket delivery infrastructure.
- CV pipeline → Feature detectors: signals · Video processing produces behavioral features for downstream analysis.
- CV pipeline → LSTM runtime: sequences · Temporal behavior sequences are prepared for ML inference.
- Feature detectors → LSTM runtime: features · Detector outputs can be consumed by temporal sequence modeling.
- LSTM runtime → Agent graph: predictions · Model outputs are handed to the agentic reasoning layer.
- Feature detectors → Agent graph: context · Rule-based detections provide context for operational reasoning.
- Agent graph → Risk scoring: decides · Reasoning stage informs risk and escalation decisions.
- Risk scoring → Alerting: raises · Risk results can create or update alerts.
- Risk scoring → Incidents: opens · High-risk events can become incidents.
- Alerting → Notifications: notifies · Alert lifecycle can trigger outbound notifications.
- Incidents → Notifications: escalates · Incident handling can escalate through notification channels.
- Services → Alerting: orchestrates · Central service layer coordinates alert management.
- Services → Risk scoring: orchestrates · Central service layer coordinates risk computation.
- Services → Incidents: orchestrates · Central service layer coordinates incident handling.
- Services → Notifications: orchestrates · Central service layer coordinates external notifications.
- Services → Analytics: feeds · Operational data is summarized into dashboard metrics.
- API routes → Services: calls · REST endpoints delegate to business services.
- API routes → Analytics: serves · Analytics routes expose curated metrics and summaries.
- API routes → Domain model: reads/writes · Endpoints exchange structured domain objects and schemas.
- Domain model → SQLite DB: persists · Domain entities are stored in the SQLite-backed local database.
- Core runtime → SQLite DB: connects · Database setup and sessions are managed from the core runtime.
- Realtime hub → React app: pushes events · Realtime backend events are delivered to the dashboard.
- API routes → React app: serves data · Frontend views query backend REST endpoints for state.
- API client → API routes: fetches · Typed API helpers call the backend route surface.
- WebSocket hook → Realtime hub: subscribes · Client hook listens to backend websocket streams.
- React app → Dashboard views: renders · Frontend app mounts route-based views.
- Dashboard views → UI components: composes · Pages are built from shared UI components.
- Dashboard views → API client: loads data · Route views fetch alerts, dashboard, and feed data.
- Dashboard views → WebSocket hook: listens live · Live views subscribe to realtime updates.
- Training assets → LSTM runtime: trains · Offline scripts and notebooks build and evaluate temporal models.
- API routes → Training assets: exposes training · Training endpoints operationalize model training workflows.