How mahmoudmater/ai_classroom_cheating_detection_system works
1. Repository shape
- This is a two-app system plus offline ML research:
- backend/ = FastAPI proctoring API and live inference runtime
- frontend/ = Next.js dashboard and analysis UI
- backend/notebooks/ = dataset preparation and model training experiments
- The architecture is centered on a real-time computer-vision proctoring pipeline with a web control plane and a persistent event store.
2. Offline ML pipeline is separate from runtime
- The notebooks implement the model lifecycle outside the web app:
- frame extraction from exam videos
- preprocessing and deduplication
- training/comparison of multiple image classifiers
- The README shows the winning production model is a custom CNN for cheating vs not_cheating classification.
- This offline path produces trained model artifacts that the runtime backend loads for inference.
3. Core runtime vision engine
- backend/proctoring.py is the main CV engine.
- It combines three signals per person:
- object detection for people and suspicious items
- pose estimation for head/gaze direction
- ROI classification with the trained CNN
- Those signals feed a per-person decision engine that emits CHEATING or OK for each frame.
4. Backend boundary: FastAPI as orchestration layer
- backend/main.py boots the API and applies app-wide concerns such as CORS and lifespan setup.
- Routers in backend/routers/ expose the system surface:
- session lifecycle endpoints
- event queries and summaries
- live WebSocket streaming
- single-image analysis
- model catalog and health checks
- The routers are thin; they delegate to domain services rather than containing CV logic.
5. Domain services split control, streaming, and persistence
- backend/services/session_manager.py owns session lifecycle and keeps running sessions mapped to proctoring workers.
- backend/services/proctoring_service.py wraps the blocking CV loop and starts it in a background thread.
- backend/services/frame_broadcaster.py bridges the thread-based CV loop to the async WebSocket layer.
- backend/services/event_store.py records cheating events and supports retrieval/summary queries.
6. Important runtime concurrency boundary
- The CV pipeline is intentionally not async.
- OpenCV/PyTorch work runs in a dedicated thread, while FastAPI/WebSocket handling stays on the asyncio event loop.
- That creates a key architectural bridge: frames and events move from the blocking CV thread to async clients through a thread-safe broadcaster.
7. Persistence model
- PostgreSQL is the long-term store, with migrations managed by Alembic.
- The data model is centered on:
- exam_sessions for session metadata and lifecycle state
- proctoring_events for per-person cheating incidents and timeline data
- The repository uses both async and sync database access paths:
- async SQLAlchemy/asyncpg for API reads
- sync DB access for the live CV thread when it writes events
8. Frontend structure
- frontend/app/ is a Next.js App Router application with routes for:
- landing
- dashboard
- live session view
- single-image analysis
- documentation pages
- The UI is componentized into reusable proctoring widgets such as live feed, event log, stats panels, session cards, model selector, and upload modal.
- frontend/lib/ contains API wrappers, hooks, types, and backend URL configuration.
9. Frontend data flow
- The dashboard and session pages fetch session data and event summaries over REST.
- The live session page opens a WebSocket to receive annotated frames and per-person verdicts.
- The canvas-based live feed renders the incoming JPEG frames and overlays boxes/labels client-side.
- This means the frontend is not doing ML; it is a visualization and control surface for backend inference.
10. Primary user flows
- Session flow: create session → choose model/source → start session → backend starts CV thread → frontend subscribes to WebSocket → stream frames and events → stop session → query summary.
- Image analysis flow: upload a single image → backend runs the same detection/classification logic without tracking → return annotated result and per-person verdicts.
- The architecture supports both long-running live monitoring and one-shot inference through the same backend domain model.
11. Model-selection and inference boundaries
- The system is model-driven: the frontend exposes a model catalog, and session creation includes a selected model file.
- The custom CNN is the production choice because it matches larger models while staying much smaller and faster.
- This makes model choice a runtime concern, but the actual inference logic remains inside the backend vision engine.
12. Infrastructure and packaging
- The backend is containerized with Docker and has a Compose setup for local orchestration.
- Uvicorn/ASGI is the serving model behind FastAPI.
- The frontend is a separate Node/Next.js application.
- The stack is therefore a classic split frontend + API backend + PostgreSQL deployment, with an additional offline ML training environment in notebooks.
13. What to show in an architecture diagram
- The most important boundaries are:
- offline dataset/model training vs online inference
- frontend control/visualization vs backend orchestration
- async web layer vs threaded CV execution
- transient live frames vs persisted cheating events
- The central flow is:
- session creation in the UI
- session state in PostgreSQL
- proctoring thread producing annotated frames/events
- WebSocket stream to the frontend
- REST queries for summaries and historical data
Main entry points: proctoring.py CV engine · routers API routers · app Next.js app · event_store.py Event store
Components 18
Backend
FastAPI proctoring API, live inference runtime, persistence, and orchestration.
- FastAPI app · API startup, CORS, lifespan wiring, and router registration.
backend/main.py - API routers · Session, event, stream, analysis, model catalog, and health endpoints.
backend/routers - Session manager · Owns session lifecycle and maps active sessions to proctoring workers.
backend/services/session_manager.py - Proctoring service · Starts the blocking CV loop in a background thread and manages live runs.
backend/services/proctoring_service.py - Frame broadcaster · Moves annotated frames and per-person verdicts from the CV thread to async clients.
backend/services/frame_broadcaster.py - Event store · Records cheating events and serves retrieval and summary queries.
backend/services/event_store.py - Image analysis · Runs the same detection and classification logic for single-image requests.
backend/services/image_analysis.py - CV engine · Core real-time proctoring loop that combines detection, pose, and ROI classification.
backend/proctoring.py - Object detector · Finds people and suspicious items in frames.
backend/proctoring.py - Pose model · Estimates head and gaze direction for each person.
backend/proctoring.py - ROI CNN · Custom CNN used to classify cheating versus not cheating from cropped regions.
backend/proctoring.py - Data model · Exam session and proctoring event tables plus ORM/Pydantic models.
backend/models/db_models.py - PostgreSQL · Long-term store for session metadata and cheating event timelines.
backend/core/database.py - Alembic migrations · Versioned database schema evolution.
backend/alembic
Frontend
Next.js dashboard and live analysis UI.
- Next.js app · App Router pages for dashboard, live sessions, analysis, and docs.
frontend/app - Proctoring widgets · Reusable live feed, event log, stats, session card, model selector, and upload UI.
frontend/components/proctoring - API client · Backend URL config, REST wrappers, hooks, and shared types.
frontend/lib
Offline ML
Notebook-based dataset preparation, model training, and comparison experiments.
- Training notebooks · Frame extraction, preprocessing, model comparison, and production model selection.
backend/notebooks
Connections 22
- Next.js app → Proctoring widgets: renders · Pages compose reusable proctoring UI components.
- Next.js app → API client: fetches · Pages use client hooks and API wrappers for data access.
- API client → API routers: REST/WebSocket · Frontend talks to backend endpoints and live streams.
- FastAPI app → API routers: mounts · App startup registers the API surface.
- API routers → Session manager: delegates · Session endpoints hand off lifecycle work.
- API routers → Event store: queries · Event endpoints retrieve and summarize proctoring events.
- API routers → Image analysis: invokes · Single-image analysis routes call the one-shot inference service.
- API routers → Frame broadcaster: streams · WebSocket routes publish annotated frames and verdicts.
- Session manager → Proctoring service: starts · Session lifecycle spins up live inference workers.
- Proctoring service → CV engine: runs · Background thread executes the blocking CV loop.
- CV engine → Object detector: uses · Per-frame person and object detection.
- CV engine → Pose model: uses · Pose and gaze estimation for each person.
- CV engine → ROI CNN: uses · CNN-based ROI classification for cheating decisions.
- CV engine → Frame broadcaster: publishes · Annotated frames and frame-level verdicts are sent to clients.
- CV engine → Event store: writes · Detected cheating incidents are persisted.
- Event store → PostgreSQL: stores in · Persistent event records live in PostgreSQL.
- Data model → PostgreSQL: maps to · ORM and schema models represent persisted tables.
- Alembic migrations → Data model: evolves · Alembic tracks schema changes for the data model.
- Training notebooks → ROI CNN: trains · Offline experiments produce the production CNN artifact.
- Training notebooks → CV engine: feeds artifacts · Trained model outputs are loaded by the runtime vision engine.
- Next.js app → API routers: REST data · Dashboard and session pages query summaries and historical data.
- Next.js app → Frame broadcaster: WebSocket frames · Live session view subscribes to real-time annotated streams.