How jasminaaa20/human-action-recognition-in-the-dark works
Main Components
• Data Preprocessing Pipeline
• Modules in the “keypoint_dataset_creation” directory (e.g., frame_extractor.py, preprocessing.py, keypoint_detector.py, etc.). These files constitute the pipeline to extract frames, apply histogram equalization, and detect key-points.
• How frames are extracted from videos (the provided .mp4 file) and then processed for further analysis.
• Model Training and Inference
• LRCN model, whose architecture is briefly mentioned in the README. This component handles both the training phase (using categorical cross-entropy loss and Adam optimizer) and inference. Note the presence of model files (.h5 and .pkl) located in the web_interface/API/models directory.
• Training steps (compilation, loss, accuracy metrics) and evaluation (e.g., confusion matrix generation).
• Web Interface (Full-Stack Application Components)
• Backend API: Represent the “web_interface/API” folder which contains server-side Python code including processing scripts (e.g., clahe_processor.py, face_recognition_class.py, pre_processing.py) and the main application script (app.py). This API acts as the bridge between the frontend and the machine learning backend.
• Frontend: Show the “web_interface/frontend” directory that includes Next.js-based React components (e.g., Face.jsx, Faces.jsx, VideoPlayer.jsx, WebCam.jsx). This provides a user interface for interacting with the backend and viewing results or live data.
• Notebook Analysis and Prototyping
Relationships and Interactions
• Data Flow: Begin with video input provided by raw video files (like drink-1-13_1uonzte1.mp4) → Frame Extraction → Preprocessing (including histogram equalization) → Keypoint Detection → Prepared data fed into the LRCN training/inference module.
• Model Artifacts: Represent storage/use of model files (e.g., .h5, .pkl) used for inference during API calls.
Architectural Patterns and Design Principles
• Modularity: Emphasize separation between data processing, model training/inference, and presentation layers.
• Full-stack Approach: A clear demarcation is present between the backend (machine learning logic and API endpoints) and the frontend (user interface).
• Pipeline Architecture: The preprocessing pipeline illustrates sequential data handling – from frame extraction and keypoint detection to feeding into the LRCN network.
Technologies, Frameworks, and Libraries
• Python modules for preprocessing and model training (Keras/TensorFlow for LRCN, YOLOv8-pose for key-point detection).
• Frontend tools: Next.js, React, Tailwind CSS indicated by files in the “web_interface/frontend” folder.
• API development framework in Python (Flask or FastAPI may be implicitly used given the structure in the API folder).