How fyp-hyz-mjj-2024/posture_mmpose works
Major Components
• User Interface & Entry Point:
• The project’s entry point is main.py. There is also a GUI.py indicating a graphical interface.
• Data Management:
• The “data” folder holds raw input data, training samples (in npy format) and pre-trained models (pth files).
• Annotation Module:
• The “step01_annotate_image_mmpose” package handles image annotation and keypoint configuration.
• Model Training (CNN):
• The “step02_train_model_cnn” module contains scripts to train a CNN model. This module likely uses annotated data derived from pose estimation outputs.
• Phone Detection Module:
• “step03_yolo_phone_detection” carries the YOLO-based phone detection. It contains configurations, scripts for model running and training (with ONNX checkpoints provided).
• Pre-configured Model & Checkpoint Repository:
• The “model_config” folder stores configuration files and checkpoint files for both the detection and the pose estimation modules.
• Core MMPose & MM Packages:
• A huge “mmpose_configs” directory indicates an extensive set of configuration submodules covering 2D/3D keypoints, face, hand, whole-body keypoint detection, etc.
• Utility Tools:
• The “utils” folder includes additional scripts (for file parsing, plotting, opencv utilities) implemented as helper tools.
• Project Documentation/Supporting Materials:
• “project_docs” holds reference materials, and there are LICENSE texts and Notes.md.
Data Flow & Interactions
• Start at the User Interface (GUI or directly via main.py). The user initiates the process.
• Raw input data is ingested:
• Videos/images (from the “data” folder) are fed into the annotation module (step01) where pose keypoint data is extracted using mmpose configurations.
• The annotated data then flows into a training pipeline:
• The “step02_train_model_cnn” module uses this output along with associated configurations to train a CNN that helps decide whether a pedestrian is using a cell phone.
• Simultaneously, the “step03_yolo_phone_detection” module is responsible for detecting phones through object detection:
• It uses YOLO-based models (with checkpoints stored under model_config and step03 directories) to separately detect the phone’s presence.
• Finally, the outputs from pose estimation and phone detection modules can be fused (within main.py or an integration layer) to arrive at the final prediction.
Architectural Patterns & Technologies
• Modularity & Separation of Concerns:
• Different steps (annotation, training, detection) are kept in separate packages.
• Configuration-Driven:
• Extensive use of configuration files (in model_config and mmpose_configs) for components indicates a decoupled design where models and parameters can be interchanged.
• Pipeline-based Processing:
• Data flows sequentially through preprocessing, pose estimation, object detection, training and inference – which can be depicted as separate pipeline stages.
• Deep Learning & Computer Vision Libraries:
• Use of PyTorch (specific 2.1.0 version), mmcv, mmdet, and mmpose is central. Represent these libraries near the respective modules.
• Environment & Dependency Management: