How prashant-1285/action_recognition works
Main Components
• User Interface: Represented by the Streamlit application (application.py) that provides a web interface for video upload and result display.
• Inference Engine: The core logic that processes video input. This includes the APIs in pyskl/apis (inference.py & train.py) that handle inference requests.
• Models & Configurations: The “pyskl” package holds model definitions, dataset pipelines, and utilities. Additionally, the “configs” directories (such as aagcn, posec3d, stgcn, etc.) contain configuration files for multiple action recognition models.
• Pre-trained Checkpoints: Stored under the “ckpts” directory, these files are used to load various pre-trained models.
• Data Handling & Pipelines: The “pyskl/datasets” and “pyskl/datasets/pipelines” directories are in charge of loading, pre-processing, and augmenting video/skeleton data.
• Auxiliary Tools: Under the “tools” directory (with train.py, test.py, dist_*.sh, etc.) and “demo” folders, there are scripts for training, testing, and demonstrating various functionalities.
• External Libraries: Integration with external packages like Streamlit (for the UI) and standard Python ecosystem components (PyTorch, etc.) is also essential.
Relationships and Interactions
• The User Interface (front-end) triggers the inference process by uploading a video via the Streamlit app.
• Upon submission, application.py calls the inference API (pyskl/apis/inference.py), which then handles pre-processing by leveraging dataset pipelines from pyskl/datasets/pipelines.
• The inference module loads the appropriate model configuration (selected from the “configs” directories) and the corresponding pre-trained checkpoint from “ckpts”.
• The loaded model (defined in the pyskl/models folder under subdirectories like cnns and gcns) processes the video data and returns an action recognition result.
• Utilities and visualization modules (from pyskl/utils) format and display the output back in the UI.
• Supporting scripts in “tools” help with training and testing the models if necessary, and “demo” scripts provide usage examples.
Important Architectural Patterns and Principles
• Modular Design: The project is organized into modules (apis, core, models, datasets, utils) to enhance separation of concerns.
• Config-Driven Architecture: The “configs” directories emphasize a highly configurable system where different model setups and pipelines can be interchanged easily.
• Pipeline Processing: The video and skeleton data flow through a pipeline of pre-processing, feature extraction, and final model inference.
• Reusability and Extensibility: The clear separation between model definitions, data handling, and inference APIs allows new models or data sources to be integrated with minimal changes.
Relevant Technologies, Frameworks, and Libraries
• Streamlit: For the web application interface.
• Python & PyTorch: Likely used for model definition, training, and inference.
• Various deep learning architectures and configurations, as indicated by directories like aagcn, ctrgcn, dgstgcn, msg3d, and stgcn.
• Utilities for visualization and environment checks (pyskl/utils).
• Front-End (Streamlit Application)
• Inference API (pyskl/apis)
• Data Processing Pipeline (pyskl/datasets/pipelines)
• Model & Configuration Loader (pyskl/models and configs)
• Checkpoint Repository (ckpts)
• Utilities and Visualization (pyskl/utils)
• Auxiliary Tools and Demos (tools/demo directories)
• User’s video upload → Front-End → Inference API → Data Pipeline → Model Processing → Output Results → Back to Front-End.
• Notations for external technologies (e.g., Streamlit, PyTorch) that are integral to the functioning of the system.