How isshikihugh/hsmr works
Major Components
• Data & Configurations:
• data_inputs: Contains example images and videos used for demo runs.
• data_outputs: Stores the output data (e.g., processed results, rendered skeletons).
• configs: Holds YAML configuration files for datasets, models, callbacks, pipelines, and experimental settings.
• Experiment Scripts:
• exp: Contains entry point scripts (run_demo.py, run_eval.py, run_train.py) that orchestrate the main workflows (demo, evaluation, training).
• Core Libraries (lib):
• body_models: Implements various body and skeleton models, including abstractions, alignment modules, and wrappers for models like SMPL-X and SPIN.
• data: Includes dataset definitions, augmentation, and data pipeline management—with different datasets such as hsmr and skel_hmr2_fashion.
• evaluation: Contains evaluators, metrics, and routines for assessing the reconstruction quality.
• modeling: Encompasses networks (backbones, heads, discriminators), loss functions, callbacks (e.g., for optimization stages like skelify), and training pipelines.
• platform and utils: Include system-level components, configuration utilities, logging, visualization, media handling, and additional helper functions.
• kits: Provides demo and evaluation kits that plugin into the main pipeline.
• Documentation & External Integrations:
• docs: Contains user/documentation tutorials on evaluation, training, optimization, and setup.
• External projects: The README acknowledges several external repositories (e.g., SPIN, SMPLify-X, Detectron2) that are integrated or adapted into the codebase.
Component Relationships & Data Flow
• Input Stage:
• Data ingestion comes through data_inputs. These inputs (images/videos) are preprocessed via relevant dataset modules in lib/data.
• Configuration Stage:
• YAML configs (under configs/) guide the behavior of pipelines, models, data processing, and training – thus acting as the “glue” that binds these components.
• Processing & Model Inference:
• The experimental scripts (in exp/) trigger processes that invoke pipelines defined in lib/modeling/pipelines.
• These pipelines coordinate model execution (using components in lib/modeling/networks, callbacks in lib/modeling/callbacks, and optimizers in lib/modeling/optim) alongside body model reconstruction routines in lib/body_models.
• Evaluation & Visualization:
• Evaluation procedures (in lib/evaluation) compute quality metrics, and visualization utilities (in lib/utils/vis and lib/info) can output results.
• Output Stage:
• Processed outputs are saved into data_outputs and also can trigger demo visualizations (refer to lib/kits/hsmr_demo.py and others).
Architectural Patterns & Design Principles
• Modularity & Separation of Concerns:
• The project separates concerns neatly: configuration management, dataset handling, model definitions, training pipelines, and evaluation are isolated in well-named directories.
• Pipeline Orchestration:
• The use of configuration-driven pipelines allows for easy swapping and tuning of processing stages. This is supported by well-defined “exp” scripts and “configs” that enable reproducibility.
• Extensibility:
• With distinct modules for handling body models, networks, and utilities, the architecture encourages extending or integrating with additional models or datasets.