How raquelmotzfeldttirach/imu_and_markers_ik_opensim works
1. Repository purpose
- This is a batch-processing pipeline for human motion analysis with OpenSim, centered on sensor fusion between IMU orientation data and optical marker tracking from webcam, stereocamera, and optionally Vicon reference recordings.
- It is not a service or UI app; the architecture is a set of scripts that preprocess data, run OpenSim workflows, tune parameters, and compute error metrics.
2. Primary data boundary
- Raw inputs live under a recordings/SubjectXX/... structure described in the README.
- The pipeline consumes:
- IMU .txt logs
- Optical .trc marker files from webcam/stereocamera
- Vicon .trc reference files
- Outputs and optimization results are intended for analytics/, making the repo split into raw data, processing scripts, and results.
3. Core biomechanics model asset
- OpenSim/Models/Rajagopal/Rajagopal_2015.osim is the central human musculoskeletal model.
- The large Geometry/ folder contains .vtp meshes for bones and body segments used by the OpenSim model.
- Marker definition files (vicon_markers.xml, webcam_markers.xml, stereocamera_markers.xml) define modality-specific observation sets against the same underlying model.
4. OpenSim/OpenSense ingestion layer
- OpenSense_IMUDataConverter.py converts raw IMU data into OpenSim-friendly orientation inputs.
- OpenSense_CalibrateModel.py calibrates the model using IMU data.
- OpenSense_OrientationTracking.py runs the orientation tracking workflow.
- OpenSense_PostProcessing.py appears to summarize or transform OpenSim outputs after tracking.
- These scripts form the IMU/OpenSense branch of the pipeline.
5. Optical tracking preprocessing layer
- TRCcorrector.py suggests cleanup or normalization of .trc marker trajectories.
- TimeSynchronization.py aligns modalities in time, which is essential before fusing IMU and optical data.
- webcam_ref_frame_correction.py, stereocamera_ref_frame_correction.py, and vicon_ref_frame_correction.py handle coordinate-frame alignment so the different capture systems share a common spatial convention.
- This preprocessing boundary is what makes heterogeneous recordings comparable.
6. Model scaling and marker setup
- modelScalingWebcam.py, modelScalingStereocamera.py, and modelScalingVicon.py indicate separate scaling workflows per capture modality.
- scaling_setup_template*.xml files are OpenSim setup templates used to parameterize scaling runs for each source.
- model_add_markers.py likely inserts or configures model markers to match the selected recording modality.
- This layer bridges raw measurements to a subject-specific OpenSim model.
7. Sensor fusion and weighted IK
- SensorFusion.py, SensorFusion_automatic.py, and SensorFusion_setWeights_AST.py are the main fusion/optimization scripts.
- The README explicitly says the project performs weighted inverse kinematics, adjusting sensor weights to minimize a defined cost function.
- This is the central algorithmic subsystem: it combines IMU orientation constraints with optical position constraints rather than treating either as fully independent.
8. Optimization loop
- run_optimization.py and WeightsTuningModule.py suggest an iterative search over weight settings.
- The README lists optuna and cma as dependencies, so the weight tuning is likely driven by numerical optimization rather than manual configuration.
- run_ik_and_rmse_AST.py ties together IK execution and objective evaluation, likely to support automated tuning.
9. Evaluation and diagnostics
- RMSECalculationModuleCameraAlone.py and run_rmse_calculation.py compute error metrics, likely against Vicon or another reference solution.
- ViconIK.py provides a Vicon-only inverse kinematics baseline.
- diff_check_mot.py indicates file-level comparison of OpenSim .mot outputs, useful for regression checks or optimization scoring.
10. Automation entrypoint
- script.sh is the main user-facing orchestration entrypoint described in the README.
- It likely prompts for a subject ID and dispatches the appropriate preprocessing, scaling, fusion, IK, and evaluation scripts.
- The architecture is therefore a scripted pipeline rather than a single monolithic executable.
11. Dependencies and runtime context
- The repo is designed for Linux with a Conda environment and the OpenSim package installed.
- simbody, numpy, pandas, cma, and optuna materially shape the architecture: OpenSim provides biomechanical simulation/IK, while Python libraries handle data wrangling and optimization.
- The repo’s runtime boundary is an analysis workstation, not a deployed cloud system.
12. Overall data flow
- A typical flow is: raw recordings → modality-specific cleanup and frame alignment → model scaling and marker setup → OpenSim/OpenSense calibration or IK → sensor-fusion weight optimization → RMSE/diagnostic output → analytics artifacts.
- The key architectural distinction is between input normalization, OpenSim model execution, and result scoring/optimization.
Main entry points: script.sh script.sh · SensorFusion.py Sensor fusion · OpenSense_IMUDataConverter.py IMU convert · run_optimization.py Optimize weights
Components 26
Raw data
Recorded IMU and optical inputs plus reference captures.
- Recordings · Subject recordings directory containing IMU logs and optical marker files.
- TRC data · Webcam, stereocamera, and Vicon marker trajectories.
- IMU logs · Raw inertial sensor logs used for OpenSense conversion.
OpenSim model
Biomechanics model assets and modality-specific marker definitions.
- Rajagopal model · Central musculoskeletal model used across scaling, IK, and tracking.
OpenSim/Models/Rajagopal/Rajagopal_2015.osim - Geometry · Body-segment meshes referenced by the OpenSim model.
OpenSim/Models/Rajagopal/Geometry - Markers · Modality-specific observation sets for model fitting.
Processing pipeline
Preprocessing, scaling, OpenSense, fusion, and optimization scripts.
- script.sh · Main entrypoint that dispatches the batch pipeline.
script.sh - IMU convert · Converts raw IMU logs into OpenSim-friendly orientation inputs.
OpenSim/OpenSense_IMUDataConverter.py - Calibrate model · Calibrates the model using IMU data before tracking.
OpenSim/OpenSense_CalibrateModel.py - Orientation tracking · Runs OpenSense orientation tracking against the calibrated model.
OpenSim/OpenSense_OrientationTracking.py - Post process · Summarizes or transforms tracking outputs for downstream use.
OpenSim/OpenSense_PostProcessing.py - TRC corrector · Normalizes or cleans marker trajectories before scaling and fusion.
OpenSim/TRCcorrector.py - Time sync · Aligns IMU and optical recordings in time.
OpenSim/TimeSynchronization.py - Frame correction · Aligns webcam, stereocamera, and Vicon reference frames to a common convention.
- Model scaling · Modality-specific scaling workflows that adapt the model to each recording source.
- Add markers · Configures or inserts markers to match the selected modality.
OpenSim/model_add_markers.py - Sensor fusion · Combines IMU orientation constraints with optical position constraints.
OpenSim/SensorFusion.py - Auto fusion · Automated sensor-fusion workflow for batch execution.
OpenSim/SensorFusion_automatic.py - Weight setup · Defines or updates sensor weights for the fusion cost function.
OpenSim/SensorFusion_setWeights_AST.py - Optimize weights · Iterative weight tuning using numerical optimizers.
OpenSim/run_optimization.py - Tuning module · Shared logic for exploring and scoring weight configurations.
OpenSim/WeightsTuningModule.py - IK + RMSE · Runs inverse kinematics and evaluates objective error for tuning.
OpenSim/run_ik_and_rmse_AST.py
Evaluation
Error metrics, baselines, and output comparison.
- RMSE calc · Computes error metrics, likely against reference motion.
OpenSim/run_rmse_calculation.py - Camera RMSE · Computes RMSE for camera-only comparisons or baselines.
OpenSim/RMSECalculationModuleCameraAlone.py - Vicon IK · Reference inverse-kinematics workflow using Vicon data.
OpenSim/ViconIK.py - Diff check · Compares OpenSim .mot outputs for consistency or scoring.
OpenSim/diff_check_mot.py
Connections 31
- script.sh → IMU convert: dispatches · Starts the IMU/OpenSense branch.
- script.sh → TRC corrector: dispatches · Starts optical preprocessing.
- script.sh → Model scaling: dispatches · Runs modality-specific scaling.
- script.sh → Sensor fusion: dispatches · Launches fusion and IK workflow.
- script.sh → RMSE calc: dispatches · Triggers evaluation outputs.
- Recordings → IMU logs: contains · Raw subject recordings include IMU logs.
- Recordings → TRC data: contains · Raw subject recordings include marker trajectories.
- IMU logs → IMU convert: feeds · IMU logs are converted for OpenSim.
- IMU convert → Calibrate model: prepares · Converted orientations feed model calibration.
- TRC data → TRC corrector: feeds · Marker trajectories are cleaned before use.
- TRC corrector → Time sync: aligns · Cleaned optical data is time-aligned.
- IMU convert → Time sync: aligns · IMU data is synchronized with optical streams.
- Time sync → Frame correction: normalizes · Aligned streams share a common spatial convention.
- Frame correction → Model scaling: prepares · Frame-aligned data supports modality-specific scaling.
- Rajagopal model → Model scaling: basis · Scaling adapts the core musculoskeletal model.
- Geometry → Rajagopal model: supports · Meshes provide model visualization and anatomy context.
- Markers → Add markers: defines · Marker sets drive modality-specific setup.
- Model scaling → Add markers: produces · Scaled subject model is prepared for marker setup.
- Add markers → Sensor fusion: configures · Marker-configured model enters fusion.
- Calibrate model → Orientation tracking: enables · Calibrated model is tracked in OpenSense.
- Orientation tracking → Post process: outputs · Tracking results are summarized downstream.
- Sensor fusion → Weight setup: uses · Fusion depends on sensor weight definitions.
- Weight setup → Optimize weights: tuned by · Weights are explored by optimization routines.
- Tuning module → Optimize weights: supports · Shared tuning logic drives the search loop.
- Optimize weights → IK + RMSE: evaluates · Candidate weights are scored via IK and RMSE.
- IK + RMSE → RMSE calc: computes · Evaluation writes RMSE metrics.
- IK + RMSE → Camera RMSE: computes · Camera-only error metrics can be produced.
- Vicon IK → RMSE calc: baseline for · Vicon IK provides a reference for error analysis.
- Diff check → Optimize weights: checks · Output comparisons can support regression or scoring.
- Auto fusion → Sensor fusion: automates · Batch fusion wrapper around the core fusion script.
- Sensor fusion → Orientation tracking: feeds · Fusion results inform downstream motion tracking.