# ybhbingo/mmfi_dataset architecture

> Architecture diagram of the GitHub repository ybhbingo/mmfi_dataset, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ybhbingo/mmfi_dataset
- Repository: https://github.com/ybhbingo/mmfi_dataset

## Overview

1. **Repository purpose**
   - This is a dataset toolbox for MM-Fi, not a full training framework.
   - Its main job is to turn a disk-backed multimodal dataset plus a YAML config into PyTorch-ready datasets and dataloaders, with evaluation support.

2. **Top-level entry points**
   - `example.py` is the usage example and integration template.
   - `config.yaml` is the main control plane for dataset selection, split policy, modality choice, and dataloader settings.
   - `README.md` defines the expected dataset layout and the supported configuration semantics.

3. **Core library boundary**
   - The reusable logic lives under `mmfi_lib/`.
   - `mmfi_lib/mmfi.py` is the dataset-facing API surface; the README shows it exposing `make_dataset` and `make_dataloader`.
   - `mmfi_lib/evaluate.py` is the evaluation subsystem, likely responsible for metric computation on model outputs versus MM-Fi annotations.

4. **Data source and storage model**
   - The system assumes a hierarchical on-disk dataset rooted at `${DATASET_ROOT}` with paths organized by environment, subject, and action:
     - environment group `E01`–`E04`
     - subject `S01`–`S10`
     - action `A01`–`A27`
     - modality subfolders such as `rgb`, `mmwave`, `wifi-csi`, `depth`, `lidar`, `infra1`, `infra2`
   - This filesystem structure is the primary data boundary: the library reads local files directly rather than calling a service.

5. **Configuration-driven behavior**
   - `modality` controls whether a single modality or a fused multimodal sample is loaded.
   - `data_unit` switches between per-frame and per-sequence sampling, which changes the granularity of each dataset item.
   - `protocol` filters which action classes are in scope: daily, rehabilitation, or all actions.
   - `split` selects a predefined split or a custom manual split for train/validation or train/test partitioning.
   - `train_loader` and `validation_loader` expose standard PyTorch dataloader knobs without hardcoding them into the library.

6. **Dataset construction flow**
   - The typical flow is:
     - load YAML config
     - call `make_dataset(dataset_root, config)`
     - receive train and validation dataset objects
   - This means the library is responsible for:
     - interpreting the split definition
     - filtering by protocol and modality
     - mapping disk files to sample records
     - producing dataset objects that can be wrapped by PyTorch loaders

7. **Batching and runtime integration**
   - `make_dataloader(...)` wraps the dataset objects into PyTorch dataloaders.
   - The example seeds a Torch RNG and passes the same generator into both loaders, indicating reproducible shuffling / sampling is part of the intended runtime behavior.
   - The repo therefore sits at the boundary between raw multimodal files and model training code.

8. **Annotation and label handling**
   - MM-Fi includes pose and action annotations, but the toolbox is oriented around dataset access rather than annotation generation.
   - The README notes that some RGB modalities are privacy-restricted and replaced with extracted keypoints, which affects how the loader should treat modality availability and feature format.
   - The action label taxonomy is fixed at 27 categories and is used to define protocol filtering.

9. **Evaluation subsystem**
   - `mmfi_lib/evaluate.py` is a separate concern from loading.
   - Architecturally, this suggests evaluation is decoupled from dataset construction so that downstream projects can reuse the metrics without depending on the full data-loading path.

10. **External dependencies that shape the design**
   - The toolbox is built around Python, PyTorch, YAML, NumPy/SciPy, and OpenCV.
   - PyTorch drives dataset/dataloader abstractions.
   - YAML drives configuration.
   - NumPy/SciPy/OpenCV imply preprocessing and file-format handling inside the library, likely for numerical signal modalities and image-like inputs.

11. **What is intentionally not in scope**
   - There is no evidence of a training loop, model definition, or experiment manager in this repository.
   - The repo is focused on data access, split management, and evaluation hooks, leaving modeling and optimization to downstream projects.

12. **How to diagram it**
   - Draw three main blocks:
     - disk dataset tree plus config file as inputs
     - `mmfi_lib/mmfi.py` as the loader/factory layer producing datasets and dataloaders
     - `mmfi_lib/evaluate.py` as a separate metrics/evaluation layer
   - Place `example.py` as a consumer of the library, showing the intended integration path into external model code.

Main entry points: [`mmfi_lib/mmfi.py`](https://github.com/ybhbingo/mmfi_dataset/blob/main/mmfi_lib/mmfi.py) (MM-Fi API), [`example.py`](https://github.com/ybhbingo/mmfi_dataset/blob/main/example.py) (Example), [`config.yaml`](https://github.com/ybhbingo/mmfi_dataset/blob/main/config.yaml) (Config), [`mmfi_lib/evaluate.py`](https://github.com/ybhbingo/mmfi_dataset/blob/main/mmfi_lib/evaluate.py) (Evaluate)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_inputs["Inputs"]
  node_dataset_root[("Dataset root<br/>disk tree")]
  node_dataset_layout["Layout spec<br/>filesystem schema<br/>[README.md]"]
  node_config_yaml["Config<br/>YAML control plane<br/>[config.yaml]"]
  node_readme["README<br/>usage spec<br/>[README.md]"]
end

subgraph group_core["Core lib"]
  node_mmfi_api["MM-Fi API<br/>dataset factory<br/>[mmfi.py]"]
  node_eval_lib["Evaluate<br/>metrics module<br/>[evaluate.py]"]
  node_protocol_filter{{"Protocol<br/>class filter"}}
  node_split_planner{{"Split policy<br/>partition logic"}}
  node_modality_reader{{"Modality load<br/>sample builder"}}
  node_annotation_map{{"Labels<br/>taxonomy map"}}
  node_train_dataset["Train set<br/>PyTorch dataset"]
  node_val_dataset["Val set<br/>PyTorch dataset"]
end

subgraph group_runtime["Runtime use"]
  node_example_py["Example<br/>integration template<br/>[example.py]"]
  node_train_loader["Train loader<br/>PyTorch dataloader"]
  node_val_loader["Val loader<br/>PyTorch dataloader"]
  node_downstream_model["Model code<br/>external consumer"]
  node_metrics_out(("Metrics<br/>evaluation output"))
end

node_readme -->|"documents"| node_dataset_layout
node_readme -->|"documents"| node_config_yaml
node_example_py -->|"loads"| node_config_yaml
node_example_py -->|"calls"| node_mmfi_api
node_dataset_root -->|"reads"| node_mmfi_api
node_config_yaml -->|"controls"| node_mmfi_api
node_mmfi_api -->|"applies"| node_protocol_filter
node_mmfi_api -->|"applies"| node_split_planner
node_mmfi_api -->|"builds"| node_modality_reader
node_mmfi_api -->|"uses"| node_annotation_map
node_protocol_filter -->|"feeds"| node_train_dataset
node_split_planner -->|"produces"| node_train_dataset
node_split_planner -->|"produces"| node_val_dataset
node_modality_reader -->|"materializes"| node_train_dataset
node_modality_reader -->|"materializes"| node_val_dataset
node_train_dataset -->|"wraps"| node_train_loader
node_val_dataset -->|"wraps"| node_val_loader
node_example_py -->|"uses"| node_train_loader
node_example_py -->|"uses"| node_val_loader
node_train_loader -->|"feeds"| node_downstream_model
node_val_loader -->|"feeds"| node_downstream_model
node_example_py -->|"imports"| node_eval_lib
node_downstream_model -->|"evaluates"| node_eval_lib
node_eval_lib -->|"emits"| node_metrics_out

click node_dataset_layout "https://github.com/ybhbingo/mmfi_dataset/blob/main/README.md"
click node_config_yaml "https://github.com/ybhbingo/mmfi_dataset/blob/main/config.yaml"
click node_readme "https://github.com/ybhbingo/mmfi_dataset/blob/main/README.md"
click node_example_py "https://github.com/ybhbingo/mmfi_dataset/blob/main/example.py"
click node_mmfi_api "https://github.com/ybhbingo/mmfi_dataset/blob/main/mmfi_lib/mmfi.py"
click node_eval_lib "https://github.com/ybhbingo/mmfi_dataset/blob/main/mmfi_lib/evaluate.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_dataset_root,node_dataset_layout,node_config_yaml,node_readme toneBlue
class node_mmfi_api,node_eval_lib,node_protocol_filter,node_split_planner,node_modality_reader,node_annotation_map,node_train_dataset,node_val_dataset toneAmber
class node_example_py,node_train_loader,node_val_loader,node_downstream_model,node_metrics_out toneMint
```

## Components

### Inputs

Repository entry points and on-disk dataset shape

- **Dataset root**: disk tree · Hierarchical MM-Fi files under environment, subject, action, and modality folders
- [Layout spec](https://github.com/ybhbingo/mmfi_dataset/blob/main/README.md): filesystem schema · E01-E04, S01-S10, A01-A27, and modality subfolders · `README.md`
- [Config](https://github.com/ybhbingo/mmfi_dataset/blob/main/config.yaml): YAML control plane · Selects modality, data unit, protocol, split, and loader settings · `config.yaml`
- [README](https://github.com/ybhbingo/mmfi_dataset/blob/main/README.md): usage spec · Documents dataset organization and configuration semantics · `README.md`

### Core lib

Dataset factory and evaluation logic

- [MM-Fi API](https://github.com/ybhbingo/mmfi_dataset/blob/main/mmfi_lib/mmfi.py): dataset factory · Public surface for make_dataset and make_dataloader · `mmfi_lib/mmfi.py`
- [Evaluate](https://github.com/ybhbingo/mmfi_dataset/blob/main/mmfi_lib/evaluate.py): metrics module · Evaluation logic for predictions against MM-Fi annotations · `mmfi_lib/evaluate.py`
- **Protocol**: class filter · Limits scope to daily, rehabilitation, or all actions
- **Split policy**: partition logic · Resolves predefined or manual train/validation or train/test splits
- **Modality load**: sample builder · Loads single or fused multimodal samples from disk
- **Labels**: taxonomy map · Maps fixed action annotations and pose-related metadata into samples
- **Train set**: PyTorch dataset · Training dataset object returned by the factory
- **Val set**: PyTorch dataset · Validation or test dataset object returned by the factory

### Runtime use

Example integration and downstream consumption

- [Example](https://github.com/ybhbingo/mmfi_dataset/blob/main/example.py): integration template · Shows dataset construction, loader creation, and seeding · `example.py`
- **Train loader**: PyTorch dataloader · Batches training samples with configured loader knobs
- **Val loader**: PyTorch dataloader · Batches validation samples with configured loader knobs
- **Model code**: external consumer · Downstream training or inference code that consumes loaders
- **Metrics**: evaluation output · Computed scores from evaluator over predictions and annotations

## Connections

- README → Layout spec: documents · Explains the expected directory hierarchy
- README → Config: documents · Explains supported configuration semantics
- Example → Config: loads · Reads YAML settings before building datasets
- Example → MM-Fi API: calls · Invokes dataset and dataloader factory functions
- Dataset root → MM-Fi API: reads · MM-Fi files are loaded directly from disk
- Config → MM-Fi API: controls · Drives modality, protocol, split, and loader behavior
- MM-Fi API → Protocol: applies · Restricts action categories by protocol
- MM-Fi API → Split policy: applies · Chooses predefined or manual partitions
- MM-Fi API → Modality load: builds · Constructs sample records from one or more modalities
- MM-Fi API → Labels: uses · Attaches labels and annotation-derived targets
- Protocol → Train set: feeds · Filtered classes flow into the training dataset
- Split policy → Train set: produces · Creates the training partition
- Split policy → Val set: produces · Creates the validation or test partition
- Modality load → Train set: materializes · Backs training samples with loaded features
- Modality load → Val set: materializes · Backs validation samples with loaded features
- Train set → Train loader: wraps · Training dataset is wrapped by a PyTorch dataloader
- Val set → Val loader: wraps · Validation dataset is wrapped by a PyTorch dataloader
- Example → Train loader: uses · Example shows runtime integration of the training loader
- Example → Val loader: uses · Example shows runtime integration of the validation loader
- Train loader → Model code: feeds · Batches are consumed by external model code
- Val loader → Model code: feeds · Validation batches are consumed by external model code
- Example → Evaluate: imports · Evaluation is used as a separate subsystem
- Model code → Evaluate: evaluates · Predictions are scored against MM-Fi annotations
- Evaluate → Metrics: emits · Produces evaluation metrics and summaries