# arnavdhole/kennedy-mouse-pipeline architecture

> Architecture diagram of the GitHub repository arnavdhole/kennedy-mouse-pipeline, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/arnavdhole/kennedy-mouse-pipeline
- Repository: https://github.com/arnavdhole/kennedy-mouse-pipeline

## Overview

This repository provides stage-one geometric preprocessing for mouse-pose datasets. The user workflow is to load Parquet data, apply dataset-specific keypoint choices and filtering, establish a scale and canonical reference, then scale, center, and rotate each frame into normalized poses for downstream models. Diagnostics include alignment plots and evaluation metrics saved as JSON. Dataset configs and notebooks support lab-specific variations. The README describes a later autoencoder stage, but its implementation is outside this repository; notebook internals were not sampled.

Main entry points: [`notebooks`](https://github.com/arnavdhole/kennedy-mouse-pipeline/tree/main/notebooks) (Run notebooks), [`pipeline/pipeline.py`](https://github.com/arnavdhole/kennedy-mouse-pipeline/blob/main/pipeline/pipeline.py) (Frame pipeline), [`pipeline/preprocessing.py`](https://github.com/arnavdhole/kennedy-mouse-pipeline/blob/main/pipeline/preprocessing.py) (Scale and center), [`pipeline/reference_pose.py`](https://github.com/arnavdhole/kennedy-mouse-pipeline/blob/main/pipeline/reference_pose.py) (Reference pose)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_dataset["Dataset setup"]
  node_configs["Dataset configs"]
  node_notebooks["Run notebooks"]
  node_loader["Parquet loader<br/>[data_loader.py]"]
  node_filtering["Dataset filtering"]
end

subgraph group_geometry["Pose geometry"]
  node_preprocess["Scale and center<br/>[preprocessing.py]"]
  node_reference["Reference pose<br/>[reference_pose.py]"]
  node_runner["Frame pipeline<br/>[pipeline.py]"]
  node_rotation["Pose rotation<br/>[rotation.py]"]
  node_arrays["Array conversion<br/>[pipeline.py]"]
end

subgraph group_quality["Quality outputs"]
  node_validation["Alignment diagnostics<br/>[validation.py]"]
  node_evaluation["Quality metrics<br/>[evaluation.py]"]
  node_results[("Metric JSON files")]
end

node_user(("Analyst"))
node_downstream["Downstream models"]
node_autoencoder["Stage-two autoencoder"]

node_user -->|"runs"| node_notebooks
node_notebooks -->|"loads settings"| node_configs
node_notebooks -->|"loads data"| node_loader
node_notebooks -->|"filters frames"| node_filtering
node_loader -->|"provides rows"| node_filtering
node_notebooks -->|"computes scale"| node_preprocess
node_notebooks -->|"builds reference"| node_reference
node_notebooks -->|"runs frames"| node_runner
node_runner -->|"scales and centers"| node_preprocess
node_runner -->|"chooses rotation"| node_rotation
node_reference -->|"scales and centers"| node_preprocess
node_reference -->|"bootstraps pose"| node_rotation
node_runner -->|"converts frames"| node_arrays
node_notebooks -->|"checks alignment"| node_validation
node_validation -->|"recenters frames"| node_preprocess
node_validation -->|"measures residuals"| node_rotation
node_evaluation -->|"saves metrics"| node_results
node_arrays -->|"feeds poses"| node_downstream
node_downstream -.->|"includes stage two"| node_autoencoder

click node_configs "https://github.com/arnavdhole/kennedy-mouse-pipeline/tree/main/configs"
click node_notebooks "https://github.com/arnavdhole/kennedy-mouse-pipeline/tree/main/notebooks"
click node_loader "https://github.com/arnavdhole/kennedy-mouse-pipeline/blob/main/pipeline/data_loader.py"
click node_filtering "https://github.com/arnavdhole/kennedy-mouse-pipeline/tree/main/notebooks"
click node_preprocess "https://github.com/arnavdhole/kennedy-mouse-pipeline/blob/main/pipeline/preprocessing.py"
click node_reference "https://github.com/arnavdhole/kennedy-mouse-pipeline/blob/main/pipeline/reference_pose.py"
click node_runner "https://github.com/arnavdhole/kennedy-mouse-pipeline/blob/main/pipeline/pipeline.py"
click node_rotation "https://github.com/arnavdhole/kennedy-mouse-pipeline/blob/main/pipeline/rotation.py"
click node_arrays "https://github.com/arnavdhole/kennedy-mouse-pipeline/blob/main/pipeline/pipeline.py"
click node_validation "https://github.com/arnavdhole/kennedy-mouse-pipeline/blob/main/pipeline/validation.py"
click node_evaluation "https://github.com/arnavdhole/kennedy-mouse-pipeline/blob/main/pipeline/evaluation.py"
click node_results "https://github.com/arnavdhole/kennedy-mouse-pipeline/tree/main/results"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_configs,node_notebooks,node_loader,node_filtering toneBlue
class node_preprocess,node_reference,node_runner,node_rotation,node_arrays toneAmber
class node_validation,node_evaluation,node_results toneMint
class node_user,node_downstream,node_autoencoder toneIndigo
```

## Components

### Dataset setup

- [Dataset configs](https://github.com/arnavdhole/kennedy-mouse-pipeline/tree/main/configs): `configs`
- [Run notebooks](https://github.com/arnavdhole/kennedy-mouse-pipeline/tree/main/notebooks): `notebooks`
- [Parquet loader](https://github.com/arnavdhole/kennedy-mouse-pipeline/blob/main/pipeline/data_loader.py): `pipeline/data_loader.py`
- [Dataset filtering](https://github.com/arnavdhole/kennedy-mouse-pipeline/tree/main/notebooks): `notebooks`

### Pose geometry

- [Scale and center](https://github.com/arnavdhole/kennedy-mouse-pipeline/blob/main/pipeline/preprocessing.py): `pipeline/preprocessing.py`
- [Reference pose](https://github.com/arnavdhole/kennedy-mouse-pipeline/blob/main/pipeline/reference_pose.py): `pipeline/reference_pose.py`
- [Frame pipeline](https://github.com/arnavdhole/kennedy-mouse-pipeline/blob/main/pipeline/pipeline.py): `pipeline/pipeline.py`
- [Pose rotation](https://github.com/arnavdhole/kennedy-mouse-pipeline/blob/main/pipeline/rotation.py): `pipeline/rotation.py`
- [Array conversion](https://github.com/arnavdhole/kennedy-mouse-pipeline/blob/main/pipeline/pipeline.py): `pipeline/pipeline.py`

### Quality outputs

- [Alignment diagnostics](https://github.com/arnavdhole/kennedy-mouse-pipeline/blob/main/pipeline/validation.py): `pipeline/validation.py`
- [Quality metrics](https://github.com/arnavdhole/kennedy-mouse-pipeline/blob/main/pipeline/evaluation.py): `pipeline/evaluation.py`
- [Metric JSON files](https://github.com/arnavdhole/kennedy-mouse-pipeline/tree/main/results): `results`

### Other components

- **Analyst**
- **Downstream models**
- **Stage-two autoencoder**

## Connections

- Analyst → Run notebooks: runs (evidence: `README.md`)
- Run notebooks → Dataset configs: loads settings (evidence: `README.md`)
- Run notebooks → Parquet loader: loads data (evidence: `README.md`)
- Run notebooks → Dataset filtering: filters frames (evidence: `README.md`)
- Parquet loader → Dataset filtering: provides rows (evidence: `README.md`)
- Run notebooks → Scale and center: computes scale (evidence: `README.md`)
- Run notebooks → Reference pose: builds reference (evidence: `README.md`)
- Run notebooks → Frame pipeline: runs frames (evidence: `README.md`)
- Frame pipeline → Scale and center: scales and centers (evidence: `pipeline/pipeline.py`)
- Frame pipeline → Pose rotation: chooses rotation (evidence: `pipeline/pipeline.py`)
- Reference pose → Scale and center: scales and centers (evidence: `pipeline/reference_pose.py`)
- Reference pose → Pose rotation: bootstraps pose (evidence: `pipeline/reference_pose.py`)
- Frame pipeline → Array conversion: converts frames (evidence: `pipeline/pipeline.py`)
- Run notebooks → Alignment diagnostics: checks alignment (evidence: `README.md`)
- Alignment diagnostics → Scale and center: recenters frames (evidence: `pipeline/validation.py`)
- Alignment diagnostics → Pose rotation: measures residuals (evidence: `pipeline/validation.py`)
- Quality metrics → Metric JSON files: saves metrics (evidence: `pipeline/evaluation.py`)
- Array conversion → Downstream models: feeds poses (evidence: `README.md`)
- Downstream models → Stage-two autoencoder: includes stage two (evidence: `README.md`)