# hiteshkumar18/orchestration-mea architecture

> Architecture diagram of the GitHub repository hiteshkumar18/orchestration-mea, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hiteshkumar18/orchestration-mea
- Repository: https://github.com/hiteshkumar18/orchestration-mea

## Overview

Orchestration-MEA is a lab control and analysis layer for MEA recordings. A user configures projects and analysis options in the web UI, starts the watcher, and receives independently tracked Network and ActivityScan results. The watcher waits for recording folders to settle, then runs the external MEA-Analysis driver locally or submits Network work to Slurm. Checkpoint and history views expose progress; reporting and NWB conversion support downstream use. Edges reflect README statements and the supplied source index/excerpts; several domain modules are present in the tree but their internal wiring was not sampled.

Main entry points: [`orchestration/api.py`](https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/api.py) (Control API), [`orchestration/watcher.py`](https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/watcher.py) (Recording watcher), [`orchestration/activity_scan.py`](https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/activity_scan.py) (ActivityScan analysis), [`orchestration/static/app.js`](https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/static/app.js) (Control UI)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_control["Lab control"]
  node_ui["Control UI<br/>[app.js]"]
  node_api["Control API<br/>[api.py]"]
  node_accounts["User attribution<br/>[accounts.py]"]
  node_picker["Folder picker<br/>[native_picker.py]"]
end

subgraph group_runtime["Run orchestration"]
  node_watcher["Recording watcher<br/>[watcher.py]"]
  node_schema["Driver options<br/>[driver_schema.py]"]
  node_mea["MEA checkout validation<br/>[mea_repo.py]"]
  node_slurm["Slurm backend<br/>[slurm_backend.py]"]
  node_memory["Job memory guard<br/>[memguard.py]"]
end

subgraph group_analysis["Analysis outputs"]
  node_activity["ActivityScan analysis<br/>[activity_scan.py]"]
  node_trends["Activity trends<br/>[activity_trends.py]"]
  node_explore["Data exploration<br/>[explore.py]"]
  node_nwb["NWB conversion"]
  node_summary["MaxWell summary<br/>[maxwell_summary.py]"]
end

subgraph group_state["Status and reports"]
  node_checkpoint["Checkpoint status<br/>[checkpoints.py]"]
  node_history[("Run history<br/>[history.py]")]
  node_monitor["Run monitoring<br/>[monitor.py]"]
  node_detail["Run details<br/>[rundetail.py]"]
  node_tabular["Tabular exports<br/>[tabular.py]"]
  node_handoff["AI report handoff<br/>[handoff.py]"]
  node_report["Report generation"]
end

subgraph group_integrations["External systems"]
  node_driver["MEA pipeline driver"]
  node_recordings[("Recording folders")]
  node_results[("Analysis outputs")]
end

node_user(("Lab user"))

node_user -->|"configures"| node_ui
node_ui -->|"calls API"| node_api
node_api -->|"controls"| node_watcher
node_api -->|"loads options"| node_schema
node_api -->|"attributes actions"| node_accounts
node_api -->|"offers selection"| node_picker
node_watcher -->|"polls for completion"| node_recordings
node_watcher -->|"validates checkout"| node_mea
node_watcher -->|"builds arguments"| node_schema
node_watcher -->|"launches subprocess"| node_driver
node_watcher -->|"dispatches scan"| node_activity
node_watcher -->|"submits Network jobs"| node_slurm
node_watcher -->|"limits job memory"| node_memory
node_driver -->|"reads recordings"| node_recordings
node_driver -->|"writes results"| node_results
node_activity -->|"reads scan data"| node_recordings
node_activity -->|"writes scan outputs"| node_results
node_slurm -->|"copies results"| node_results
node_api -->|"queries status"| node_checkpoint
node_watcher -->|"records run changes"| node_history
node_watcher -->|"reads checkpoint state"| node_checkpoint
node_api -->|"exposes monitoring"| node_monitor
node_api -->|"serves run details"| node_detail
node_api -->|"serves run history"| node_history
node_api -->|"prepares handoff"| node_handoff
node_handoff -->|"builds report package"| node_report
node_summary -->|"reads recordings"| node_recordings
node_explore -->|"uses tabular helpers"| node_tabular

click node_ui "https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/static/app.js"
click node_api "https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/api.py"
click node_accounts "https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/accounts.py"
click node_picker "https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/native_picker.py"
click node_watcher "https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/watcher.py"
click node_schema "https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/driver_schema.py"
click node_mea "https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/mea_repo.py"
click node_slurm "https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/slurm_backend.py"
click node_memory "https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/memguard.py"
click node_activity "https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/activity_scan.py"
click node_trends "https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/activity_trends.py"
click node_explore "https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/explore.py"
click node_nwb "https://github.com/hiteshkumar18/orchestration-mea/tree/main/orchestration/nwb"
click node_summary "https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/maxwell_summary.py"
click node_checkpoint "https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/checkpoints.py"
click node_history "https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/history.py"
click node_monitor "https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/monitor.py"
click node_detail "https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/rundetail.py"
click node_tabular "https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/tabular.py"
click node_handoff "https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/handoff.py"
click node_report "https://github.com/hiteshkumar18/orchestration-mea/tree/main/orchestration/report"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_ui,node_api,node_accounts,node_picker,node_user toneBlue
class node_watcher,node_schema,node_mea,node_slurm,node_memory toneAmber
class node_activity,node_trends,node_explore,node_nwb,node_summary toneMint
class node_checkpoint,node_history,node_monitor,node_detail,node_tabular,node_handoff,node_report toneRose
class node_driver,node_recordings,node_results toneIndigo
```

## Components

### Lab control

- [Control UI](https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/static/app.js): `orchestration/static/app.js`
- [Control API](https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/api.py): `orchestration/api.py`
- [User attribution](https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/accounts.py): `orchestration/accounts.py`
- [Folder picker](https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/native_picker.py): `orchestration/native_picker.py`

### Run orchestration

- [Recording watcher](https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/watcher.py): `orchestration/watcher.py`
- [Driver options](https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/driver_schema.py): `orchestration/driver_schema.py`
- [MEA checkout validation](https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/mea_repo.py): `orchestration/mea_repo.py`
- [Slurm backend](https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/slurm_backend.py): `orchestration/slurm_backend.py`
- [Job memory guard](https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/memguard.py): `orchestration/memguard.py`

### Analysis outputs

- [ActivityScan analysis](https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/activity_scan.py): `orchestration/activity_scan.py`
- [Activity trends](https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/activity_trends.py): `orchestration/activity_trends.py`
- [Data exploration](https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/explore.py): `orchestration/explore.py`
- [NWB conversion](https://github.com/hiteshkumar18/orchestration-mea/tree/main/orchestration/nwb): `orchestration/nwb`
- [MaxWell summary](https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/maxwell_summary.py): `orchestration/maxwell_summary.py`

### Status and reports

- [Checkpoint status](https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/checkpoints.py): `orchestration/checkpoints.py`
- [Run history](https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/history.py): `orchestration/history.py`
- [Run monitoring](https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/monitor.py): `orchestration/monitor.py`
- [Run details](https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/rundetail.py): `orchestration/rundetail.py`
- [Tabular exports](https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/tabular.py): `orchestration/tabular.py`
- [AI report handoff](https://github.com/hiteshkumar18/orchestration-mea/blob/main/orchestration/handoff.py): `orchestration/handoff.py`
- [Report generation](https://github.com/hiteshkumar18/orchestration-mea/tree/main/orchestration/report): `orchestration/report`

### External systems

- **MEA pipeline driver**
- **Recording folders**
- **Analysis outputs**

### Other components

- **Lab user**

## Connections

- Lab user → Control UI: configures (evidence: `README.md`)
- Control UI → Control API: calls API (evidence: `orchestration/api.py`)
- Control API → Recording watcher: controls (evidence: `orchestration/api.py`)
- Control API → Driver options: loads options (evidence: `orchestration/api.py`)
- Control API → User attribution: attributes actions (evidence: `orchestration/api.py`)
- Control API → Folder picker: offers selection (evidence: `orchestration/api.py`)
- Recording watcher → Recording folders: polls for completion (evidence: `README.md`)
- Recording watcher → MEA checkout validation: validates checkout (evidence: `orchestration/watcher.py`)
- Recording watcher → Driver options: builds arguments (evidence: `orchestration/watcher.py`)
- Recording watcher → MEA pipeline driver: launches subprocess (evidence: `README.md`)
- Recording watcher → ActivityScan analysis: dispatches scan (evidence: `README.md`)
- Recording watcher → Slurm backend: submits Network jobs (evidence: `README.md`)
- Recording watcher → Job memory guard: limits job memory (evidence: `orchestration/watcher.py`)
- MEA pipeline driver → Recording folders: reads recordings (evidence: `README.md`)
- MEA pipeline driver → Analysis outputs: writes results (evidence: `README.md`)
- ActivityScan analysis → Recording folders: reads scan data (evidence: `README.md`)
- ActivityScan analysis → Analysis outputs: writes scan outputs (evidence: `README.md`)
- Slurm backend → Analysis outputs: copies results (evidence: `README.md`)
- Control API → Checkpoint status: queries status (evidence: `orchestration/api.py`)
- Recording watcher → Run history: records run changes (evidence: `orchestration/watcher.py`)
- Recording watcher → Checkpoint status: reads checkpoint state (evidence: `orchestration/watcher.py`)
- Control API → Run monitoring: exposes monitoring (evidence: `orchestration/api.py`)
- Control API → Run details: serves run details (evidence: `orchestration/api.py`)
- Control API → Run history: serves run history (evidence: `orchestration/api.py`)
- Control API → AI report handoff: prepares handoff (evidence: `orchestration/api.py`)
- AI report handoff → Report generation: builds report package (evidence: `orchestration/handoff.py`)
- MaxWell summary → Recording folders: reads recordings (evidence: `orchestration/maxwell_summary.py`)
- Data exploration → Tabular exports: uses tabular helpers (evidence: `orchestration/explore.py`)
