# firnenemerald/junglab architecture

> Architecture diagram of the GitHub repository firnenemerald/junglab, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/firnenemerald/junglab
- Repository: https://github.com/firnenemerald/junglab

## Overview

##### Project type and purpose
- This is not a traditional client–server or web-app system. It is a codebase for neuroscience experiment analysis (Jung Lab). It contains multiple analysis “pipelines” or modules (e.g., CLOI, ChAT, PV Analysis, behavior, Inscopix, DeepLabCut) that ingest raw data, preprocess it, run domain-specific computations, and produce results/plots.

##### File structure
- Top-level directories each represent a pipeline or experiment type: Behavior, CLOI, ChAT, Cluster, Inscopix, PVEPM, PVOF, PVYM, PV_Analysis.
- File finders and readers (e.g., filepathfinder.m, filereader_video.m, filereader_doric.m)
- Preprocessing steps (sync_dlc.m, smooth_dlc.m, fixerror_doric.m, kalmanfilter_h.m)
- Main orchestration scripts (mainscript.m, CLOI_MainScript.m, PV_YM_MainAnalysis.m)
- Analysis functions (clustering, SVM, PCA, correlation, behavior clustering)
- Plotting functions (PlotDLC, PlotCustomBar, plotCorrVsDistance)
- Data folders holding intermediate MAT or CSV dumps (data/*.mat, Verification/*.csv)
- Dev or Verification subdirectories for experiments and testing.
- Mixed MATLAB (.m) and Python (.py) code; external dependencies like DeepLabCut, Doric signal acquisition, Inscopix macros.

##### README for additional insights

a. Main components
- Data Sources (video files, Doric photometry, DeepLabCut outputs, Inscopix imaging)
- Shared Preprocessing Layer (file finder, reader, error correction, synchronization, smoothing, Kalman filters)
- Analysis Engine (clustering, SVM, PCA, correlation, behavioral metrics)
- Visualization/Reporting (plot functions, CSV exports)
- External Tools & Libraries (MATLAB runtime, Python environment, DeepLabCut library, Doric SDK, Inscopix macro)
b. Relationships and interactions
- From Preprocessing into each Pipeline Module’s Main Script
- Within each module: call sequence filefinder → filereader → preprocess → analysis functions → plot/export
- Dependencies on external toolboxes wherever applicable (DeepLabCut, Doric SDK)
c. Architectural patterns or design principles
- Modular pipeline design: each experiment type is isolated in its own namespace
- Reuse of common preprocessing functions across pipelines
- Clear separation of concerns: ingestion, cleaning, analysis, visualization
d. Technologies, frameworks, libraries
- MATLAB scripts (.m), MATLAB toolboxes (Signal Processing, Statistics)
- Python scripts for some real-time closed-loop tasks and DeepLabCut hooks
- DeepLabCut (pose estimation)
- Doric photometry hardware interface
- Inscopix imaging macros

##### Elements
- Blue for ingestion/preprocessing
- Green for analysis engines
- Orange for visualization
- Gray for external services/libraries

##### Detailed capture of architectural elements
- For each pipeline, call out its unique analysis functions (e.g., CLOI_GetMv, CHat_PlotSignal, PV_BatchProcess)
- CI/test subfolders under Dev/Verification showing where experimental code and validation data reside

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Sources
    subgraph "Data Sources" 
        direction TB
        DS_Video["Video Files & DLC Outputs"]:::external
        DS_Doric["Doric Photometry"]:::external
        DS_Inscopix["Inscopix Imaging"]:::external
    end

    %% Shared Preprocessing
    subgraph "Shared Preprocessing" 
        direction TB
        FileFinder["File Path Discovery"]:::pre
        FileReader["File Reader"]:::pre
        ErrorFixer["Error Correction"]:::pre
        Sync["DLC Synchronization"]:::pre
        Smoothing["DLC Smoothing"]:::pre
        Kalman["Kalman Filtering"]:::pre
    end

    DS_Video -->|".mat/.csv"| FileFinder
    DS_Doric -->|"raw signals"| FileFinder
    DS_Inscopix -->|"raw imaging"| FileFinder

    FileFinder --> FileReader
    FileReader --> ErrorFixer
    ErrorFixer --> Sync
    Sync --> Smoothing
    Smoothing --> Kalman

    %% Pipeline Modules
    subgraph "CLOI Pipeline" 
        direction TB
        CLOI_Main["CLOI_MainScript.m"]:::main
        CLOI_GetMv["CLOI_GetMv"]:::analysis
        CLOI_GetDLC["CLOI_GetDLC"]:::analysis
        CLOI_PlotDLC["CLOI_PlotDLC"]:::viz
        PlotCustomBar_C["PlotCustomBar"]:::viz
    end

    subgraph "ChAT Pipeline"
        direction TB
        ChAT_Sync["GetSyncTime & GetExpDetail"]:::pre
        ChAT_GetData["ChAT_GetData"]:::main
        ChAT_PlotSignal["ChAT_PlotSignal"]:::viz
        ChAT_PlotCorr["ChAT_PlotCorr"]:::viz
    end

    subgraph "Behavior Pipeline"
        direction TB
        Behavior_Main["behavioranalysis.m"]:::main
    end

    subgraph "Cluster Analysis"
        direction TB
        PCA["PCAtrajectory"]:::analysis
        NeuralSignal["getNeuralSignal"]:::analysis
        CorrDist["plotCorrVsDistance"]:::viz
        CorrSmooth["plotCorrVsDistanceSmooth"]:::viz
        BarScatter["plotBarWithScatter"]:::viz
    end

    subgraph "Inscopix Pipeline"
        direction TB
        Inscopix_Crop["inscopix_crop.m"]:::pre
    end

    subgraph "PVEPM Pipeline"
        direction TB
        PVEPM_Main["mainscript.m"]:::main
        SVM["da_SVM"]:::analysis
        PlotOnset["plot_onset"]:::viz
        PlotNorm["plot_normhmap"]:::viz
    end

    subgraph "PVOF Pipeline"
        direction TB
        PVOF_Main["mainscript_doricof.m"]:::main
        BehavClust["behavclustering"]:::analysis
        FnBehavOF["fn_behavcluster_OF_v2"]:::analysis
        OnsetPlot["onsetplott"]:::viz
        FPlotRG1["f_plotrg"]:::viz
    end

    subgraph "PVYM Pipeline"
        direction TB
        PVYM_Main["PV_YM_MainAnalysis.m"]:::main
        PV_GetDLC["PV_GetDLCData"]:::pre
        PV_GetDoric["PV_GetDoricSignal"]:::pre
        PV_Sync["PV_SyncData"]:::pre
        PV_CorrAuto["PV_CorrectAutomatic"]:::analysis
        PV_CorrMan["PV_CorrectManual"]:::analysis
    end

    subgraph "PV_Analysis Pipeline"
        direction TB
        PVAnalysis_Main["mainscript_doricof.m"]:::main
        PVAF_FileFinder["filepathfinder"]:::pre
        PVAF_Sync["sync_dlc"]:::pre
        PVAF_Smooth["smooth_dlc"]:::pre
        PVAF_ErrorFix["fixerror_doric"]:::pre
        PVAF_Kalman["kalmanfilter_h"]:::pre
        PVAF_PlotRG["f_plotrg"]:::viz
        PVAF_BehavOF["fn_behavcluster_OF_v2"]:::analysis
    end

    %% Connections to Pipelines
    Kalman --> CLOI_Main
    Kalman --> ChAT_Sync
    Kalman --> Behavior_Main
    Kalman --> PCA
    Kalman --> Inscopix_Crop
    Kalman --> PVEPM_Main
    Kalman --> PVOF_Main
    Kalman --> PVYM_Main
    Kalman --> PVAnalysis_Main

    CLOI_Main --> CLOI_GetMv --> CLOI_PlotDLC
    CLOI_Main --> CLOI_GetDLC --> PlotCustomBar_C

    ChAT_Sync --> ChAT_GetData --> ChAT_PlotSignal
    ChAT_GetData --> ChAT_PlotCorr

    PCA --> CorrDist
    PCA --> CorrSmooth
    NeuralSignal --> BarScatter

    PVEPM_Main --> SVM
    PVEPM_Main --> PlotOnset
    PVEPM_Main --> PlotNorm

    PVOF_Main --> BehavClust
    BehavClust --> FnBehavOF
    PVOF_Main --> OnsetPlot
    PVOF_Main --> FPlotRG1

    PVAnalysis_Main --> PVAF_FileFinder --> PVAF_Sync --> PVAF_Smooth --> PVAF_ErrorFix --> PVAF_Kalman
    PVAnalysis_Main --> PVAF_PlotRG
    PVAnalysis_Main --> PVAF_BehavOF

    %% External Tools
    subgraph "External Tools & Libraries" 
        direction TB
        DLC["DeepLabCut"]:::external
        DoricSDK["Doric SDK"]:::external
        InscopixMacro["Inscopix Macro"]:::external
    end

    DLC -.-> CLOI_GetDLC
    DoricSDK -.-> FileReader
    InscopixMacro -.-> Inscopix_Crop

    %% Click Events
    click DS_Video "https://github.com/firnenemerald/junglab/blob/main/PVEPM/filereader_video.m"
    click DS_Doric "https://github.com/firnenemerald/junglab/blob/main/PVEPM/filereader_doric.m"
    click DS_Inscopix "https://github.com/firnenemerald/junglab/blob/main/Inscopix/inscopix_macro.py"
    click FileFinder "https://github.com/firnenemerald/junglab/blob/main/PVEPM/filepathfinder.m"
    click Sync "https://github.com/firnenemerald/junglab/blob/main/PVEPM/sync_dlc.m"
    click Smoothing "https://github.com/firnenemerald/junglab/blob/main/PVEPM/smooth_dlc.m"
    click ErrorFixer "https://github.com/firnenemerald/junglab/blob/main/PVEPM/fixerror_doric.m"
    click Kalman "https://github.com/firnenemerald/junglab/blob/main/PVEPM/kalmanfilter_h.m"
    click CLOI_Main "https://github.com/firnenemerald/junglab/blob/main/CLOI/Analysis/CLOI_MainScript.m"
    click CLOI_GetMv "https://github.com/firnenemerald/junglab/blob/main/CLOI/Analysis/CLOI_GetMv.m"
    click CLOI_GetDLC "https://github.com/firnenemerald/junglab/blob/main/CLOI/Analysis/CLOI_GetDLC.m"
    click CLOI_PlotDLC "https://github.com/firnenemerald/junglab/blob/main/CLOI/Analysis/CLOI_PlotDLC.m"
    click PlotCustomBar_C "https://github.com/firnenemerald/junglab/blob/main/CLOI/Analysis/PlotCustomBar.m"
    click ChAT_Sync "https://github.com/firnenemerald/junglab/blob/main/ChAT/GetSyncTime.m"
    click ChAT_Sync "https://github.com/firnenemerald/junglab/blob/main/ChAT/GetExpDetail.m"
    click ChAT_GetData "https://github.com/firnenemerald/junglab/blob/main/ChAT/ChAT_GetData.m"
    click ChAT_PlotSignal "https://github.com/firnenemerald/junglab/blob/main/ChAT/ChAT_PlotSignal.m"
    click ChAT_PlotCorr "https://github.com/firnenemerald/junglab/blob/main/ChAT/ChAT_PlotCorr.m"
    click Behavior_Main "https://github.com/firnenemerald/junglab/blob/main/Behavior/behavioranalysis.m"
    click PCA "https://github.com/firnenemerald/junglab/blob/main/Cluster/PCAtrajectory_4.m"
    click CorrDist "https://github.com/firnenemerald/junglab/blob/main/Cluster/plotCorrVsDistance.m"
    click CorrSmooth "https://github.com/firnenemerald/junglab/blob/main/Cluster/plotCorrVsDistanceSmooth.m"
    click NeuralSignal "https://github.com/firnenemerald/junglab/blob/main/Cluster/getNeuralSignal.m"
    click BarScatter "https://github.com/firnenemerald/junglab/blob/main/Cluster/plotBarWithScatter.m"
    click Inscopix_Crop "https://github.com/firnenemerald/junglab/blob/main/Inscopix/inscopix_crop.m"
    click PVEPM_Main "https://github.com/firnenemerald/junglab/blob/main/PVEPM/mainscript.m"
    click SVM "https://github.com/firnenemerald/junglab/blob/main/PVEPM/da_SVM.m"
    click PlotOnset "https://github.com/firnenemerald/junglab/blob/main/PVEPM/plot_onset.m"
    click PlotNorm "https://github.com/firnenemerald/junglab/blob/main/PVEPM/plot_normhmap.m"
    click PVOF_Main "https://github.com/firnenemerald/junglab/blob/main/PVOF/mainscript_doricof.m"
    click BehavClust "https://github.com/firnenemerald/junglab/blob/main/PVOF/behavclustering.m"
    click FnBehavOF "https://github.com/firnenemerald/junglab/blob/main/PVOF/fn_behavcluster_OF_v2.m"
    click OnsetPlot "https://github.com/firnenemerald/junglab/blob/main/PVOF/onsetplott.m"
    click FPlotRG1 "https://github.com/firnenemerald/junglab/blob/main/PVOF/f_plotrg.m"
    click PVYM_Main "https://github.com/firnenemerald/junglab/blob/main/PVYM/PV_YM_MainAnalysis.m"
    click PV_GetDLC "https://github.com/firnenemerald/junglab/blob/main/PVYM/PV_GetDLCData.m"
    click PV_GetDoric "https://github.com/firnenemerald/junglab/blob/main/PVYM/PV_GetDoricSignal.m"
    click PV_Sync "https://github.com/firnenemerald/junglab/blob/main/PVYM/PV_SyncData.m"
    click PV_CorrAuto "https://github.com/firnenemerald/junglab/blob/main/PVYM/PV_CorrectAutomatic.m"
    click PV_CorrMan "https://github.com/firnenemerald/junglab/blob/main/PVYM/PV_CorrectManual.m"
    click PVAnalysis_Main "https://github.com/firnenemerald/junglab/blob/main/PV_Analysis/mainscript_doricof.m"
    click PVAF_FileFinder "https://github.com/firnenemerald/junglab/blob/main/PV_Analysis/filepathfinder.m"
    click PVAF_Sync "https://github.com/firnenemerald/junglab/blob/main/PV_Analysis/sync_dlc.m"
    click PVAF_Smooth "https://github.com/firnenemerald/junglab/blob/main/PV_Analysis/smooth_dlc.m"
    click PVAF_ErrorFix "https://github.com/firnenemerald/junglab/blob/main/PV_Analysis/fixerror_doric.m"
    click PVAF_Kalman "https://github.com/firnenemerald/junglab/blob/main/PV_Analysis/kalmanfilter_h.m"
    click PVAF_PlotRG "https://github.com/firnenemerald/junglab/blob/main/PV_Analysis/f_plotrg.m"
    click PVAF_BehavOF "https://github.com/firnenemerald/junglab/blob/main/PV_Analysis/fn_behavcluster_OF_v2.m"

    %% Styles
    classDef pre fill:#cce5ff,stroke:#3399ff,stroke-width:1px
    classDef analysis fill:#d4edda,stroke:#28a745,stroke-width:1px
    classDef viz fill:#fff3cd,stroke:#ffcc00,stroke-width:1px
    classDef main fill:#e2e3e5,stroke:#6c757d,stroke-width:1px
    classDef external fill:#f8f9fa,stroke:#6c757d,stroke-dasharray: 5 5
    class DS_Video,DS_Doric,DS_Inscopix,DLC,DoricSDK,InscopixMacro external
```
