# cwindolf/dartsort architecture

> Architecture diagram of the GitHub repository cwindolf/dartsort, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/cwindolf/dartsort
- Repository: https://github.com/cwindolf/dartsort

## Overview

##### Main System Components
The system architecture should be organized into these primary components:

a. Core Processing Pipeline
- Detection Module (src/dartsort/detect)
- Localization Module (src/dartsort/localize)
- Peeling Module (src/dartsort/peel)
- Clustering Module (src/dartsort/cluster)
- Templates Module (src/dartsort/templates)
- Transform Module (src/dartsort/transform)

b. Support Systems
- Utilities (src/dartsort/util)
- Visualization (src/dartsort/vis)
- Configuration (src/dartsort/config.py)
- Pretrained Models (src/dartsort/pretrained)

##### Data Flow Architecture

**Raw Neural Data**
- ↓
**Detection (spike detection)**
- ↓
**Localization (spatial positioning)**
- ↓
**Peeling (signal separation)**
- ↓
**Clustering (spike sorting)**
- ↓
**Template Generation**
- ↓
**Transformed Output**

##### Key Design Elements

Use these visual elements:
- Hexagons for pretrained models
- Diamonds for decision points
- Blue: Core processing modules
- Green: Data input/output
- Yellow: Support utilities
- Purple: Visualization components

##### Integration Points

Connections with:
- PyTorch integration (localize_torch.py)
- DREDge motion correction
- External data formats
- Visualization pipeline

##### Technical Details

- GPU acceleration support (indicated by PyTorch usage)
- Parallel processing capabilities (multiprocessing_util.py)
- Model storage (pretrained directory)
- Testing framework integration

##### Specific Implementation Notes

a. Interface Layer
- CLI interface (cli.py)
- Pipeline scripts

b. Processing Layer
- Core algorithms
- Mathematical operations

c. Infrastructure Layer
- Data utilities
- Configuration management
- Testing framework

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph Input
        RawData[Raw Neural Data]:::input
    end

    subgraph "Core Processing Pipeline"
        Detection["Detection Module"]:::core
        Localization["Localization Module"]:::core
        Peeling["Peeling Module"]:::core
        Clustering["Clustering Module"]:::core
        Templates["Templates Module"]:::core
        Transform["Transform Module"]:::core
    end

    subgraph "Support Systems"
        Config["Configuration System"]:::support
        Utils["Utilities"]:::support
        Vis["Visualization System"]:::support
    end

    subgraph "Pretrained Models"
        PretrainedNP1["NP1 Detection Model"]:::model
        PretrainedNP2["NP2 Detection Model"]:::model
    end

    subgraph "Interface Layer"
        CLI["CLI Interface"]:::interface
        Pipeline["Main Pipeline"]:::interface
    end

    subgraph "Clustering Operations"
        InitialCluster["Initial Clustering"]:::cluster
        MergeOps["Merge Operations"]:::cluster
        SplitOps["Split Operations"]:::cluster
    end

    RawData --> Pipeline
    Pipeline --> Detection
    Detection --> Localization
    Localization --> Peeling
    Peeling --> Clustering
    
    Clustering --> InitialCluster
    InitialCluster --> MergeOps
    InitialCluster --> SplitOps
    MergeOps --> Templates
    SplitOps --> Templates
    
    Templates --> Transform

    PretrainedNP1 -.-> Detection
    PretrainedNP2 -.-> Detection
    
    Config --> Pipeline
    Utils --> Pipeline
    Pipeline --> Vis
    CLI --> Pipeline

    %% Component Mappings
    click Detection "https://github.com/cwindolf/dartsort/blob/main/src/dartsort/detect/detect.py"
    click Localization "https://github.com/cwindolf/dartsort/blob/main/src/dartsort/localize/localize_torch.py"
    click Peeling "https://github.com/cwindolf/dartsort/blob/main/src/dartsort/peel/peel_base.py"
    click Clustering "https://github.com/cwindolf/dartsort/tree/main/src/dartsort/cluster/"
    click InitialCluster "https://github.com/cwindolf/dartsort/blob/main/src/dartsort/cluster/initial.py"
    click MergeOps "https://github.com/cwindolf/dartsort/blob/main/src/dartsort/cluster/merge.py"
    click SplitOps "https://github.com/cwindolf/dartsort/blob/main/src/dartsort/cluster/split.py"
    click Templates "https://github.com/cwindolf/dartsort/blob/main/src/dartsort/templates/templates.py"
    click Transform "https://github.com/cwindolf/dartsort/blob/main/src/dartsort/transform/transform_base.py"
    click Vis "https://github.com/cwindolf/dartsort/tree/main/src/dartsort/vis/"
    click Config "https://github.com/cwindolf/dartsort/blob/main/src/dartsort/config.py"
    click PretrainedNP1 "https://github.com/cwindolf/dartsort/blob/main/src/dartsort/pretrained/detect_np1.pt"
    click PretrainedNP2 "https://github.com/cwindolf/dartsort/blob/main/src/dartsort/pretrained/detect_np2.pt"
    click Utils "https://github.com/cwindolf/dartsort/blob/main/src/dartsort/util/data_util.py"
    click Pipeline "https://github.com/cwindolf/dartsort/blob/main/src/dartsort/main.py"
    click CLI "https://github.com/cwindolf/dartsort/blob/main/src/dartsort/cli.py"

    %% Styling
    classDef core fill:#2196F3,stroke:#1565C0,color:white
    classDef input fill:#4CAF50,stroke:#2E7D32,color:white
    classDef support fill:#FFC107,stroke:#FFA000,color:black
    classDef model fill:#9C27B0,stroke:#7B1FA2,color:white
    classDef interface fill:#FF5722,stroke:#E64A19,color:white
    classDef cluster fill:#00BCD4,stroke:#0097A7,color:white

    %% Legend
    subgraph Legend
        L1[Core Processing]:::core
        L2[Input/Output]:::input
        L3[Support Systems]:::support
        L4[Pretrained Models]:::model
        L5[Interface]:::interface
        L6[Clustering Ops]:::cluster
    end
```
