# alleninstitute/sonata architecture

> Architecture diagram of the GitHub repository alleninstitute/sonata, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/alleninstitute/sonata
- Repository: https://github.com/alleninstitute/sonata

## Overview

##### Project Type and Purpose
- Open-source Python library and reference API (pySONATA) for the SONATA Data Format.
- Provides read/write support for node/edge tables (HDF5/CSV), simulation configuration (JSON), and output reports (e.g., membrane potentials, spike trains).
- Ships with specification docs, tutorials, examples, and automated tests.

##### High-Level File Structure
- Top level
- docs/ – Developer spec and format guide
- examples/ – Sample networks, build scripts, analysis and plotting code
- src/pysonata/sonata/ – Core library modules
- tutorials/ – Jupyter notebooks demonstrating usage
- tests/ – Unit tests under src/pysonata/sonata/tests
- Core modules under src/pysonata/sonata/
- circuit/ – Classes for populations, nodes, edges, file roots
- reports/ – Subpackages for compartment reports & spike trains (with adapters for CSV, HDF5, NWB)
- iotools.py – Low-level HDF5/CSV read-write helpers
- config.py – JSON simulation configuration parsing
- utils.py – Shared utilities

##### Key Insights from README
- SONATA uses HDF5 and CSV for large-scale, efficient parallelizable storage.
- Configuration is JSON-based.
- Reports schema is systematic and extensible via adaptor pattern (e.g., nwb_adaptors.py, csv_adaptors.py).

##### Main Components
a. User Layer
- Examples, tutorials, downstream tools (BMTK, PyNN, NetPyNE)
b. API Layer (pysonata)
- Entry point: setup.py → imports sonata package
- config.py: JSON simulation config → simulation parameters, report directives
c. Circuit Module
- file_root.py & file.py: open/select HDF5 file groups
- node.py, edge.py, population.py, types_table.py: in-memory representations of network components
d. I/O and Adapters
- iotools.py: HDF5/CSV low-level I/O
- Reports → compartment_reader/writer & spike_trains adaptors (CSV, HDF5, NWB)
e. External Dependencies
- h5py, numpy, pandas, pynwb (for NWB)
f. Output
- Writes/reads .h5, .csv, .json; plotting helpers in examples

##### Component Interactions
- User script → calls pysonata API
- config.py parses JSON → instructs which files to load or write
- circuit module uses iotools to open HDF5/CSV → loads node & edge tables → constructs Python objects
- reports module reads/writes simulation outputs via adaptor classes → optionally converts to NWB
- Analysis scripts (examples) consume data objects or raw files → produce plots

##### Architectural Patterns and Principles
- Layered architecture: Separation between API, domain logic (circuit & reports), and I/O adapters.
- Adapter pattern for file formats: swap CSV, HDF5, NWB with minimal changes.
- Modular: each subpackage focuses on one responsibility (single responsibility principle).
- Extensible: users can add custom node/edge properties or new report formats.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User Layer
    subgraph "User Layer" 
        direction TB
        DevDocs["Developer Docs"]:::user
        Examples["Examples"]:::user
        Tutorials["Tutorials"]:::user
        Downstream["Downstream Tools\n(BMTK, PyNN, NetPyNE)"]:::user
    end

    %% API Layer
    subgraph "API Layer" 
        direction TB
        Setup["setup.py"]:::api
        Init["__init__.py"]:::api
        Config["config.py"]:::api
    end

    %% Core Circuit Module
    subgraph "Circuit Module" 
        direction TB
        FileRoot["file_root.py"]:::core
        FileGroup["file.py"]:::core
        Node["node.py"]:::core
        Edge["edge.py"]:::core
        Population["population.py"]:::core
        TypesTable["types_table.py"]:::core
        CircuitUtils["utils.py,\ngroup.py,\ncolumn_property.py"]:::core
    end

    %% I/O Helpers
    IOTools["iotools.py"]:::io

    %% Reports Layer
    subgraph "Reports Module"
        direction TB
        subgraph "Compartment Reports"
            direction TB
            CompReport["compartment_report.py"]:::reports
            CompReader["compartment_reader.py"]:::reports
            CompWriter["compartment_writer.py"]:::reports
            CompCore["core.py"]:::reports
        end
        subgraph "Spike Trains Reports"
            direction TB
            STCore["core.py"]:::reports
            STData["spike_trains.py"]:::reports
            STFile["spike_trains_file.py"]:::reports
            STBuffer["spike_train_buffer.py,\nstats.py"]:::reports
            STPlot["plotting.py"]:::reports
        end
        subgraph "Spike Trains Adaptors"
            direction TB
            CSVAdapt["csv_adaptors.py"]:::io
            SonataAdapt["sonata_adaptors.py"]:::io
            NWBAdapt["nwb_adaptors.py"]:::io
        end
    end

    %% External Dependencies
    subgraph "External Dependencies"
        direction TB
        H5PY["h5py"]:::external
        NumPy["numpy"]:::external
        Pandas["pandas"]:::external
        PyNWB["pynwb"]:::external
    end

    %% Output Formats
    subgraph "Output Formats"
        direction TB
        H5File[".h5"]:::output
        CSVFile[".csv"]:::output
        JSONFile[".json"]:::output
    end

    %% Interactions
    DevDocs -->|reads| Setup
    Examples -->|uses API| Setup
    Tutorials -->|imports| Init
    Downstream -->|calls| Init

    Setup -->|imports| Init
    Init -->|loads| Config
    Config -->|parses JSON| Config

    Config -->|"select files"| FileRoot
    FileRoot -->|"group access"| FileGroup
    FileGroup -->|"load tables"| IOTools
    IOTools -->|"reads/writes"| Node
    IOTools -->|"reads/writes"| Edge
    IOTools -->|"reads/writes"| Population
    IOTools -->|"reads/writes"| TypesTable
    IOTools -->|"helpers"| CircuitUtils

    Node -->|"constructs"| Population
    Edge -->|"constructs"| Population

    Config -->|"directs reports"| CompReport
    CompReport -->|uses| CompReader
    CompReport -->|uses| CompWriter
    CompReader -->|"reads via"| IOTools
    CompWriter -->|"writes via"| IOTools

    Config -->|"directs reports"| STCore
    STCore -->|uses| STData
    STCore -->|uses| STFile
    STCore -->|uses| STBuffer
    STCore -->|uses| STPlot

    STCore -->|adapts CSV| CSVAdapt
    STCore -->|adapts HDF5| SonataAdapt
    STCore -->|adapts NWB| NWBAdapt

    IOTools -->|depends on| H5PY
    IOTools -->|depends on| NumPy
    IOTools -->|depends on| Pandas
    NWBAdapt -->|depends on| PyNWB

    CompReport -->|outputs| H5File
    CompReport -->|outputs| CSVFile
    STCore -->|outputs| H5File
    STCore -->|outputs| CSVFile
    STCore -->|outputs| JSONFile

    %% Click Events
    click DevDocs "https://github.com/alleninstitute/sonata/blob/master/docs/SONATA_DEVELOPER_GUIDE.md"
    click Examples "https://github.com/alleninstitute/sonata/tree/master/examples/"
    click Tutorials "https://github.com/alleninstitute/sonata/tree/master/tutorials/"
    click Setup "https://github.com/alleninstitute/sonata/blob/master/src/pysonata/setup.py"
    click Init "https://github.com/alleninstitute/sonata/blob/master/src/pysonata/sonata/__init__.py"
    click Config "https://github.com/alleninstitute/sonata/blob/master/src/pysonata/sonata/config.py"
    click IOTools "https://github.com/alleninstitute/sonata/blob/master/src/pysonata/sonata/iotools.py"
    click FileRoot "https://github.com/alleninstitute/sonata/blob/master/src/pysonata/sonata/circuit/file_root.py"
    click FileGroup "https://github.com/alleninstitute/sonata/blob/master/src/pysonata/sonata/circuit/file.py"
    click Node "https://github.com/alleninstitute/sonata/blob/master/src/pysonata/sonata/circuit/node.py"
    click Edge "https://github.com/alleninstitute/sonata/blob/master/src/pysonata/sonata/circuit/edge.py"
    click Population "https://github.com/alleninstitute/sonata/blob/master/src/pysonata/sonata/circuit/population.py"
    click TypesTable "https://github.com/alleninstitute/sonata/blob/master/src/pysonata/sonata/circuit/types_table.py"
    click CircuitUtils "https://github.com/alleninstitute/sonata/blob/master/src/pysonata/sonata/circuit/utils.py"
    click CompReport "https://github.com/alleninstitute/sonata/blob/master/src/pysonata/sonata/reports/compartment/compartment_report.py"
    click CompReader "https://github.com/alleninstitute/sonata/blob/master/src/pysonata/sonata/reports/compartment/compartment_reader.py"
    click CompWriter "https://github.com/alleninstitute/sonata/blob/master/src/pysonata/sonata/reports/compartment/compartment_writer.py"
    click CompCore "https://github.com/alleninstitute/sonata/blob/master/src/pysonata/sonata/reports/compartment/core.py"
    click STCore "https://github.com/alleninstitute/sonata/blob/master/src/pysonata/sonata/reports/spike_trains/core.py"
    click STData "https://github.com/alleninstitute/sonata/blob/master/src/pysonata/sonata/reports/spike_trains/spike_trains.py"
    click STFile "https://github.com/alleninstitute/sonata/blob/master/src/pysonata/sonata/reports/spike_trains/spike_trains_file.py"
    click STBuffer "https://github.com/alleninstitute/sonata/blob/master/src/pysonata/sonata/reports/spike_trains/spike_train_buffer.py"
    click CSVAdapt "https://github.com/alleninstitute/sonata/blob/master/src/pysonata/sonata/reports/spike_trains/adaptors/csv_adaptors.py"
    click SonataAdapt "https://github.com/alleninstitute/sonata/blob/master/src/pysonata/sonata/reports/spike_trains/adaptors/sonata_adaptors.py"
    click NWBAdapt "https://github.com/alleninstitute/sonata/blob/master/src/pysonata/sonata/reports/spike_trains/adaptors/nwb_adaptors.py"

    %% Styles
    classDef user fill:#FFDAB9,stroke:#E9967A
    classDef api fill:#B0E0E6,stroke:#4682B4
    classDef core fill:#E6E6FA,stroke:#9370DB
    classDef io fill:#F0E68C,stroke:#BDB76B
    classDef reports fill:#98FB98,stroke:#2E8B57
    classDef external fill:#D3D3D3,stroke:#A9A9A9
    classDef output fill:#FFE4E1,stroke:#CD5C5C
```
