How alleninstitute/sonata works
Project Type and Purpose
• Open-source Python library and reference API (pySONATA) for the SONATA Data Format.
• Provides read/write support for node/edge tables (HDF5/CSV), simulation configuration (JSON), and output reports (e.g., membrane potentials, spike trains).
• Ships with specification docs, tutorials, examples, and automated tests.
High-Level File Structure
• Top level
• docs/ – Developer spec and format guide
• examples/ – Sample networks, build scripts, analysis and plotting code
• src/pysonata/sonata/ – Core library modules
• tutorials/ – Jupyter notebooks demonstrating usage
• tests/ – Unit tests under src/pysonata/sonata/tests
• Core modules under src/pysonata/sonata/
• circuit/ – Classes for populations, nodes, edges, file roots
• reports/ – Subpackages for compartment reports & spike trains (with adapters for CSV, HDF5, NWB)
• iotools.py – Low-level HDF5/CSV read-write helpers
• config.py – JSON simulation configuration parsing
• utils.py – Shared utilities
Key Insights from README
• SONATA uses HDF5 and CSV for large-scale, efficient parallelizable storage.
• Configuration is JSON-based.
• Reports schema is systematic and extensible via adaptor pattern (e.g., nwb_adaptors.py, csv_adaptors.py).
Main Components
a. User Layer
• Examples, tutorials, downstream tools (BMTK, PyNN, NetPyNE)
b. API Layer (pysonata)
• Entry point: setup.py → imports sonata package
• config.py: JSON simulation config → simulation parameters, report directives
c. Circuit Module
• file_root.py & file.py: open/select HDF5 file groups
• node.py, edge.py, population.py, types_table.py: in-memory representations of network components
d. I/O and Adapters
• iotools.py: HDF5/CSV low-level I/O
• Reports → compartment_reader/writer & spike_trains adaptors (CSV, HDF5, NWB)
e. External Dependencies
• h5py, numpy, pandas, pynwb (for NWB)
f. Output
• Writes/reads .h5, .csv, .json; plotting helpers in examples
Component Interactions
• User script → calls pysonata API
• config.py parses JSON → instructs which files to load or write
• circuit module uses iotools to open HDF5/CSV → loads node & edge tables → constructs Python objects
• reports module reads/writes simulation outputs via adaptor classes → optionally converts to NWB
• Analysis scripts (examples) consume data objects or raw files → produce plots
Architectural Patterns and Principles
• Layered architecture: Separation between API, domain logic (circuit & reports), and I/O adapters.
• Adapter pattern for file formats: swap CSV, HDF5, NWB with minimal changes.
• Modular: each subpackage focuses on one responsibility (single responsibility principle).
• Extensible: users can add custom node/edge properties or new report formats.