# sciencecorp/synapse-api architecture

> Architecture diagram of the GitHub repository sciencecorp/synapse-api, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sciencecorp/synapse-api
- Repository: https://github.com/sciencecorp/synapse-api

## Overview

##### Project Type and Purpose
- Project type: A protocol‐only repository providing gRPC/Protobuf definitions and UDP streaming interfaces for neural interface devices.
- Purpose: Standardize control‐plane operations via gRPC and high‐throughput data streaming via UDP across a variety of “nodes” (e.g., spike detector, optical stimulation).

##### File Structure
- Top‐level folders:
– `.github/`: CI checks (e.g., proto‐check).
– `api/`: All `.proto` files organized into a flat set plus a `nodes/` subfolder.
- Key files:
– `api/*.proto`: Common messages (channel, time, status, etc.).
– `api/nodes/*.proto`: Definitions of each processing “node.”
– `build.sh`: Build or validate the proto files.
- No runtime code here—this repo is the canonical schema for clients and servers in multiple languages.

##### README for Insights
- Describes two main I/O planes:
– Control plane over gRPC (RPC calls for configuration, status, commands).
– Data plane over UDP (raw/time‐sensitive streaming).
- Encourages usage of client libraries (Python, C++, TS/Node) that wrap these proto definitions.

##### Main Components
a. Protobuf Schema Layer
– The `api/` folder: shared message definitions and service contracts.
b. Control Plane (gRPC)
– gRPC server implementing the Synapse services.
– Client libraries (Python, C++, TS/Node) invoking RPC methods.
c. Data Plane (UDP)
– High‐throughput streaming endpoints (e.g., raw neural data, spikes).
– Packet receiver on server side and packet sender on device side.
d. Node Processing Units
– Each `.proto` in `api/nodes/` describes a functional block (e.g., `spike_detector`, `disk_writer`).
– These can be standalone microservices, plugins, or threads in the server.
e. External Neural Interface Devices
– Actual hardware or simulators producing or consuming data.

##### Relationships & Architecture Patterns
- Clients ↔ gRPC Control Plane ↔ Node Config & Status
- Clients ↔ UDP Data Plane ↔ Node Data Streams
- Node Chaining
– Some nodes forward data to others (e.g., `spike_source.proto` → `spike_detector.proto` → `spike_binner.proto`).
- Layering
– Top layer: Client applications
– Middle layer: Synapse API services (control and data planes)
– Bottom layer: Hardware/Node implementations

##### Important Architectural Patterns
- Separation of Concerns: Control vs. Data plane.
- Pluggable Node Architecture: Each node proto defines a self‐contained processing unit.
- Language‐agnostic Interfaces: Proto contracts ensure every client—regardless of runtime—sees the same API.
- CI Gatekeeping: `.github/workflows/proto-check.yaml` enforces schema consistency.

##### Final Checklist
[ ] Node chaining illustrated where applicable
[ ] Call out the build/CI step (build.sh, proto‐check) as an ancillary process

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Clients
    Clients["Client Libraries"]:::client

    %% API Schema Definitions
    subgraph "Synapse Protobuf Definitions"
        dir_api["api/"]:::schema
        channel["channel.proto"]:::schema
        datatype["datatype.proto"]:::schema
        files["files.proto"]:::schema
        logging["logging.proto"]:::schema
        node_def["node.proto"]:::schema
        query["query.proto"]:::schema
        status["status.proto"]:::schema
        time["time.proto"]:::schema
        synapse_proto["synapse.proto"]:::schema
        stream_in["stream_in.proto"]:::schema
        stream_out["stream_out.proto"]:::schema
    end

    %% Control Plane
    subgraph "Control Plane (gRPC)"
        grpc_server["gRPC Server"]:::control
    end

    %% Data Plane
    subgraph "Data Plane (UDP)"
        udp_endpoints["UDP Streaming Endpoints"]:::data
    end

    %% Node Processing Units
    subgraph "Node Processing Units"
        bb["broadband_source.proto"]:::node
        dw["disk_writer.proto"]:::node
        es["electrical_stimulation.proto"]:::node
        os["optical_stimulation.proto"]:::node
        sc["signal_config.proto"]:::node
        ss["signal_status.proto"]:::node
        sf["spectral_filter.proto"]:::node
        sb["spike_binner.proto"]:::node
        sd["spike_detector.proto"]:::node
        ssr["spike_source.proto"]:::node
    end

    %% Node Chaining
    ssr --> sd
    sd --> sb

    %% Hardware
    Devices["Neural Interface Devices"]:::external

    %% CI / Build
    CI[".github/workflows/proto-check.yaml\nbuild.sh"]:::ci

    %% Connections
    Clients -->|"RPC Calls"| grpc_server
    grpc_server -->|"Uses Schema"| dir_api
    Clients -->|"UDP Streams"| udp_endpoints
    udp_endpoints -->|"Streams Data"| Devices
    grpc_server -->|"Config/Status RPCs"| bb
    grpc_server -->|"Config/Status RPCs"| dw
    grpc_server -->|"Config/Status RPCs"| es
    grpc_server -->|"Config/Status RPCs"| os
    grpc_server -->|"Config/Status RPCs"| sc
    grpc_server -->|"Config/Status RPCs"| ss
    grpc_server -->|"Config/Status RPCs"| sf
    grpc_server -->|"Config/Status RPCs"| sb
    grpc_server -->|"Config/Status RPCs"| sd
    grpc_server -->|"Config/Status RPCs"| ssr
    CI -.->|"Validates"| dir_api

    %% Click Events
    click dir_api "https://github.com/sciencecorp/synapse-api/tree/main/api/"
    click channel "https://github.com/sciencecorp/synapse-api/blob/main/api/channel.proto"
    click datatype "https://github.com/sciencecorp/synapse-api/blob/main/api/datatype.proto"
    click files "https://github.com/sciencecorp/synapse-api/blob/main/api/files.proto"
    click logging "https://github.com/sciencecorp/synapse-api/blob/main/api/logging.proto"
    click node_def "https://github.com/sciencecorp/synapse-api/blob/main/api/node.proto"
    click query "https://github.com/sciencecorp/synapse-api/blob/main/api/query.proto"
    click status "https://github.com/sciencecorp/synapse-api/blob/main/api/status.proto"
    click time "https://github.com/sciencecorp/synapse-api/blob/main/api/time.proto"
    click synapse_proto "https://github.com/sciencecorp/synapse-api/blob/main/api/synapse.proto"
    click stream_in "https://github.com/sciencecorp/synapse-api/blob/main/api/nodes/stream_in.proto"
    click stream_out "https://github.com/sciencecorp/synapse-api/blob/main/api/nodes/stream_out.proto"
    click bb "https://github.com/sciencecorp/synapse-api/blob/main/api/nodes/broadband_source.proto"
    click dw "https://github.com/sciencecorp/synapse-api/blob/main/api/nodes/disk_writer.proto"
    click es "https://github.com/sciencecorp/synapse-api/blob/main/api/nodes/electrical_stimulation.proto"
    click os "https://github.com/sciencecorp/synapse-api/blob/main/api/nodes/optical_stimulation.proto"
    click sc "https://github.com/sciencecorp/synapse-api/blob/main/api/nodes/signal_config.proto"
    click ss "https://github.com/sciencecorp/synapse-api/blob/main/api/nodes/signal_status.proto"
    click sf "https://github.com/sciencecorp/synapse-api/blob/main/api/nodes/spectral_filter.proto"
    click sb "https://github.com/sciencecorp/synapse-api/blob/main/api/nodes/spike_binner.proto"
    click sd "https://github.com/sciencecorp/synapse-api/blob/main/api/nodes/spike_detector.proto"
    click ssr "https://github.com/sciencecorp/synapse-api/blob/main/api/nodes/spike_source.proto"
    click CI "https://github.com/sciencecorp/synapse-api/blob/main/.github/workflows/proto-check.yaml"
    click CI "https://github.com/sciencecorp/synapse-api/blob/main/build.sh"

    %% Styles
    classDef schema fill:#cce5ff,stroke:#66b0ff,color:#004085
    classDef control fill:#d4edda,stroke:#81c784,color:#155724
    classDef data fill:#fff3cd,stroke:#ffecb5,color:#856404
    classDef node fill:#e2e3e5,stroke:#bcbfc4,color:#383d41
    classDef client fill:#f0f8ff,stroke:#a0c4ff,color:#1a1f71
    classDef external fill:#f8d7da,stroke:#f5c6cb,color:#721c24
    classDef ci fill:#e8daef,stroke:#d6b3e0,color:#4a235a
```
