# eonsystemspbc/drosophila_brain_model_lif architecture

> Architecture diagram of the GitHub repository eonsystemspbc/drosophila_brain_model_lif, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-03-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/eonsystemspbc/drosophila_brain_model_lif
- Repository: https://github.com/eonsystemspbc/drosophila_brain_model_lif

## Overview

This repo is a **research-grade computational neuroscience simulation** project: a **Python + Brian2** leaky integrate-and-fire (LIF) network model built from the **adult Drosophila connectome (FlyWire)**. The primary “product” is not a deployed service but a **reproducible simulation pipeline** that:
- loads connectome tables (connectivity + neuron completeness lists),
- configures experimental manipulations (activation/silencing by FlyWire IDs),
- runs a Brian2 simulation,
- writes large result artifacts (spike times/rates) to `results/`,
- and provides notebooks (`example.ipynb`, `figures.ipynb`) to reproduce paper figures/analyses.

- **data inputs** (CSV/Parquet/Pickle),
- **core simulation engine** (Brian2 model code),
- **experiment configuration / manipulation layer**,
- **outputs** (parquet results, derived analyses),
- and **execution surfaces** (local Jupyter vs Colab, optional C++ codegen performance path).

##### File tree analysis (map files to architectural components)
Top-level assets imply these components:

##### Core code (the “engine”)
- `model.py` — the actual Brian2-based LIF model + simulation orchestration (core component).
- `utils.py` — helper utilities (likely loading/parsing connectome, ID handling, saving outputs, parallelism helpers).

##### Primary execution / user interface
- `example.ipynb` — tutorial “how to run a simulation & manipulate neurons”.
- `figures.ipynb` — paper reproduction pipeline (reads/writes results, computes metrics, generates figures).
- `results/eon_1/demo_notebook.ipynb` — example demo tied to a specific result set.

##### Input datasets (connectome / neuron metadata)
- `2023_03_23_connectivity_630_final.parquet` — connectivity edges/weights for FlyWire v630.
- `2023_03_23_completeness_630_final.csv` — canonical list of neurons (completeness) for v630.
- `Connectivity_783.parquet`, `Completeness_783.csv` — alternative public version 783 inputs.
- `sez_neurons.pickle` — a precomputed neuron set used in analyses/experiments.

##### Outputs / artifacts
- `results/` — simulation outputs and examples (parquet outputs like `sugarR*.parquet`).
- The README notes “raw output is several GB” and stored externally; local results are under `results/`.

##### Environment / reproducibility
- `environment.yml`, `environment_full.yml` — conda env specs (full pinning in `environment_full.yml`).
- Notes on **Brian2 C++ code generation** as a performance mode.

##### README insights that should show up
Key architectural behaviors described explicitly:
- **Manipulations**
- Activation: Poisson spiking at fixed frequency; supports two neuron sets with distinct frequencies.
- Silencing: zero out synapses to/from neuron set.
- **Addressing** by FlyWire ID.
- **Outputs**: spike times and rates of affected neurons (and potentially downstream population effects).
- **Two connectome versions** (630 default; 783 via `config` dictionary changes).
- **Execution modes**: local conda + Jupyter; Colab; performance improvement with Brian2 C++ codegen.

##### Jupyter Notebook Layer
- `example.ipynb` (interactive experiments)
- `figures.ipynb` (reproduction/analysis pipeline)
##### Configuration & Experiment Definition
- `config` dict (paths, n_proc, version selection)
- Manipulation definitions (activation sets A/B + silencing set)
##### Core Simulation Package
- `model.py` (network construction, LIF equations, Brian2 objects, run loop)
- `utils.py` (I/O, preprocessing, helpers)
##### Connectome Data Store (Local Files)
- Connectivity Parquet (v630/v783)
- Completeness CSV (v630/v783)
- `sez_neurons.pickle` (optional metadata set)
##### Simulation Runtime
- **Brian2 Python runtime**
- Optional **Brian2 C++ codegen** path (performance accelerator)
- Optional **multiprocessing** (implied by `n_proc`)
##### Results Store
- `results/.../*.parquet` (spike times/rates outputs)
##### Post-processing / Figure Generation
- analysis steps inside `figures.ipynb` reading results and producing paper plots

- **User → Notebooks**
- **Notebooks → Config/Experiment Definition**
- **Config → Data Loader (utils/model) → Connectome Files**
- **Config/Experiment → model.py (build network + apply activation/silencing rules)**
- **model.py → Brian2 Runtime (simulate)**
- **Brian2 Runtime → Results Store (`results/`)**
- **figures.ipynb → Results Store (read) → Derived metrics/figures**

- Notebooks **invoke** `model.py` functions (control flow),
- connectome files **feed** network construction,
- results files **feed** analysis.

##### Architectural patterns
- **Data-driven model construction**: network topology derived from Parquet/CSV, not hardcoded.
- **Experiment-as-configuration**: manipulations are parameterized by FlyWire IDs and frequencies.
- **Separation of concerns**:
- notebooks = orchestration / narrative,
- `model.py` = simulation logic,
- `utils.py` = support I/O/preprocessing,
- `results/` = artifact boundary.
- **Performance “backend swap”**: Brian2 interpreted vs C++ codegen.

##### Technologies explicitly
- Python
- Brian2 (with optional C++ codegen)
- Pandas/Parquet (implied by `.parquet`)
- Conda environment (environment.yml)
- Jupyter/Colab

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% ======================
    %% Global actors / entry
    %% ======================
    U["User / Researcher"]:::actor

    subgraph "Execution Surfaces (Interactive)"
        NB["Jupyter / Colab"]:::surface
        EX["example.ipynb (run simulations & manipulations)"]:::notebook
        FIG["figures.ipynb (analysis & paper figure reproduction)"]:::notebook
        DEMO["results/eon_1/demo_notebook.ipynb (demo tied to result set)"]:::notebook
    end

    U -->|"runs"| NB
    NB -->|"open"| EX
    NB -->|"open"| FIG
    NB -->|"open"| DEMO

    %% ======================
    %% Reproducibility layer
    %% ======================
    subgraph "Reproducibility & Environment"
        ENV["Conda Environment (Python deps)"]:::env
        ENVY["environment.yml"]:::file
        ENVF["environment_full.yml (fully pinned)"]:::file
        README["Readme.md (usage/config/version/codegen notes)"]:::doc
    end

    ENV -->|"spec"| ENVY
    ENV -->|"spec"| ENVF
    README -->|"documents"| EX
    README -->|"documents"| FIG
    ENV -.->|"enables runtime"| NB

    %% ======================
    %% Experiment configuration
    %% ======================
    subgraph "Experiment Configuration & Manipulations"
        CFG["Config dictionary (paths, n_proc, version switch)"]:::config
        VER["Connectome version switch (v630 default / v783 optional)"]:::config
        MAN["Manipulations (by FlyWire IDs)"]:::config
        ACT["Activation: Poisson spikes @ f1/f2 (two sets)"]:::config
        SIL["Silencing: zero synapses to/from set"]:::config
    end

    EX -.->|"defines"| CFG
    EX -.->|"defines"| MAN
    FIG -.->|"defines/reads"| CFG

    CFG -->|"selects"| VER
    MAN -->|"includes"| ACT
    MAN -->|"includes"| SIL

    %% ======================
    %% Core simulation package
    %% ======================
    subgraph "Core Simulation Package (Python)"
        MODEL["model.py (Brian2 LIF network + orchestration)"]:::core
        UTILS["utils.py (I/O, preprocessing, helpers)"]:::core

        subgraph "Internal Responsibilities (conceptual)"
            LOAD["Load/connectome parsing (Pandas/Parquet/CSV/Pickle)"]:::step
            BUILD["Construct LIF network topology from connectome"]:::step
            APPLY["Apply manipulations (activation/silencing rules)"]:::step
            WRITE["Write outputs (spike times/rates) to results/"]:::step
        end

        UTILS -->|"supports"| LOAD
        MODEL -->|"uses"| LOAD
        MODEL -->|"builds"| BUILD
        MODEL -->|"applies"| APPLY
        MODEL -->|"calls"| WRITE
    end

    EX -.->|"invokes"| MODEL
    EX -.->|"invokes"| UTILS
    FIG -.->|"invokes/reads"| UTILS

    CFG -->|"parameters"| MODEL
    MAN -->|"parameters"| MODEL

    %% ======================
    %% Input datasets (local files)
    %% ======================
    subgraph "Connectome & Metadata Inputs (Local Files)"
        DS[("Local dataset files")]:::datastore

        subgraph "FlyWire v630 (default)"
            C630P[("Connectivity v630 (.parquet)")]:::datastore
            C630C[("Completeness v630 (.csv)")]:::datastore
        end

        subgraph "FlyWire v783 (optional)"
            C783P[("Connectivity v783 (.parquet)")]:::datastore
            C783C[("Completeness v783 (.csv)")]:::datastore
        end

        SEZ[("sez_neurons.pickle (optional neuron set)")]:::datastore
    end

    DS --> C630P
    DS --> C630C
    DS --> C783P
    DS --> C783C
    DS --> SEZ

    VER -->|"v630"| C630P
    VER -->|"v630"| C630C
    VER -.->|"v783"| C783P
    VER -.->|"v783"| C783C

    C630P -->|"load_connectivity"| LOAD
    C630C -->|"load_neuron_list"| LOAD
    C783P -.->|"load_connectivity"| LOAD
    C783C -.->|"load_neuron_list"| LOAD
    SEZ -.->|"load_optional_set"| LOAD

    %% ======================
    %% Simulation runtime
    %% ======================
    subgraph "Simulation Runtime"
        B2["Brian2 Runtime (Python)"]:::runtime
        CPP["Optional: Brian2 C++ codegen (performance)"]:::runtimeOpt
        MP["Optional: multiprocessing (n_proc)"]:::runtimeOpt
    end

    MODEL -->|"run_simulation"| B2
    CFG -.->|"n_proc"| MP
    MP -.->|"parallelize where applicable"| MODEL
    B2 -.->|"accelerate via"| CPP

    %% ======================
    %% Outputs & artifacts
    %% ======================
    subgraph "Results & Artifacts"
        RROOT[("results/ (artifact boundary)")]:::datastore
        REX[("results/example/ (example artifacts)")]:::datastore

        OUT1[("sugarR.parquet (spike times/rates)")]:::datastore
        OUT2[("sugarR_100Hz.parquet (100Hz condition)")]:::datastore
        OUT3[("sugarR-720575940617937543.parquet (per-neuron)")]:::datastore
        OUT4[("sugarR-720575940621754367.parquet (per-neuron)")]:::datastore
        OUT5[("sugarR-720575940622695448.parquet (per-neuron)")]:::datastore

        EXT[("External raw output archive (several GB, referenced in README)")]:::datastoreExt
    end

    WRITE -->|"write_parquet"| RROOT
    RROOT -->|"contains"| REX

    REX --> OUT1
    REX --> OUT2
    REX --> OUT3
    REX --> OUT4
    REX --> OUT5

    B2 -->|"spike_times/rates"| WRITE

    %% Post-processing path
    FIG -->|"read_results"| RROOT
    FIG -->|"compute_metrics_and_make_figures"| FIGOUT["Derived metrics / paper figures"]:::analysis

    %% External archive relationship
    RROOT -.->|"subset / local copy of"| EXT
    README -.->|"links to"| EXT

    %% ======================
    %% Notes / annotations
    %% ======================
    NOTE1["Data-driven model construction: topology from Parquet/CSV (not hardcoded)"]:::note
    NOTE2["Experiment-as-configuration: FlyWire IDs + frequencies drive activation/silencing"]:::note
    NOTE3["Optional backend swap: Brian2 interpreted vs C++ codegen"]:::note

    LOAD -.->|"pattern"| NOTE1
    MAN -.->|"pattern"| NOTE2
    CPP -.->|"pattern"| NOTE3

    %% ======================
    %% Click events (from mapping)
    %% ======================
    click MODEL "https://github.com/eonsystemspbc/drosophila_brain_model_lif/blob/main/model.py"
    click UTILS "https://github.com/eonsystemspbc/drosophila_brain_model_lif/blob/main/utils.py"
    click EX "https://github.com/eonsystemspbc/drosophila_brain_model_lif/blob/main/example.ipynb"
    click FIG "https://github.com/eonsystemspbc/drosophila_brain_model_lif/blob/main/figures.ipynb"
    click DEMO "https://github.com/eonsystemspbc/drosophila_brain_model_lif/blob/main/results/eon_1/demo_notebook.ipynb"
    click C630P "https://github.com/eonsystemspbc/drosophila_brain_model_lif/blob/main/2023_03_23_connectivity_630_final.parquet"
    click C630C "https://github.com/eonsystemspbc/drosophila_brain_model_lif/blob/main/2023_03_23_completeness_630_final.csv"
    click C783P "https://github.com/eonsystemspbc/drosophila_brain_model_lif/blob/main/Connectivity_783.parquet"
    click C783C "https://github.com/eonsystemspbc/drosophila_brain_model_lif/blob/main/Completeness_783.csv"
    click SEZ "https://github.com/eonsystemspbc/drosophila_brain_model_lif/blob/main/sez_neurons.pickle"
    click RROOT "https://github.com/eonsystemspbc/drosophila_brain_model_lif/tree/main/results/"
    click REX "https://github.com/eonsystemspbc/drosophila_brain_model_lif/tree/main/results/example/"
    click OUT1 "https://github.com/eonsystemspbc/drosophila_brain_model_lif/blob/main/results/example/sugarR.parquet"
    click OUT2 "https://github.com/eonsystemspbc/drosophila_brain_model_lif/blob/main/results/example/sugarR_100Hz.parquet"
    click OUT3 "https://github.com/eonsystemspbc/drosophila_brain_model_lif/blob/main/results/example/sugarR-720575940617937543.parquet"
    click OUT4 "https://github.com/eonsystemspbc/drosophila_brain_model_lif/blob/main/results/example/sugarR-720575940621754367.parquet"
    click OUT5 "https://github.com/eonsystemspbc/drosophila_brain_model_lif/blob/main/results/example/sugarR-720575940622695448.parquet"
    click ENVY "https://github.com/eonsystemspbc/drosophila_brain_model_lif/blob/main/environment.yml"
    click ENVF "https://github.com/eonsystemspbc/drosophila_brain_model_lif/blob/main/environment_full.yml"
    click README "https://github.com/eonsystemspbc/drosophila_brain_model_lif/blob/main/Readme.md"

    %% ======================
    %% Styles
    %% ======================
    classDef actor fill:#0f172a,stroke:#0f172a,color:#ffffff,stroke-width:1px
    classDef surface fill:#1d4ed8,stroke:#1e40af,color:#ffffff,stroke-width:1px
    classDef notebook fill:#2563eb,stroke:#1e40af,color:#ffffff,stroke-width:1px
    classDef env fill:#0ea5e9,stroke:#0284c7,color:#ffffff,stroke-width:1px
    classDef file fill:#38bdf8,stroke:#0284c7,color:#0b1220,stroke-width:1px
    classDef doc fill:#22c55e,stroke:#15803d,color:#06210f,stroke-width:1px
    classDef config fill:#a855f7,stroke:#6b21a8,color:#ffffff,stroke-width:1px
    classDef core fill:#f97316,stroke:#c2410c,color:#1b0b02,stroke-width:1px
    classDef step fill:#fb923c,stroke:#c2410c,color:#1b0b02,stroke-width:1px
    classDef runtime fill:#ef4444,stroke:#b91c1c,color:#ffffff,stroke-width:1px
    classDef runtimeOpt fill:#fecaca,stroke:#b91c1c,color:#450a0a,stroke-width:1px,stroke-dasharray: 5 4
    classDef datastore fill:#f59e0b,stroke:#b45309,color:#1f1403,stroke-width:1px
    classDef datastoreExt fill:#fde68a,stroke:#b45309,color:#1f1403,stroke-width:1px,stroke-dasharray: 5 4
    classDef analysis fill:#14b8a6,stroke:#0f766e,color:#05201c,stroke-width:1px
    classDef note fill:#e5e7eb,stroke:#6b7280,color:#111827,stroke-width:1px,stroke-dasharray: 3 3
```
