How alleninstitute/micronsbinder works
Project type and purpose
• This is a teaching‐tool repository (“MicronsBinder”) comprised of Jupyter notebooks for interactive analysis of large‐scale connectomics data (EM reconstructions and two‐photon calcium imaging).
• It is designed to launch on mybinder.org (via a Docker container) or locally via a conda environment. Users explore data through notebooks that pull from cloud services, run analyses, and produce figures.
File structure
• Top‐level config:
• .gitignore, .travis.yml, LICENSE, CONTRIBUTING.md
• environment.yml, apt.txt (binder requirements)
• postBuild (runs after binder build)
• Makefile (data‐download orchestration)
• notebooks/ directory broken into:
• intro/ (Phase 1 demo notebooks)
• mm3_intro/ (Phase 3 introductory notebooks with CAVE setup, mesh access, synapse queries)
• vignette_analysis/ (deeper analyses in subfolders function, mitochondria, motifs, resource_statistics)
• Within each vignette_analysis subfolder:
• lib/ (shared Python modules)
• scripts/ (data‐prep and compute scripts)
• assets/ or figures/ (output)
README for additional insights
• Describes the binder build process, Python 3.7+ conda environment, and how notebooks connect to:
• CAVE services (Connectome Annotation Versioning Engine)
• CloudVolume (volumetric data, meshes, skeletons)
• MeshParty (mesh download, analysis, visualization via VTK)
• Neuroglancer (3D web visualization)
• DashDataFrame/Dash for interactive plots
Main components
a. Binder build environment
• Dockerfile (implied by binder), apt.txt → apt packages
• environment.yml → conda environment
• postBuild script
b. JupyterLab front‐end
• Notebook UI where users launch and run .ipynb files
c. Notebook code
• notebooks/intro, mm3_intro, vignette_analysis/*
• lib/ modules and scripts for data processing
• Makefile targets for bulk data download
d. Data access layer
• CAVE REST API (annotation/versioning)
• CloudVolume API (EM images, segmentation, meshes, skeletons)
e. Analysis & visualization libraries
• MeshParty, trimesh, scipy, vtk
• neuroglancer (JS viewer), DashDataFrame / Dash
• pandas, matplotlib, plotly
f. External services & storage
• AWS/GCP buckets or cloud storage hosting micrograph volumes and segmentation
• Binder service (mybinder.org) or local conda environment
Component relationships & data flows
• Binder build → Docker container → conda env → JupyterLab
• User opens a notebook → notebook loads Python libs → calls CAVE & CloudVolume → fetches raw/segmented volumes, meshes, skeletons
• Data flows into lib/ scripts for processing (e.g. compute_degree.py) → outputs saved under assets/ or figures/
• Visualization routines call neuroglancer (link generation) or render locally with VTK/Dash
• Makefile orchestrates bulk data downloads into notebook subfolders
Architectural patterns & design principles
• Separation of concerns:
• Environment provisioning (environment.yml, apt.txt, postBuild)
• Interactive layer (Jupyter notebooks)
• Data‐access layer (CloudVolume, CAVE)
• Analysis modules (lib/), scripts (compute_*.py)
• Output storage (assets/, figures/)
• Declarative orchestration:
• Makefile targets for reproducible data fetch
• Modular code:
• Python packages in lib/ usable by multiple notebooks
• Containerized reproducibility:
• Binder build ensures identical Python environment on cloud and local