# pnnl/neuromancer architecture

> Architecture diagram of the GitHub repository pnnl/neuromancer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pnnl/neuromancer
- Repository: https://github.com/pnnl/neuromancer

## Overview

##### Project Type and Purpose
- NeuroMANCER is an open-source Python library for differentiable programming, physics-informed system identification, parametric optimization, and model-based control.
- It is not a client-server or web app but a modular software toolkit built on PyTorch (and optionally PyTorch Lightning, TorchSDE, cvxpylayers, etc.).

##### High-Level File-Structure Analysis
- **src/neuromancer**: the core Python package, subdivided into subpackages:
- analysis, arg, callbacks, constraint, dataset, dynamics, gradients, loggers, loss, modules, psl, slim, system, trainer, utils, plot, problem
- **assistant/**: helper scripts for generating RAG-friendly embeddings of the code/docs for an LLM assistant.
- **docs/**: Sphinx documentation source.
- **examples/**: Jupyter notebooks and scripts demonstrating use cases (ODEs, PDEs, control, KANs, etc.).
- **tests/**: unit tests validating core functionality.
- Configuration files (pyproject.toml, environment.yml), CI workflows, and markdown guides round out the repo.

##### Key Insights from README
- Core abstractions: **Variable**, **Node**, **System**, **Problem**, **Loss**, **Trainer**
- Integration points: PyTorch for tensor ops, PyTorch Lightning for training loop, optional TorchSDE for SDEs, cvxpylayers for convex layers
- Example workflows: define symbolic variables → wrap modules in Nodes → assemble into a System/Problem → hand off to Trainer → optimize parameters

##### Components
- PyTorch
- PyTorch Lightning
- TorchSDE (optional)
- cvxpylayers (optional)
1. **Constraint & Variable Engine** (`constraint.py`)
2. **Symbolic System Builder** (`system.py`, Node, System classes)
3. **Neural Modules** (`modules/blocks.py`, `modules/activations.py`, `modules/rnn.py`, `slim/`, etc.)
4. **Dynamics Library** (`dynamics/ode.py`, `dynamics/sde.py`, `dynamics/physics.py`, etc.)
5. **Loss & Objective Manager** (`loss.py`, `problem.py`)
6. **Trainer** (`trainer.py`, Lightning‐based training logic)
7. **Callbacks & Logging** (`callbacks.py`, `loggers.py`)
8. **Analysis Tools** (`analysis/pwa_maps.py`, etc.)
c. **Data Layer** (`dataset.py` and user data): feeding training/validation batches
d. **User Code / Examples**: imports the Core API, defines Variables, Nodes, Systems, Problems, calls Trainer
e. **Documentation & Assistant**: ingest scripts under `assistant/`, Sphinx docs under `docs/` (not in runtime path but part of the ecosystem)

##### Interactions and Data-Flow
- **User** → defines **Variables** & **Problem** using **Modules** →
- **System Builder** composes **Nodes** into a computation graph →
- **Trainer** pulls **batches** from **Dataset**, runs **forward** on System →
- **Loss Manager** evaluates objectives & constraints →
- **Backward/Optimizer** (PyTorch) updates module parameters →
- **Callbacks/Loggers** record metrics →
- **Analysis Tools** can be invoked at any stage for post-hoc evaluation (e.g., PWA map analysis)

##### Architectural Patterns & Principles
- **Modularity**: each subpackage encapsulates a concern (e.g., modules vs. dynamics vs. loss).
- **Symbolic Programming**: Variables and Nodes allow for late binding of computation graphs.
- **Plugin‐style Extensibility**: external convex solvers, TorchSDE, custom modules can be slotted in.
- **Separation of Concerns**: modeling (System), optimization formulation (Problem/Loss), and execution (Trainer) are cleanly separated.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External Libraries
    subgraph "External Libraries"
        pytorch["PyTorch"]:::external
        lightning["PyTorch Lightning"]:::external
        tsde["TorchSDE (optional)"]:::external
        cvxp["cvxpylayers (optional)"]:::external
    end

    %% Core Components
    subgraph "Neuromancer Core"
        constraint["Constraint & Variable Engine"]:::core
        system["Symbolic System Builder"]:::core
        modules["Neural Modules"]:::core
        dynamics["Dynamics Library"]:::core
        loss["Loss & Objective Manager"]:::core
        trainer["Trainer"]:::core
        callbacks["Callbacks & Logging"]:::core
        analysis["Analysis Tools"]:::core
    end

    %% Data Layer
    dataset["Data Layer (Dataset)"]:::data

    %% User and Documentation
    user["User Code / Examples"]:::user
    docs["Documentation & Assistant Scripts"]:::docs

    %% Data and Control Flow
    user -->|"defines Variables & Problems"| constraint
    constraint -->|"provides Variables"| system
    modules -->|"wraps Modules"| system
    dynamics -->|"provides ODE/SDE/Physics"| system
    system -->|"builds Computation Graph"| trainer
    dataset -->|"feeds batches"| trainer
    trainer -->|"calls forward()"| system
    trainer -->|"computes loss"| loss
    loss -->|"uses constraints & objectives"| constraint
    trainer -->|"updates params"| pytorch
    trainer -->|"triggers callbacks"| callbacks
    trainer -->|"logs metrics"| callbacks
    trainer -->|"post-hoc evaluation"| analysis

    %% Dependency Arrows
    pytorch --> system
    pytorch --> dynamics
    lightning --> trainer
    tsde --> dynamics
    cvxp --> loss

    %% Click Events
    click constraint "https://github.com/pnnl/neuromancer/blob/master/src/neuromancer/constraint.py"
    click system "https://github.com/pnnl/neuromancer/blob/master/src/neuromancer/system.py"
    click modules "https://github.com/pnnl/neuromancer/blob/master/src/neuromancer/modules/blocks.py"
    click modules "https://github.com/pnnl/neuromancer/blob/master/src/neuromancer/modules/activations.py"
    click modules "https://github.com/pnnl/neuromancer/blob/master/src/neuromancer/modules/rnn.py"
    click modules "https://github.com/pnnl/neuromancer/tree/master/src/neuromancer/slim/"
    click dynamics "https://github.com/pnnl/neuromancer/blob/master/src/neuromancer/dynamics/ode.py"
    click dynamics "https://github.com/pnnl/neuromancer/blob/master/src/neuromancer/dynamics/sde.py"
    click dynamics "https://github.com/pnnl/neuromancer/blob/master/src/neuromancer/dynamics/physics.py"
    click dynamics "https://github.com/pnnl/neuromancer/blob/master/src/neuromancer/dynamics/integrators.py"
    click dynamics "https://github.com/pnnl/neuromancer/blob/master/src/neuromancer/dynamics/interpolation.py"
    click dynamics "https://github.com/pnnl/neuromancer/blob/master/src/neuromancer/dynamics/library.py"
    click loss "https://github.com/pnnl/neuromancer/blob/master/src/neuromancer/loss.py"
    click loss "https://github.com/pnnl/neuromancer/blob/master/src/neuromancer/problem.py"
    click trainer "https://github.com/pnnl/neuromancer/blob/master/src/neuromancer/trainer.py"
    click callbacks "https://github.com/pnnl/neuromancer/blob/master/src/neuromancer/callbacks.py"
    click callbacks "https://github.com/pnnl/neuromancer/blob/master/src/neuromancer/loggers.py"
    click analysis "https://github.com/pnnl/neuromancer/blob/master/src/neuromancer/analysis/pwa_maps.py"
    click dataset "https://github.com/pnnl/neuromancer/blob/master/src/neuromancer/dataset.py"
    click user "https://github.com/pnnl/neuromancer/tree/master/examples/"
    click docs "https://github.com/pnnl/neuromancer/tree/master/docs/"
    click docs "https://github.com/pnnl/neuromancer/tree/master/assistant/"

    %% Styles
    classDef external fill:#fdebd0,stroke:#d35400,stroke-width:1px
    classDef core fill:#d6eaf8,stroke:#2e86c1,stroke-width:1px
    classDef data fill:#d5f5e3,stroke:#27ae60,stroke-width:1px
    classDef user fill:#fcf3cf,stroke:#b7950b,stroke-width:1px
    classDef docs fill:#f5eef8,stroke:#8e44ad,stroke-width:1px
```
