# ptycho/ptypy architecture

> Architecture diagram of the GitHub repository ptycho/ptypy, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ptycho/ptypy
- Repository: https://github.com/ptycho/ptypy

## Overview

##### Project Type and Overall Purpose
- PtyPy is a scientific framework for ptychography reconstruction. It is designed to cover the entire workflow – from data management and input/output through reconstruction algorithms to visualization.
- It supports various reconstruction “engines” (e.g., Difference Map, Maximum Likelihood, RAAR, sDR, ePIE) and utilizes advanced features such as GPU acceleration (via custom CUDA kernels, CuPy, PyCUDA/reikna) and parallel processing using MPI.
- The project is modular and highly extensible, catering to both simulation and real experimental data processing.

##### File Structure to Identify Main Components
- Core Library Components: Look at the “ptypy” directory. This contains:
– ptypy/core: Contains core classes and data management, geometry, and fundamental ptychography algorithms.
– ptypy/io: Manages data file input/output, conversion, and interfacing with various file formats.
– ptypy/resources: Holds default parameter configurations.
- Reconstruction Engines: In “ptypy/engines” and also in “archive/engines” and “templates/engines”. These modules are the implementations of different reconstruction algorithms (DM, ML, etc.) and their variants.
- Acceleration and GPU Support:
– “ptypy/accelerate” and its subdirectories (cuda_common, cuda_cupy, cuda_pycuda, ocl_pyopencl) indicate layers abstracting GPU-based operations, interfacing with CUDA platforsm, and providing accelerated kernels.
– Additionally, the “cufft” directory contains a filtered FFT implementation for GPU.
- Experiment and Simulation: “ptypy/experiment” and “ptypy/simulations” focus on experiment-specific configurations and synthetic testing scenarios.
- Command-Line and Visualization: “ptypy/cli” implements the command-line interface, including modules for launching reconstruction jobs and possibly integrating ZeroMQ based remote visualization.
- Auxiliary Areas:
– “templates” holds sample scripts and example configurations that demonstrate how all components fit together.
– “benchmark” and “test” directories indicate robust testing and performance evaluation infrastructure.

##### Relationships and Interactions
- Data Flow: Data enters through the I/O modules (ptypy/io) and resource/configuration modules in ptypy/resources. The data is then passed to core processing routines (ptypy/core) that orchestrate the overall reconstruction.
- Engine Processing: Reconstruction “engines” (in ptypy/engines, archive/engines, etc.) take the prepared data and feed it through iterative algorithms. They are designed in a modular way to allow switching between approaches (DM, ML, etc.) and can leverage GPU acceleration modules.
- Acceleration Layer: Certain modules in ptypy/accelerate serve as an abstraction layer so that the reconstruction engines can tap into CUDA kernels (via cuda_common, cuda_cupy, or cuda_pycuda) without rewriting algorithm logic. This layer also handles integration with external libraries (like CuPy, PyCUDA).
- Control and Visualization: The CLI (ptypy/cli) and the ZeroMQ-based client-server components (as mentioned in the README) display progress and enable on-the-fly control of the reconstruction. They interact with the core engine processes.

##### Architectural Patterns and Key Technologies
- Modular Abstraction: Nearly every key concept (data management, reconstruction engine, GPU acceleration, etc.) is encapsulated in its own module or package; this enables extensibility and separation of concerns.
- Multiple Implementations: Many components (engines, GPU acceleration paths) have alternative implementations supported by conditional configuration. Demonstrate these as interchangeable blocks.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data & Config Layer (Bottom)
    subgraph "Data & Config"
        IO["Data I/O"]:::data
        RES["Resources"]:::data
    end

    %% Core Processing Layer
    subgraph "Core Processing"
        CORE["Core Processing"]:::core
    end

    %% Reconstruction Engines Layer
    subgraph "Reconstruction Engines"
        PrimaryEng["Primary Engines"]:::engine
        ArchiveEng["Archive Engines"]:::engine
        TemplatesEng["Template Engines"]:::engine
    end

    %% GPU/Acceleration Layer
    subgraph "GPU/Acceleration"
        ACC["General Acceleration"]:::acceleration
        CUDACommon["CUDA Common"]:::acceleration
        CUDACupy["CUDA CuPy"]:::acceleration
        CUDAPyCUDA["CUDA PyCUDA"]:::acceleration
        OpenCL["OpenCL Integration"]:::acceleration
        FFT["Filtered FFT"]:::acceleration
    end

    %% Experiment & Simulations Layer
    subgraph "Experiment & Simulations"
        EXP["Experiment Setups"]:::experiment
        SIM["Simulated Processing"]:::experiment
    end

    %% CLI & Visualization Layer (Top)
    subgraph "CLI & Visualization"
        CLI["CLI & Visualization"]:::cli
    end

    %% Auxiliary Areas
    subgraph "Auxiliary"
        TEMPLATES["Templates & Samples"]:::auxiliary
        BENCHMARK["Benchmarking & Testing"]:::auxiliary
        TEST["Tests"]:::auxiliary
    end

    %% Communications (External)
    subgraph "Communications"
        MPI["MPI Communication"]:::communication
        ZMQ["ZeroMQ Channel"]:::communication
    end

    %% Connections between layers
    IO --> CORE
    RES --> CORE

    CORE -->|"processes"| PrimaryEng
    CORE -->|"processes"| ArchiveEng
    CORE -->|"processes"| TemplatesEng

    PrimaryEng -->|"acceleratedBy"| ACC
    ArchiveEng -->|"acceleratedBy"| ACC
    TemplatesEng -->|"acceleratedBy"| ACC

    ACC --> CUDACommon
    ACC --> CUDACupy
    ACC --> CUDAPyCUDA
    ACC --> OpenCL
    ACC --> FFT

    CORE -->|"configures"| EXP
    CORE -->|"configures"| SIM

    CLI -->|"controls"| CORE

    %% Communications with core and engines
    CORE --- MPI
    PrimaryEng --- MPI
    ArchiveEng --- MPI
    TemplatesEng --- MPI
    CLI --- ZMQ

    %% Auxiliary connections (reference/support)
    TEMPLATES --- CLI
    BENCHMARK --- CORE
    TEST --- CORE

    %% Class definitions for colors
    classDef data fill:#cce5ff,stroke:#004085,stroke-width:2px;
    classDef core fill:#d4edda,stroke:#155724,stroke-width:2px;
    classDef engine fill:#fff3cd,stroke:#856404,stroke-width:2px;
    classDef acceleration fill:#f8d7da,stroke:#721c24,stroke-width:2px;
    classDef experiment fill:#d1ecf1,stroke:#0c5460,stroke-width:2px;
    classDef cli fill:#e2e3e5,stroke:#6c757d,stroke-width:2px;
    classDef auxiliary fill:#fefefe,stroke:#6c757d,stroke-width:2px;
    classDef communication fill:#e6d3f8,stroke:#663399,stroke-width:2px;

    %% Click Events
    click IO "https://github.com/ptycho/ptypy/tree/master/ptypy/io"
    click RES "https://github.com/ptycho/ptypy/tree/master/ptypy/resources"
    click CORE "https://github.com/ptycho/ptypy/tree/master/ptypy/core"
    click PrimaryEng "https://github.com/ptycho/ptypy/tree/master/ptypy/engines"
    click ArchiveEng "https://github.com/ptycho/ptypy/tree/master/archive/engines"
    click TemplatesEng "https://github.com/ptycho/ptypy/tree/master/templates/engines"
    click ACC "https://github.com/ptycho/ptypy/tree/master/ptypy/accelerate"
    click CUDACommon "https://github.com/ptycho/ptypy/tree/master/ptypy/accelerate/cuda_common"
    click CUDACupy "https://github.com/ptycho/ptypy/tree/master/ptypy/accelerate/cuda_cupy"
    click CUDAPyCUDA "https://github.com/ptycho/ptypy/tree/master/ptypy/accelerate/cuda_pycuda"
    click OpenCL "https://github.com/ptycho/ptypy/tree/master/ptypy/accelerate/ocl_pyopencl"
    click FFT "https://github.com/ptycho/ptypy/tree/master/cufft/filtered_fft"
    click EXP "https://github.com/ptycho/ptypy/tree/master/ptypy/experiment"
    click SIM "https://github.com/ptycho/ptypy/tree/master/ptypy/simulations"
    click CLI "https://github.com/ptycho/ptypy/tree/master/ptypy/cli"
    click TEMPLATES "https://github.com/ptycho/ptypy/tree/master/templates"
    click BENCHMARK "https://github.com/ptycho/ptypy/tree/master/benchmark"
    click TEST "https://github.com/ptycho/ptypy/tree/master/test"
```
