# chembl/fpsim2 architecture

> Architecture diagram of the GitHub repository chembl/fpsim2, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/chembl/fpsim2
- Repository: https://github.com/chembl/fpsim2

## Overview

After reviewing the file tree and README, it’s clear that FPSim2 is a high-performance molecular similarity search package with a Python/C++ hybrid architecture. Its main purpose is to quickly search for chemical compound similarities (with an emphasis on high similarity thresholds) using optimized computation approaches like CPU POPCNT and GPU acceleration, as well as providing efficient I/O backends.

##### Project’s Main Components
- Core API Layer: Represented by the FPSim2 package (files like FPSim2.py, FPSim2Cuda.py, base.py) which serves as the public API.
- Performance Engine: The C++ source code under the “src” directory (sim.cpp, utils.cpp, wraps.cpp, and the corresponding header files in src/include) that is wrapped and called by Python for high-performance computations.
- I/O Subsystem: This includes the “io” directory with its backends (pytables and sqla) and additional modules like “chem.py” for handling chemical file formats. This layer abstracts data input and output, optimized for disk-based databases.
- Optional/Extension Modules: The “contrib” folder, especially the “isim” submodule, which appears to provide additional similarity computation features.
- Utility/Script Layer: Scripts like “create_fpsim2_fp_db.py” for creating databases, which help set up or pre-process data.
- Testing & CI/Deployment Infrastructure: The “tests” directory, along with the .github/workflows configuration (for CI, docs deployment, wheels, etc.), ensuring reliability and continuous integration.
- Documentation: The “docs” directory and “mkdocs.yml” provide user guides and in-depth technical documentation.

##### Relationships & Data Flow
- The API also leverages the I/O Subsystem to read and write data. Represent connections from the API into both the “backends” (e.g., pytables and sqla) and the “chem” module.
- CI/CD workflows and tests ensure every component functions correctly. While these may not be “runtime” components, note them as supporting infrastructure for development and deployment.

##### Architectural Patterns & Design Principles
- Layered Architecture: Illustrate the separation between the high-level Python API and the low-level C++ performance engine.
- Modular Design: Emphasize distinct, interchangeable modules (I/O backends, GPU vs. CPU computation, external tool integration via contrib) which allow for easy extensibility.
- Performance Optimization: Note the use of CPU-specific instructions (POPCNT) and GPU acceleration, which are central performance characteristics of the system.
- Testing & CI Integration: Indicate that automated testing and continuous integration/deployment are integral to ensuring quality.

##### Technologies, Frameworks, and Libraries
- Python as the primary language for API and glue code.
- C++ for performance-sensitive computation (with header files and source code wrappers).
- RDKit for chemical informatics.
- NumPy for efficient numerical operations.
- PyTables, BLOSC2 (as hinted by the README) for fast, compressed file I/O.
- SQLAlchemy (sqla backend) for relational database support.
- GitHub Actions for CI workflows.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Core API Layer
    subgraph "Core API Layer"
        API1["FPSim2.py"]:::api
        API2["FPSim2Cuda.py"]:::api
        API3["base.py"]:::api
    end

    %% Performance Engine (C++ Modules)
    subgraph "Performance Engine (C++ Modules)"
        PE1["sim.cpp"]:::cpp
        PE2["utils.cpp"]:::cpp
        PE3["wraps.cpp"]:::cpp
        PE4["C++ Headers"]:::cpp
    end

    %% I/O Subsystem
    subgraph "I/O Subsystem"
        IO1["pytables.py"]:::io
        IO2["sqla.py"]:::io
        IO3["chem.py"]:::io
    end

    %% Optional/Extension Modules
    subgraph "Optional/Extension Modules"
        EXT1["isim_comp.py"]:::ext
        EXT2["utils.py"]:::ext
    end

    %% Utility & Support Infrastructure
    subgraph "Utility & Support Infrastructure"
        UTIL["create_fpsim2_fp_db.py"]:::util
        CI["Tests & CI Deployment"]:::ci
        DOC["Documentation"]:::doc
    end

    %% Relationships and Data Flow
    API1 -->|"calls"| PE1
    API2 -->|"calls"| PE1
    API3 -->|"calls"| PE1
    API1 -->|"I/O-readwrite"| IO3
    API2 -->|"I/O-readwrite"| IO1
    API3 -->|"I/O-readwrite"| IO2
    UTIL -->|"sets-up DB"| IO1
    UTIL -->|"sets-up DB"| IO2
    API1 -->|"extended-by"| EXT1

    CI ---|"monitors"| API1
    CI ---|"monitors"| PE1
    CI ---|"monitors"| IO1
    DOC ---|"guides"| API1

    %% Click Events for Core API Layer
    click API1 "https://github.com/chembl/fpsim2/blob/master/FPSim2/FPSim2.py"
    click API2 "https://github.com/chembl/fpsim2/blob/master/FPSim2/FPSim2Cuda.py"
    click API3 "https://github.com/chembl/fpsim2/blob/master/FPSim2/base.py"

    %% Click Events for Performance Engine
    click PE1 "https://github.com/chembl/fpsim2/blob/master/FPSim2/src/sim.cpp"
    click PE2 "https://github.com/chembl/fpsim2/blob/master/FPSim2/src/utils.cpp"
    click PE3 "https://github.com/chembl/fpsim2/blob/master/FPSim2/src/wraps.cpp"
    click PE4 "https://github.com/chembl/fpsim2/tree/master/FPSim2/src/include"

    %% Click Events for I/O Subsystem
    click IO1 "https://github.com/chembl/fpsim2/blob/master/FPSim2/io/backends/pytables.py"
    click IO2 "https://github.com/chembl/fpsim2/blob/master/FPSim2/io/backends/sqla.py"
    click IO3 "https://github.com/chembl/fpsim2/blob/master/FPSim2/io/chem.py"

    %% Click Events for Optional/Extension Modules
    click EXT1 "https://github.com/chembl/fpsim2/blob/master/FPSim2/contrib/isim/isim_comp.py"
    click EXT2 "https://github.com/chembl/fpsim2/blob/master/FPSim2/contrib/isim/utils.py"

    %% Click Event for Utility/Script Layer
    click UTIL "https://github.com/chembl/fpsim2/blob/master/FPSim2/scripts/create_fpsim2_fp_db.py"

    %% Click Events for Testing & CI and Documentation
    click CI "https://github.com/chembl/fpsim2/tree/master/tests"
    click DOC "https://github.com/chembl/fpsim2/tree/master/docs"

    %% Styles
    classDef api fill:#ADD8E6,stroke:#000,stroke-width:2px;
    classDef cpp fill:#90EE90,stroke:#000,stroke-width:2px;
    classDef io fill:#FFFFE0,stroke:#000,stroke-width:2px;
    classDef ext fill:#FFB6C1,stroke:#000,stroke-width:2px;
    classDef util fill:#DDA0DD,stroke:#000,stroke-width:2px;
    classDef ci fill:#F4A460,stroke:#000,stroke-width:2px;
    classDef doc fill:#E0FFFF,stroke:#000,stroke-width:2px;
```
