How chembl/fpsim2 works
After reviewing the file tree and README, it’s clear that FPSim2 is a high-performance molecular similarity search package with a Python/C++ hybrid architecture. Its main purpose is to quickly search for chemical compound similarities (with an emphasis on high similarity thresholds) using optimized computation approaches like CPU POPCNT and GPU acceleration, as well as providing efficient I/O backends.
Project’s Main Components
• Core API Layer: Represented by the FPSim2 package (files like FPSim2.py, FPSim2Cuda.py, base.py) which serves as the public API.
• Performance Engine: The C++ source code under the “src” directory (sim.cpp, utils.cpp, wraps.cpp, and the corresponding header files in src/include) that is wrapped and called by Python for high-performance computations.
• I/O Subsystem: This includes the “io” directory with its backends (pytables and sqla) and additional modules like “chem.py” for handling chemical file formats. This layer abstracts data input and output, optimized for disk-based databases.
• Optional/Extension Modules: The “contrib” folder, especially the “isim” submodule, which appears to provide additional similarity computation features.
• Utility/Script Layer: Scripts like “create_fpsim2_fp_db.py” for creating databases, which help set up or pre-process data.
• Testing & CI/Deployment Infrastructure: The “tests” directory, along with the .github/workflows configuration (for CI, docs deployment, wheels, etc.), ensuring reliability and continuous integration.
• Documentation: The “docs” directory and “mkdocs.yml” provide user guides and in-depth technical documentation.
Relationships & Data Flow
• The API also leverages the I/O Subsystem to read and write data. Represent connections from the API into both the “backends” (e.g., pytables and sqla) and the “chem” module.
• CI/CD workflows and tests ensure every component functions correctly. While these may not be “runtime” components, note them as supporting infrastructure for development and deployment.
Architectural Patterns & Design Principles
• Layered Architecture: Illustrate the separation between the high-level Python API and the low-level C++ performance engine.
• Modular Design: Emphasize distinct, interchangeable modules (I/O backends, GPU vs. CPU computation, external tool integration via contrib) which allow for easy extensibility.
• Performance Optimization: Note the use of CPU-specific instructions (POPCNT) and GPU acceleration, which are central performance characteristics of the system.
• Testing & CI Integration: Indicate that automated testing and continuous integration/deployment are integral to ensuring quality.
Technologies, Frameworks, and Libraries
• Python as the primary language for API and glue code.
• C++ for performance-sensitive computation (with header files and source code wrappers).
• RDKit for chemical informatics.
• NumPy for efficient numerical operations.
• PyTables, BLOSC2 (as hinted by the README) for fast, compressed file I/O.
• SQLAlchemy (sqla backend) for relational database support.
• GitHub Actions for CI workflows.