# adaptyvbio/proteinflow architecture

> Architecture diagram of the GitHub repository adaptyvbio/proteinflow, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-02-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/adaptyvbio/proteinflow
- Repository: https://github.com/adaptyvbio/proteinflow

## Overview

##### Project Type & Purpose
- ProteinFlow is not a traditional full-stack application or a service, but rather a specialized library and command-line tool for protein design data processing.
- The README highlights its use in downloading pre-computed datasets, generating new datasets with various filtering criteria, and even splitting/clustering data using external tools like MMseqs and rcsbsearch.
- Pipeline nature and modular design – it processes raw input (protein structures) and outputs processed datasets for deep learning.

##### File Structure Analysis
- The top-level directories include configuration files (.dockerignore, .gitignore, LICENSE, Dockerfile), CI/CD configuration (.github/workflows), and documentation folders (docs, pdoc). This indicates a well-organized project with automated testing, deployment, and clear documentation.
- The “proteinflow” directory contains the core modules:
- Entry point/CLI: proteinflow/cli.py (handles command-line interactions).
- Data Handling & Processing: Modules such as proteinflow/data, proteinflow/download, proteinflow/processing, and proteinflow/split set up the core processing pipeline.
- The “dev” scripts are used for tasks like version bumping and documentation updates.
- Tests directory provides unit testing ensuring functionality across modules.

##### Main Components
a. Command Line Interface:
- Component labeled “CLI (proteinflow/cli.py)” where user commands (download, generate, split) enter the system.
b. Core Processing Modules:
- “Data Handling and Conversion” module encompassing proteinflow/data and download capabilities (e.g., “Download Module” with submodule proteinflow/download using boto for S3 access).
- “Dataset Management” module: proteinflow/split for clustering and splitting datasets, including interactions with external tools like MMseqs2 and rcsbsearch.
c. Logging, Metrics, and Utilities:
- “Logging Module”: proteinflow/logging that captures pipeline activity.
- “Metrics Module”: proteinflow/metrics to compute feature metrics and validations.
d. External Dependencies & Integrations:
- Integrations with external packages (e.g., ProDy, MMseqs2, rcsbsearch) and data sources such as PDB and SAbDab.
- Docker containerization support via Dockerfile (representing a deployment/packaging component).
e. Documentation & Testing:
- “Documentation” block (docs, pdoc) that supports API and usage documentation.
- “Testing Framework” block (tests directory) ensuring module correctness.

##### Relationships & Interactions
- Data flows from external sources into the Download Module, then through data processing and feature extraction pipelines.
- Processed outputs are then handled by Dataset Management for splitting/clustering.
- Logging and Metrics modules monitor and report the operational status.

##### Architectural Patterns & Technologies
- The design follows a modular decomposition where each submodule focuses on a single aspect of the pipeline.
- The use of CLI as an entry point encourages a decoupled design between the user interface and the processing logic.
- Extensibility and the possibility of substituting external dependencies (e.g., different clustering algorithms or alternative visualization modules).
- Containerization aspect (Docker support) which aids in creating reproducible environments.

##### Additional Elements
- “Command-Line Interface” (the entry point).
- “Data Processing & Conversion” (integrating modules: data, processing, ligand, visualization).
- “Dataset Management & Clustering” (split module).
- “External Integrations” (MMseqs2, rcsbsearch, PDB/SAbDab, Docker).
- “Support & Utilities” (logging, metrics, tests, documentation).

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "ProteinFlow Library"
        CLI["CLI (proteinflow/cli.py)"]:::cliClass

        subgraph "Core Pipeline"
            subgraph "Data Handling & Conversion"
                DP["Data Processing Module (proteinflow/data)"]:::coreClass
                DM["Download Module (proteinflow/download)"]:::coreClass
            end
            subgraph "Processing & Feature Extraction"
                PM["Processing Module (proteinflow/processing)"]:::coreClass
                LP["Ligand Processing (proteinflow/ligand.py)"]:::coreClass
                VI["Visualization (proteinflow/visualize.py)"]:::coreClass
            end
            subgraph "Dataset Management"
                SM["Split Module (proteinflow/split)"]:::coreClass
            end
        end
    end

    subgraph "Logging & Metrics"
        LM["Logging Module (proteinflow/logging)"]:::supportClass
        MT["Metrics Module (proteinflow/metrics)"]:::supportClass
    end

    subgraph "External Integrations & Deployment"
        EX["External Tools (MMseqs2,rcsbsearch,PDB,SAbDab)"]:::externalClass
        DC["Docker Container (Dockerfile)"]:::externalClass
    end

    subgraph "Documentation & Testing"
        DOC["Documentation (docs,pdoc)"]:::cliClass
        TEST["Testing Framework (tests)"]:::supportClass
    end

    %% Pipeline Flow
    CLI -->|"initiate"| DP
    CLI -->|"initiate"| DM
    DP -->|"prepares"| PM
    DM -->|"supplies_data"| PM
    PM -->|"transforms"| SM

    %% Logging & Metrics connections
    DP -->|"logs"| LM
    DM -->|"logs"| LM
    PM -->|"logs"| LM
    SM -->|"logs"| LM
    DP -->|"metrics"| MT
    DM -->|"metrics"| MT
    PM -->|"metrics"| MT
    SM -->|"metrics"| MT

    %% External Integrations
    EX -->|"fetches"| DM
    EX -->|"clusters"| SM
    DC ---|"container_env"| CLI
    DC ---|"container_env"| DP
    DC ---|"container_env"| PM
    DC ---|"container_env"| SM

    %% Documentation & Testing Connections
    CLI ---|"usage_info"| DOC
    TEST ---|"verifies"| CLI

    %% Styles
    classDef cliClass fill:#FFCC00,stroke:#333,stroke-width:2px;
    classDef coreClass fill:#66CCFF,stroke:#333,stroke-width:2px;
    classDef externalClass fill:#C5E1A5,stroke:#333,stroke-width:2px;
    classDef supportClass fill:#FFAB91,stroke:#333,stroke-width:2px;

    %% Click Events
    click CLI "https://github.com/adaptyvbio/proteinflow/blob/main/proteinflow/cli.py"
    click DP "https://github.com/adaptyvbio/proteinflow/tree/main/proteinflow/data"
    click DM "https://github.com/adaptyvbio/proteinflow/blob/main/proteinflow/download/boto.py"
    click PM "https://github.com/adaptyvbio/proteinflow/blob/main/proteinflow/processing/__init__.py"
    click LP "https://github.com/adaptyvbio/proteinflow/blob/main/proteinflow/ligand.py"
    click VI "https://github.com/adaptyvbio/proteinflow/blob/main/proteinflow/visualize.py"
    click SM "https://github.com/adaptyvbio/proteinflow/tree/main/proteinflow/split"
    click LM "https://github.com/adaptyvbio/proteinflow/blob/main/proteinflow/logging/__init__.py"
    click MT "https://github.com/adaptyvbio/proteinflow/blob/main/proteinflow/metrics/__init__.py"
    click DC "https://github.com/adaptyvbio/proteinflow/tree/main/Dockerfile"
    click DOC "https://github.com/adaptyvbio/proteinflow/tree/main/docs"
    click TEST "https://github.com/adaptyvbio/proteinflow/tree/main/tests"
```
