# amuizab/protein-3d-structure-pipeline-web-interface architecture

> Architecture diagram of the GitHub repository amuizab/protein-3d-structure-pipeline-web-interface, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-02-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/amuizab/protein-3d-structure-pipeline-web-interface
- Repository: https://github.com/amuizab/protein-3d-structure-pipeline-web-interface

## Overview

##### Project Type & Purpose
- The project is a hybrid application with two distinct parts:
- A data pipeline that processes protein gene data, maps gene names to UniProt IDs, generates validated URLs, downloads PDB files, manages metadata via an SQLite database, and finally uploads the files to Google Cloud Storage.
- The overall purpose is to collect and process protein structure data and let users explore the structures via a web interface deployed on Google Cloud Run.

##### Analyzing the File Structure
- The top-level project contains two main directories:
- data_pipelines: Contains submodules like "datagenes", "downloadpdb", and "map2uniprot". Each submodule is responsible for a step in the pipeline:
▸ datagenes: Likely handles gene name collection and preprocessing.
▸ map2uniprot: Manages the extraction of UniProt IDs from gene names.
▸ downloadpdb: Responsible for downloading PDB files based on provided URLs.
▸ Additionally, there are main scripts (main.py, uploadtoGCS.py) and a Jupyter notebook (projectProteins.ipynb) to orchestrate pipeline execution.
- web_server: Contains the web interface code including a Docker configuration (Dockerfile.txt), HTML file (index.html), and Python script (main.py) that sets up the server and likely integrates the Mol* viewer.
- Other files: requirements.txt lists dependencies important for both backend and frontend operations.

##### Key Architectural Components
- Data Pipeline Components:
- Gene Data Processing Module: Represented by the “datagenes” component.
- UniProt ID Mapping Module: Represented by the “map2uniprot” component.
- PDB Download Module: Represented by the “downloadpdb” component.
- Orchestration and Execution Flow: The main.py and the Jupyter Notebook, which coordinate the pipeline steps.
- Storage Operation: The SQLite database (for metadata) and uploadtoGCS.py which handles uploading to Google Cloud Storage.
- Web Server Components:
- Frontend Interface: HTML (index.html) for user interaction, including search and display elements.
- Backend Service: main.py in the web_server directory. This module manages API routes, integrates with the SQLite database for metadata, and loads PDB files (via links to Google Cloud Storage).
- Containerization: Dockerfile.txt and dockerignore.txt, which package the web server for deployment to Google Cloud Run.
- External Systems:
- Google Cloud Storage: Holds the PDB files.
- Google Cloud Run: Deployment platform for the web interface.
- SQLite Database: Local database holding metadata and relationships.

##### Elements
- Clear and descriptive labels for each component (e.g., “Gene Processing”, “UniProt Mapping”, “Downloader”, “Uploader to GCS”, “SQLite Metadata DB”, “Web API Server”, “Mol* Viewer UI”).

##### Final Advice
- Be as detailed as possible when separating the architecture into its core modules. The more granular you are, the clearer it will be for stakeholders to understand the multistage pipeline and its integration with the web service.
- Avoid oversimplifying; given the project’s complexity (over 17,300 PDB files processed, multiple integration points with Google Cloud, and real-time web interaction for 3D visualization), it is critical that every key component and dependency is noted.

## Diagram (Mermaid)

```mermaid
flowchart TD

    %% Data Pipeline Section
    subgraph "Data Pipeline"
        P_Orch("Pipeline Orchestrator (Jupyter Notebook & main.py)"):::pipeline
        DG("Gene Collection & Processing (datagenes)"):::pipeline
        UP("UniProt ID Mapping (map2uniprot)"):::pipeline
        PD("PDB Download (downloadpdb)"):::pipeline
        DB("SQLite Metadata DB"):::external
        CU("Cloud Upload to GCS (uploadtoGCS)"):::pipeline
    end

    %% Web Server Section
    subgraph "Web Server"
        WS_Back("Web Server Backend (main.py)"):::web
        WS_Front("Web Server Frontend (index.html & Mol* Viewer)"):::web
        Cont("Containerization (Docker)"):::container
    end

    %% External Services Section
    subgraph "External Services"
        GCS("Google Cloud Storage (PDB Files)"):::external
        GCR("Google Cloud Run (Deployment)"):::external
    end

    %% Data Pipeline Flow
    P_Orch -->|"initiates"| DG
    DG -->|"sendsGeneData"| UP
    UP -->|"mapsIDs"| PD
    PD -->|"downloadsPDB"| DB
    PD -->|"downloadsPDB"| CU
    CU -->|"uploadsFiles"| GCS

    %% Web Server Flow
    DB -->|"metadataQuery"| WS_Back
    GCS -->|"retrievesPDB"| WS_Back
    WS_Back -->|"servesData"| WS_Front
    WS_Back -->|"packagedVia"| Cont
    Cont -->|"deployedOn"| GCR

    %% Legend
    subgraph "Legend"
        L1["Rectangle: Module/Service"]:::pipeline
        L2["Cylinder: Database"]:::external
        L3["Cloud: External Service"]:::external
        L4["Rounded Rectangle: Containerization"]:::container
        L5["Solid Arrow: DataFlow"]
    end

    %% Click Events
    click DG "https://github.com/amuizab/protein-3d-structure-pipeline-web-interface/blob/main/data_pipelines/datagenes/datagenes.py"
    click UP "https://github.com/amuizab/protein-3d-structure-pipeline-web-interface/blob/main/data_pipelines/map2uniprot/map2uniprot.py"
    click PD "https://github.com/amuizab/protein-3d-structure-pipeline-web-interface/blob/main/data_pipelines/downloadpdb/downloadpdb.py"
    click P_Orch "https://github.com/amuizab/protein-3d-structure-pipeline-web-interface/blob/main/data_pipelines/main.py"
    click P_Orch "https://github.com/amuizab/protein-3d-structure-pipeline-web-interface/blob/main/data_pipelines/projectProteins.ipynb"
    click CU "https://github.com/amuizab/protein-3d-structure-pipeline-web-interface/blob/main/data_pipelines/uploadtoGCS.py"
    click WS_Back "https://github.com/amuizab/protein-3d-structure-pipeline-web-interface/blob/main/web_server/main.py"
    click WS_Front "https://github.com/amuizab/protein-3d-structure-pipeline-web-interface/blob/main/web_server/index.html"
    click Cont "https://github.com/amuizab/protein-3d-structure-pipeline-web-interface/blob/main/web_server/Dockerfile.txt"
    click Cont "https://github.com/amuizab/protein-3d-structure-pipeline-web-interface/blob/main/web_server/dockerignore.txt"

    %% Styles
    classDef pipeline fill:#AED6F1,stroke:#1F618D,stroke-width:2px;
    classDef web fill:#ABEBC6,stroke:#27AE60,stroke-width:2px;
    classDef external fill:#F9E79F,stroke:#F1C40F,stroke-width:2px;
    classDef container fill:#F5B7B1,stroke:#E74C3C,stroke-width:2px;
```
