# jonathanking/protein-transformer architecture

> Architecture diagram of the GitHub repository jonathanking/protein-transformer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-02-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jonathanking/protein-transformer
- Repository: https://github.com/jonathanking/protein-transformer

## Overview

Based on your project’s file tree and README, this solution is focused on a research-oriented deep-learning application that uses Transformer-based architectures (among other models) to predict full protein structures. Your system encapsulates data preprocessing, model training (with configurable loss functions and hyperparameters), and analysis—all wrapped up in a well-organized Python package.

##### Project Type and Purpose
- This is a scientific deep-learning project that adapts language-modeling methodologies (specifically the Transformer and attention models) for protein structure prediction.
- The key purpose is to take processed protein data (from ProteinNet and various environmental datasets), train models (encoder-only and convolutional encoder variants), and then evaluate predictions (potentially logging outputs to tools like Weights and Biases).

##### Analyzing the File Structure
- Core Package: The “protein_transformer” directory is where all the main modules reside:
- dataset.py for handling input data formats.
- log.py and losses.py for logging and loss function implementations.
- A “models” subdirectory that includes different model architectures (Transformer, RNN, convolutional encoder, etc.). Notice the “models/transformer” folder with its breakdown into Attention, Decoder, Encoder, and other submodules.
- A “protein” subpackage for domain-specific utilities like handling amino acid sequences, PDB creation, structure building, and even error handling.
- Training and Execution: The “train.py” file is the entry point to train the models.
- Data Directory: The “data” folder holds development/testing datasets and supplementary files (like forcefields and images for the wandb dashboard).
- Research and Experimentation: The “research” directory collects notebooks, analysis scripts, and cluster job files used for hyper-parameter tuning and experimental validation.
- Supporting Infrastructure: Scripts under “scripts” manage dataset alignment, plotting, prediction, and conversion tasks. Devtools, CI configurations, and documentation structures provide continuous integration, testing, and build support.

##### Key System Components
a. Data Layer:
- Data Sources: “data/proteinnet” and “data/development” which include the training/validation/test sets.
- Data Preprocessing: Handled by scripts and routines within “protein_transformer/dataset.py” as well as dedicated scripts in “scripts” (e.g., for downsampling and aligning datasets).

b. Core Processing and Model Training:
- Entry Point: “train.py” initiates the training pipeline.
- Model Architectures: The “protein_transformer/models” submodules. This includes several architectures such as:
– Encoder Only Transformer (and its submodules in “models/transformer”)
– Convolutional Encoder (located partly in “models/convolutional_encoder.py”)
– RNN-based options.
- Training Utilities: Loss functions (“losses.py”), logging (“log.py”), command-line configuration (detailed help in the README), and integration of adjustable hyperparameters.

c. Domain-Specific Protein Processing:
- Protein Utilities: Located in “protein_transformer/protein/” for managing amino acid sequences, PDB file creation, structure building, and related exceptions.

d. Experimentation and Research:
- Notebooks and Analyses: “research/notebooks” and “research/analysis” where various experiments (e.g., hyperparameter comparisons, DRMSD analysis) are performed.
- Cluster Scripts: Located in “research/cluster” for executing experiments on HPC environments.

e. Logging and Visualization:
- Integration with Weights and Biases for dashboard logging (as mentioned in the README and visible in the example dashboard image).

##### Relationships and Interactions
- Data flows from the Data Layer into the preprocessing routines and dataset builders (protein_transformer/dataset.py), which then feed into the training module via train.py.
- The model components (from “models”) process input data and are optimized using loss functions (from “losses.py”), with feedback logged by the logging module.
- Domain-specific utilities (under “protein”) interact closely with both preprocessing and output analysis—transforming raw protein data into structured formats and vice versa.
- The scripts and notebooks in “research” perform analyses that may also influence the model configuration choices (like hyperparameter tuning).
- CI configurations (Travis, GitHub actions) connect to maintain code quality and integration of various modules.

##### Architectural Patterns and Design Principles
- Modularity: The clear separation between data handling, model definitions, and protein-specific processing reinforces modularity.
- Separation of Concerns: By splitting training logic (train.py), model definitions (models/ subdirectory), and protein utilities (protein/ subpackage), each component has a single responsibility.
- Extensibility: The use of command-line arguments to customize training, model parameters, and loss functions hints at a design geared toward easy experimentation and extension.
- Integration: Use of external logging (wandb) and established libraries (PyTorch, ProDy, numpy, etc.) shows adherence to best practices in scientific computing.

## Diagram (Mermaid)

```mermaid
graph TD
    subgraph "Data Layer"
        DS1["Data Proteinnet"]:::data
        DS2["Data Development"]:::data
    end

    subgraph "Preprocessing Layer"
        DP["Data Preprocessing"]:::preprocessing
        PS["Preprocessing Scripts"]:::preprocessing
    end

    subgraph "Training Layer"
        TP["Model Training Pipeline"]:::training
    end

    subgraph "Model Architectures"
        TM["Transformer Models"]:::model
        CE["Convolutional Encoder"]:::model
        RN["RNN Models"]:::model
        MU["Additional Model Utils"]:::model
    end

    subgraph "Training Utilities"
        LF["Loss Functions"]:::utilities
        LG["Logging"]:::utilities
    end

    subgraph "Domain Layer"
        PU["Protein Utilities"]:::domain
    end

    subgraph "Experimentation Layer"
        NA["Notebooks & Analysis"]:::research
        CS["Cluster Scripts"]:::research
    end

    subgraph "External Services"
        WB["Weights & Biases"]:::external
    end

    %% Connections
    DS1 --> DP
    DS2 --> DP
    DP --> PS
    DP --> TP
    PS --> TP

    TP --> TM
    TP --> CE
    TP --> RN
    TP --> MU
    TP --> LF
    TP --> LG

    DP --> PU
    TP --> PU

    NA --> TP
    CS --> TP

    LG --> WB

    %% Class Definitions
    classDef data fill:#ffdddd, stroke:#cc0000, stroke-width:2px;
    classDef preprocessing fill:#fff2cc, stroke:#ff9900, stroke-width:2px;
    classDef training fill:#e6f2ff, stroke:#0088cc, stroke-width:2px;
    classDef model fill:#ddffdd, stroke:#33aa33, stroke-width:2px;
    classDef utilities fill:#e6e6fa, stroke:#6666ff, stroke-width:2px;
    classDef domain fill:#ffe6e6, stroke:#ff3333, stroke-width:2px;
    classDef research fill:#f0e6ff, stroke:#9933ff, stroke-width:2px;
    classDef external fill:#e6ffe6, stroke:#33cc33, stroke-width:2px;

    %% Click Events
    click DS1 "https://github.com/jonathanking/protein-transformer/tree/master/data/proteinnet"
    click DS2 "https://github.com/jonathanking/protein-transformer/tree/master/data/development"
    click DP "https://github.com/jonathanking/protein-transformer/blob/master/protein_transformer/dataset.py"
    click PS "https://github.com/jonathanking/protein-transformer/blob/master/scripts/downsample_proteinnet.py"
    click TP "https://github.com/jonathanking/protein-transformer/blob/master/protein_transformer/train.py"
    click TM "https://github.com/jonathanking/protein-transformer/tree/master/protein_transformer/models/transformer"
    click CE "https://github.com/jonathanking/protein-transformer/blob/master/protein_transformer/models/convolutional_encoder.py"
    click RN "https://github.com/jonathanking/protein-transformer/blob/master/protein_transformer/models/rnn.py"
    click MU "https://github.com/jonathanking/protein-transformer/blob/master/protein_transformer/models/ModelPrediction.py"
    click LF "https://github.com/jonathanking/protein-transformer/blob/master/protein_transformer/losses.py"
    click LG "https://github.com/jonathanking/protein-transformer/blob/master/protein_transformer/log.py"
    click PU "https://github.com/jonathanking/protein-transformer/tree/master/protein_transformer/protein"
    click NA "https://github.com/jonathanking/protein-transformer/tree/master/research/notebooks"
    click CS "https://github.com/jonathanking/protein-transformer/tree/master/research/cluster"
```
