How jonathanking/protein-transformer works
Based on your project’s file tree and README, this solution is focused on a research-oriented deep-learning application that uses Transformer-based architectures (among other models) to predict full protein structures. Your system encapsulates data preprocessing, model training (with configurable loss functions and hyperparameters), and analysis—all wrapped up in a well-organized Python package.
Project Type and Purpose
• This is a scientific deep-learning project that adapts language-modeling methodologies (specifically the Transformer and attention models) for protein structure prediction.
• The key purpose is to take processed protein data (from ProteinNet and various environmental datasets), train models (encoder-only and convolutional encoder variants), and then evaluate predictions (potentially logging outputs to tools like Weights and Biases).
Analyzing the File Structure
• Core Package: The “protein_transformer” directory is where all the main modules reside:
• dataset.py for handling input data formats.
• log.py and losses.py for logging and loss function implementations.
• A “models” subdirectory that includes different model architectures (Transformer, RNN, convolutional encoder, etc.). Notice the “models/transformer” folder with its breakdown into Attention, Decoder, Encoder, and other submodules.
• A “protein” subpackage for domain-specific utilities like handling amino acid sequences, PDB creation, structure building, and even error handling.
• Training and Execution: The “train.py” file is the entry point to train the models.
• Data Directory: The “data” folder holds development/testing datasets and supplementary files (like forcefields and images for the wandb dashboard).
• Research and Experimentation: The “research” directory collects notebooks, analysis scripts, and cluster job files used for hyper-parameter tuning and experimental validation.
• Supporting Infrastructure: Scripts under “scripts” manage dataset alignment, plotting, prediction, and conversion tasks. Devtools, CI configurations, and documentation structures provide continuous integration, testing, and build support.
Key System Components
a. Data Layer:
• Data Sources: “data/proteinnet” and “data/development” which include the training/validation/test sets.
• Data Preprocessing: Handled by scripts and routines within “protein_transformer/dataset.py” as well as dedicated scripts in “scripts” (e.g., for downsampling and aligning datasets).
b. Core Processing and Model Training:
• Entry Point: “train.py” initiates the training pipeline.
• Model Architectures: The “protein_transformer/models” submodules. This includes several architectures such as:
– Encoder Only Transformer (and its submodules in “models/transformer”)
– Convolutional Encoder (located partly in “models/convolutional_encoder.py”)
– RNN-based options.
• Training Utilities: Loss functions (“losses.py”), logging (“log.py”), command-line configuration (detailed help in the README), and integration of adjustable hyperparameters.
c. Domain-Specific Protein Processing:
• Protein Utilities: Located in “protein_transformer/protein/” for managing amino acid sequences, PDB file creation, structure building, and related exceptions.
d. Experimentation and Research:
• Notebooks and Analyses: “research/notebooks” and “research/analysis” where various experiments (e.g., hyperparameter comparisons, DRMSD analysis) are performed.
• Cluster Scripts: Located in “research/cluster” for executing experiments on HPC environments.
e. Logging and Visualization:
• Integration with Weights and Biases for dashboard logging (as mentioned in the README and visible in the example dashboard image).
Relationships and Interactions
• Data flows from the Data Layer into the preprocessing routines and dataset builders (protein_transformer/dataset.py), which then feed into the training module via train.py.
• The model components (from “models”) process input data and are optimized using loss functions (from “losses.py”), with feedback logged by the logging module.
• Domain-specific utilities (under “protein”) interact closely with both preprocessing and output analysis—transforming raw protein data into structured formats and vice versa.
• The scripts and notebooks in “research” perform analyses that may also influence the model configuration choices (like hyperparameter tuning).
• CI configurations (Travis, GitHub actions) connect to maintain code quality and integration of various modules.
Architectural Patterns and Design Principles
• Modularity: The clear separation between data handling, model definitions, and protein-specific processing reinforces modularity.
• Separation of Concerns: By splitting training logic (train.py), model definitions (models/ subdirectory), and protein utilities (protein/ subpackage), each component has a single responsibility.
• Extensibility: The use of command-line arguments to customize training, model parameters, and loss functions hints at a design geared toward easy experimentation and extension.
• Integration: Use of external logging (wandb) and established libraries (PyTorch, ProDy, numpy, etc.) shows adherence to best practices in scientific computing.