# biolib/openprotein architecture

> Architecture diagram of the GitHub repository biolib/openprotein, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/biolib/openprotein
- Repository: https://github.com/biolib/openprotein

## Overview

##### Main Components

a. Data Processing Layer:
- Raw Data Input (data/raw/)
- Preprocessing Pipeline (preprocessing.py)
- HDF5 Storage (data/preprocessed/)

b. Core Framework:
- Model Framework (openprotein.py)
- Training Engine (training.py)
- Prediction Engine (prediction.py)
- Utility Functions (util.py)

c. Experiment Management:
- Experiment Modules (experiments/)
- Model Implementations
- RRN Models (experiments/rrn/)
- TMHMM3 Models (experiments/tmhmm3/)
- Example Models (experiments/example/)

d. Output Management:
- Model Storage (output/models/)
- Predictions Storage (output/predictions/)

e. External Interfaces:
- CLI Interface (op_cli.py, preprocessing_cli.py)
- Dashboard Interface (dashboard.py)
- BioLib Integration

**[Data Layer]**
- ↓
**[Processing Layer]**
- ↓
**[Core Framework Layer]**
- ↓
**[Model Layer]**
- ↓
**[Output Layer]**

##### Technical Details

- PyTorch Integration
- HDF5 File Format
- ProteinNet Data Format
- BioLib Dashboard Connection

**[Raw Data] → [Preprocessor] → [HDF5 Storage]**
- ↓
**[Data Loader] → [Training Engine] → [Model Framework]**
- ↑                    ↓
- [Experiment Modules] ← [Model Implementations]
- ↓
- [Output Storage] → [Dashboard]

##### Additional Notes

- Version information (v0.1 as mentioned in README)
- CI/CD pipeline integration (.github/workflows)
- Test coverage (tests/ directory)

##### The integration with external tools (BioLib dashboard)

This will provide a clear understanding of how the system processes protein data, trains models, and generates predictions while maintaining memory efficiency and extensibility.

## Diagram (Mermaid)

```mermaid
graph TB
    %% Styles
    classDef data fill:#b3d9ff,stroke:#333
    classDef processing fill:#90EE90,stroke:#333
    classDef core fill:#ffe066,stroke:#333
    classDef interface fill:#d9b3ff,stroke:#333
    classDef output fill:#e6e6e6,stroke:#333
    classDef experiment fill:#ffc966,stroke:#333

    %% Data Layer
    subgraph DataLayer
        RawData["Raw Data Input"]:::data
        PreprocessedData["HDF5 Storage"]:::data
    end

    %% Processing Layer
    subgraph ProcessingLayer
        Preprocessor["Preprocessing Pipeline"]:::processing
        DataLoader["Data Loader"]:::processing
    end

    %% Core Framework Layer
    subgraph CoreLayer
        ModelFramework["Model Framework"]:::core
        TrainingEngine["Training Engine"]:::core
        PredictionEngine["Prediction Engine"]:::core
        Utils["Utility Functions"]:::core
    end

    %% Experiment Layer
    subgraph ExperimentLayer
        ExpFramework["Experiment Framework"]:::experiment
        RRNModels["RRN Models"]:::experiment
        TMHMMModels["TMHMM3 Models"]:::experiment
        ExampleModels["Example Models"]:::experiment
        TMUtils["TMHMM3 Utilities"]:::experiment
    end

    %% Output Layer
    subgraph OutputLayer
        ModelStorage["Model Storage"]:::output
        PredStorage["Predictions Storage"]:::output
    end

    %% Interface Layer
    subgraph InterfaceLayer
        CLI["CLI Interface"]:::interface
        Dashboard["Dashboard Interface"]:::interface
    end

    %% Relationships
    RawData --> Preprocessor
    Preprocessor --> PreprocessedData
    PreprocessedData --> DataLoader
    DataLoader --> TrainingEngine
    
    ModelFramework --> TrainingEngine
    ModelFramework --> PredictionEngine
    Utils --> ModelFramework
    Utils --> TrainingEngine
    Utils --> PredictionEngine
    
    ExpFramework --> ModelFramework
    RRNModels --> ExpFramework
    TMHMMModels --> ExpFramework
    ExampleModels --> ExpFramework
    TMUtils --> TMHMMModels
    
    TrainingEngine --> ModelStorage
    PredictionEngine --> PredStorage
    
    CLI --> TrainingEngine
    CLI --> PredictionEngine
    ModelStorage --> Dashboard
    PredStorage --> Dashboard

    %% Click Events
    click RawData "https://github.com/biolib/openprotein/tree/master/data/raw/"
    click PreprocessedData "https://github.com/biolib/openprotein/tree/master/data/preprocessed/"
    click Preprocessor "https://github.com/biolib/openprotein/blob/master/preprocessing.py"
    click ModelFramework "https://github.com/biolib/openprotein/blob/master/openprotein.py"
    click TrainingEngine "https://github.com/biolib/openprotein/blob/master/training.py"
    click PredictionEngine "https://github.com/biolib/openprotein/blob/master/prediction.py"
    click Utils "https://github.com/biolib/openprotein/blob/master/util.py"
    click RRNModels "https://github.com/biolib/openprotein/blob/master/experiments/rrn/models.py"
    click TMHMMModels "https://github.com/biolib/openprotein/blob/master/experiments/tmhmm3/tm_models.py"
    click ExampleModels "https://github.com/biolib/openprotein/blob/master/experiments/example/models.py"
    click ModelStorage "https://github.com/biolib/openprotein/tree/master/output/models/"
    click PredStorage "https://github.com/biolib/openprotein/tree/master/output/predictions/"
    click CLI "https://github.com/biolib/openprotein/blob/master/op_cli.py"
    click Dashboard "https://github.com/biolib/openprotein/blob/master/dashboard.py"
    click ExpFramework "https://github.com/biolib/openprotein/tree/master/experiments/"
    click TMUtils "https://github.com/biolib/openprotein/blob/master/experiments/tmhmm3/tm_util.py"

    %% Legend
    subgraph Legend
        L1["Data Components"]:::data
        L2["Processing Components"]:::processing
        L3["Core Components"]:::core
        L4["Interface Components"]:::interface
        L5["Output Components"]:::output
        L6["Experiment Components"]:::experiment
    end
```
