# henriqwuchryn/chembl_miner architecture

> Architecture diagram of the GitHub repository henriqwuchryn/chembl_miner, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/henriqwuchryn/chembl_miner
- Repository: https://github.com/henriqwuchryn/chembl_miner

## Overview

##### Project type and purpose
- This is a standalone Python-based chemoinformatics and machine-learning pipeline/library.
- It ingests molecular data, computes chemical fingerprints, runs ML workflows (classification, hyperparameter optimization, outlier detection, feature selection), performs exploratory analyses, and supports deployment of trained models.

##### File structure
Top-level components:
- config/ & config/parameter_grids: configuration files for runs (e.g., hyperparameter grids).
- data_obtaining.py & dataset_wrapper.py: data ingestion and abstraction layer.
- fingerprint/: set of XML definitions for various chemical fingerprints, plus fingerprint_calculation.py to compute them.
- machine_learning_methods.py: core ML algorithms and pipelines (e.g., classifiers, GA for hyperparam opt).
- miscelanneous_methods.py & molecules_manipulation_methods.py: utility functions.
- ml_analysis.py, multivariate_exploratory_analysis.py, univariate_exploratory_analysis.py: analysis and visualization modules.
- package_functions.py: top-level entry points.
- deployment.py: code to serialize/export models, possibly serve them.

##### README for additional insights
- Roadmap items point to expanding GA hyperparameter tuning, outlier detection, feature selection, classifier pipeline.

a. Main components:
1. Data Layer
- Data Obtainer (data_obtaining.py)
- Dataset Wrapper (dataset_wrapper.py)
2. Feature Extraction
- Fingerprint Engine (fingerprint_calculation.py + fingerprint/*.xml)
3. Machine‐Learning Layer
- ML Methods & Pipelines (machine_learning_methods.py)
- Hyperparam Optimization (GA within ML methods)
4. Analysis & Reporting
- Univariate EDA (univariate_exploratory_analysis.py)
- Multivariate EDA (multivariate_exploratory_analysis.py)
- ML Analysis & Metrics (ml_analysis.py)
5. Configuration
- Global Config (config/config)
- Parameter Grids (config/parameter_grids)
6. Deployment
- Model Export & Serving (deployment.py)
7. Utilities
- Utility Functions (miscelanneous_methods.py, molecules_manipulation_methods.py)
- Top-Level APIs (package_functions.py)

b. Relationships & interactions
- Data Layer feeds raw molecular data into Feature Extraction.
- Feature Extraction outputs feature vectors to the ML Layer.
- ML Layer reads config/parameter_grids for hyperparameter settings and trains models.
- Trained models and results flow into the Analysis & Reporting modules.
- Deployment ingests final models from ML Layer for serialization/serving.
- Utilities are used by all stages; Configuration is read at startup by each component.

c. Architectural patterns & principles
- Modular separation: each stage is its own module/package.
- Pipeline abstraction: clear “data → features → train → analyze → deploy” flow.
- Configuration-driven: no hard-coded parameters; everything lives in config.
- Extensibility: new fingerprint types, ML methods, analyses, or deployment targets can be added by dropping new files.

d. Key technologies & libraries
- RDKit (implied for fingerprinting)
- scikit-learn (ML algorithms, pipelines)
- pandas/numpy (data handling)
- XML fingerprint definitions (custom extension points)

##### Detail
- Clearly label XML fingerprint definitions as configurable extensions.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Data Layer"
        DO["Data Obtainer"]:::data
        DW["Dataset Wrapper"]:::data
    end

    subgraph "Feature Extraction"
        FE["Fingerprint Engine"]:::data
        FDs["Fingerprint Definitions"]:::data
    end

    subgraph "Machine-Learning Layer"
        ML["ML Methods & Pipelines"]:::ml
    end

    subgraph "Analysis & Reporting"
        UNI["Univariate EDA"]:::analysis
        MULTI["Multivariate EDA"]:::analysis
        MLAN["ML Analysis & Metrics"]:::analysis
    end

    subgraph "Deployment"
        DEP["Model Export & Serving"]:::deploy
    end

    subgraph "Configuration"
        GC["Global Config"]:::config
        PG["Hyperparameter Grids"]:::config
    end

    subgraph "Utilities & API"
        UF["Miscellaneous Functions"]:::config
        MM["Molecule Manipulation"]:::config
        API["Top-Level API/CLI Entry"]:::config
    end

    subgraph "External Dependencies"
        RD["RDKit"]:::external
        SK["scikit-learn"]:::external
    end

    %% Data flow
    DO -->|"raw molecular data"| FE
    DW -->|"wrapped dataset"| FE
    FE -->|"feature vectors"| ML
    ML -->|"trained model"| UNI
    ML -->|"trained model"| MULTI
    ML -->|"trained model"| MLAN
    ML -->|"trained model"| DEP

    %% Config flow
    GC --> DO
    GC --> DW
    GC --> FE
    GC --> ML
    GC --> UNI
    GC --> MULTI
    GC --> MLAN
    GC --> DEP
    PG --> ML

    %% Utils flow
    UF --> DO
    UF --> FE
    UF --> ML
    UF --> UNI
    UF --> MULTI
    UF --> MLAN
    UF --> DEP
    MM --> DO
    MM --> FE
    MM --> ML
    MM --> UNI
    MM --> MULTI
    MM --> MLAN
    MM --> DEP

    %% API entry
    API --> DO
    API --> FE
    API --> ML
    API --> UNI
    API --> DEP

    %% External links
    RD --> FE
    SK --> ML

    %% Click Events
    click DO "https://github.com/henriqwuchryn/chembl_miner/blob/main/data_obtaining.py"
    click DW "https://github.com/henriqwuchryn/chembl_miner/blob/main/dataset_wrapper.py"
    click FE "https://github.com/henriqwuchryn/chembl_miner/blob/main/fingerprint_calculation.py"
    click FDs "https://github.com/henriqwuchryn/chembl_miner/tree/main/fingerprint/"
    click ML "https://github.com/henriqwuchryn/chembl_miner/blob/main/machine_learning_methods.py"
    click UNI "https://github.com/henriqwuchryn/chembl_miner/blob/main/univariate_exploratory_analysis.py"
    click MULTI "https://github.com/henriqwuchryn/chembl_miner/blob/main/multivariate_exploratory_analysis.py"
    click MLAN "https://github.com/henriqwuchryn/chembl_miner/blob/main/ml_analysis.py"
    click GC "https://github.com/henriqwuchryn/chembl_miner/tree/main/config/config"
    click PG "https://github.com/henriqwuchryn/chembl_miner/tree/main/config/parameter_grids"
    click DEP "https://github.com/henriqwuchryn/chembl_miner/blob/main/deployment.py"
    click UF "https://github.com/henriqwuchryn/chembl_miner/blob/main/miscelanneous_methods.py"
    click MM "https://github.com/henriqwuchryn/chembl_miner/blob/main/molecules_manipulation_methods.py"
    click API "https://github.com/henriqwuchryn/chembl_miner/blob/main/package_functions.py"

    %% Styles
    classDef data fill:#D0E8FF,stroke:#2B7CE9,color:#000
    classDef ml fill:#D0F2D0,stroke:#28A745,color:#000
    classDef analysis fill:#FFF4C1,stroke:#FFC107,color:#000
    classDef deploy fill:#FFE0B2,stroke:#FF5722,color:#000
    classDef config fill:#F0F0F0,stroke:#999,color:#000
    classDef external fill:#E0E0E0,stroke:#666,color:#000,stroke-dasharray: 5 5
```
