How henriqwuchryn/chembl_miner works
Project type and purpose
• This is a standalone Python-based chemoinformatics and machine-learning pipeline/library.
• It ingests molecular data, computes chemical fingerprints, runs ML workflows (classification, hyperparameter optimization, outlier detection, feature selection), performs exploratory analyses, and supports deployment of trained models.
File structure
Top-level components:
• config/ & config/parameter_grids: configuration files for runs (e.g., hyperparameter grids).
• data_obtaining.py & dataset_wrapper.py: data ingestion and abstraction layer.
• fingerprint/: set of XML definitions for various chemical fingerprints, plus fingerprint_calculation.py to compute them.
• machine_learning_methods.py: core ML algorithms and pipelines (e.g., classifiers, GA for hyperparam opt).
• miscelanneous_methods.py & molecules_manipulation_methods.py: utility functions.
• ml_analysis.py, multivariate_exploratory_analysis.py, univariate_exploratory_analysis.py: analysis and visualization modules.
• package_functions.py: top-level entry points.
• deployment.py: code to serialize/export models, possibly serve them.
README for additional insights
• Roadmap items point to expanding GA hyperparameter tuning, outlier detection, feature selection, classifier pipeline.
a. Main components:
1. Data Layer
• Data Obtainer (data_obtaining.py)
• Dataset Wrapper (dataset_wrapper.py)
2. Feature Extraction
• Fingerprint Engine (fingerprint_calculation.py + fingerprint/*.xml)
3. Machine‐Learning Layer
• ML Methods & Pipelines (machine_learning_methods.py)
• Hyperparam Optimization (GA within ML methods)
4. Analysis & Reporting
• Univariate EDA (univariate_exploratory_analysis.py)
• Multivariate EDA (multivariate_exploratory_analysis.py)
• ML Analysis & Metrics (ml_analysis.py)
5. Configuration
• Global Config (config/config)
• Parameter Grids (config/parameter_grids)
6. Deployment
• Model Export & Serving (deployment.py)
7. Utilities
• Utility Functions (miscelanneous_methods.py, molecules_manipulation_methods.py)
• Top-Level APIs (package_functions.py)
b. Relationships & interactions
• Data Layer feeds raw molecular data into Feature Extraction.
• Feature Extraction outputs feature vectors to the ML Layer.
• ML Layer reads config/parameter_grids for hyperparameter settings and trains models.
• Trained models and results flow into the Analysis & Reporting modules.
• Deployment ingests final models from ML Layer for serialization/serving.
• Utilities are used by all stages; Configuration is read at startup by each component.
c. Architectural patterns & principles
• Modular separation: each stage is its own module/package.
• Pipeline abstraction: clear “data → features → train → analyze → deploy” flow.
• Configuration-driven: no hard-coded parameters; everything lives in config.
• Extensibility: new fingerprint types, ML methods, analyses, or deployment targets can be added by dropping new files.
d. Key technologies & libraries
• RDKit (implied for fingerprinting)
• scikit-learn (ML algorithms, pipelines)
• pandas/numpy (data handling)
• XML fingerprint definitions (custom extension points)
Detail
• Clearly label XML fingerprint definitions as configurable extensions.