# mwasifanwar/automl_framework architecture

> Architecture diagram of the GitHub repository mwasifanwar/automl_framework, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-11-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mwasifanwar/automl_framework
- Repository: https://github.com/mwasifanwar/automl_framework

## Overview

##### Project Type and Purpose
- This is a Python-based AutoML framework (“automl_framework”) designed to automate the end-to-end machine learning workflow: data preprocessing, feature engineering, model selection, hyperparameter optimization, neural architecture search, and deployment/monitoring.
- It’s distributed as a library (setup.py, requirements.txt), provides example pipelines (examples/), and exposes a CLI/entrypoint (main.py).

##### File Structure Analysis
- Top-level files:
- README.md, config.yaml – project overview and configuration
- setup.py, requirements.txt – packaging and dependencies
- main.py – CLI or orchestrator entrypoint
- Directories:
- core/ – the heart of the ML pipeline
– data_processor.py, feature_engineer.py, hyperparameter_optimizer.py, model_selector.py, neural_architecture_search.py
- models/ – model definitions and ensemble builder
- deployment/ – serving and monitoring modules
- utils/ – config loader, metrics calculator, pipeline utilities
- examples/ – usage demonstrations
- tests/ – unit tests ensuring correctness

##### Key Components
a. Configuration & Orchestration
- config.yaml + utils/config_loader.py
- main.py orchestrates the flow
b. Core Pipeline
- DataProcessor → FeatureEngineer → ModelSelector → HyperparameterOptimizer → NeuralArchitectureSearch
c. Model Library
- custom_models.py → EnsembleBuilder
d. Deployment & Monitoring
- model_serving.py → monitoring.py
e. Utilities & Examples
- pipeline_utils.py, metrics_calculator.py
- examples/basic_usage.py, advanced_pipeline.py

##### Relationships and Data Flow
- Start: User invokes main.py (CLI/API)
→ loads config.yaml via Utils
→ passes raw data to Core.DataProcessor
→ pipeline flows through FeatureEngineering, ModelSelection, HPO, NAS
→ selected model(s) passed to Models.CustomModels / EnsembleBuilder
→ final model artifact handed off to Deployment.ModelServing
→ live predictions logged to Deployment.Monitoring
- Throughout, Utils provide shared functions (metrics, config parsing, pipeline helpers).

##### Architectural Patterns & Principles
- Modular Pipeline: each core step is a separate component
- Layered Architecture:
- Presentation/Orchestration (main.py)
- Core ML Pipeline
- Model Definitions
- Deployment/Operations
- Utilities & Configuration
- Single Responsibility: each file/class handles one stage or concern
- Extensibility: add new feature engineers, optimizers, models by dropping modules into core/ or models/

## Diagram (Mermaid)

```mermaid
graph TD
    subgraph "Presentation / Orchestration"
        Orchestrator["Orchestrator\n(main.py)"]:::orchestrator
    end

    subgraph "Utils & Configuration"
        Config["YAML Config"]:::external
        ConfigLoader["Config Loader"]:::utils
        MetricsCalc["Metrics Calculator"]:::utils
        PipelineUtils["Pipeline Utils"]:::utils
    end

    subgraph "Core Pipeline"
        DataProcessor["Data Processor"]:::core
        FeatureEngineer["Feature Engineer"]:::core
        ModelSelector["Model Selector"]:::core
        HPO["Hyperparameter Optimizer"]:::core
        NAS["Neural Architecture Search"]:::core
    end

    subgraph "Model Library"
        CustomModels["Custom Models"]:::models
        EnsembleBuilder["Ensemble Builder"]:::models
    end

    subgraph "Deployment & Monitoring"
        ModelServing["Model Serving"]:::deployment
        Monitoring["Monitoring"]:::deployment
    end

    subgraph "Examples"
        BasicExample["Basic Usage"]:::external
        AdvancedExample["Advanced Pipeline"]:::external
    end

    Config -->|loads| ConfigLoader
    ConfigLoader -->|provides config| Orchestrator

    Orchestrator -->|raw data| DataProcessor
    DataProcessor -->|processed data| FeatureEngineer
    FeatureEngineer -->|features| ModelSelector
    ModelSelector -->|candidate models| HPO
    HPO -->|optimized models| NAS

    NAS -->|architectures| CustomModels
    ModelSelector -->|selected models| CustomModels
    HPO -->|hyperparams| CustomModels

    CustomModels -->|model definitions| EnsembleBuilder
    EnsembleBuilder -->|model artifact| ModelServing
    ModelServing -->|predictions| Monitoring
    Monitoring -->|feedback| Orchestrator

    MetricsCalc -.-> DataProcessor
    MetricsCalc -.-> FeatureEngineer
    MetricsCalc -.-> ModelSelector
    PipelineUtils -.-> DataProcessor

    BasicExample --> Orchestrator
    AdvancedExample --> Orchestrator

    click Config "https://github.com/mwasifanwar/automl_framework/blob/main/config.yaml"
    click ConfigLoader "https://github.com/mwasifanwar/automl_framework/blob/main/utils/config_loader.py"
    click Orchestrator "https://github.com/mwasifanwar/automl_framework/blob/main/main.py"
    click DataProcessor "https://github.com/mwasifanwar/automl_framework/blob/main/core/data_processor.py"
    click FeatureEngineer "https://github.com/mwasifanwar/automl_framework/blob/main/core/feature_engineer.py"
    click ModelSelector "https://github.com/mwasifanwar/automl_framework/blob/main/core/model_selector.py"
    click HPO "https://github.com/mwasifanwar/automl_framework/blob/main/core/hyperparameter_optimizer.py"
    click NAS "https://github.com/mwasifanwar/automl_framework/blob/main/core/neural_architecture_search.py"
    click CustomModels "https://github.com/mwasifanwar/automl_framework/blob/main/models/custom_models.py"
    click EnsembleBuilder "https://github.com/mwasifanwar/automl_framework/blob/main/models/ensemble_builder.py"
    click ModelServing "https://github.com/mwasifanwar/automl_framework/blob/main/deployment/model_serving.py"
    click Monitoring "https://github.com/mwasifanwar/automl_framework/blob/main/deployment/monitoring.py"
    click MetricsCalc "https://github.com/mwasifanwar/automl_framework/blob/main/utils/metrics_calculator.py"
    click PipelineUtils "https://github.com/mwasifanwar/automl_framework/blob/main/utils/pipeline_utils.py"
    click BasicExample "https://github.com/mwasifanwar/automl_framework/blob/main/examples/basic_usage.py"
    click AdvancedExample "https://github.com/mwasifanwar/automl_framework/blob/main/examples/advanced_pipeline.py"

    classDef core fill:#ADD8E6,stroke:#000,stroke-width:1px;
    classDef models fill:#90EE90,stroke:#000,stroke-width:1px;
    classDef deployment fill:#FFA500,stroke:#000,stroke-width:1px;
    classDef utils fill:#D3D3D3,stroke:#000,stroke-width:1px;
    classDef external fill:#eeeeee,stroke:#000,stroke-width:1px;
    classDef orchestrator fill:#ffffff,stroke:#000,stroke-width:1px;
```
