# muzammilck/metatune architecture

> Architecture diagram of the GitHub repository muzammilck/metatune, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-04-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/muzammilck/metatune
- Repository: https://github.com/muzammilck/metatune

## Overview

1. **What this repo is**
   MetaTune is a single-process Python ML orchestration system for hyperparameter optimization. Its core architecture is a pipeline that takes a CSV dataset, analyzes its statistical characteristics, chooses a model family, tunes hyperparameters, trains a model, packages the result, and stores the run outcome for future learning.

2. **Primary entry points**
   There are two user-facing fronts: `app.py` for the Streamlit dashboard and `pipeline.py` for headless CLI execution. Both drive the same core modules, so the UI and batch path are separate shells around a shared optimization/training core.

3. **Data ingestion and dataset “DNA”**
   `data_analyzer.py` is the first real subsystem in the workflow. It loads a CSV, detects the target column, classifies the task as classification or regression, and computes the dataset “DNA” meta-features described in the README: size, feature mix, missingness, sparsity, skew/kurtosis, correlations, cardinality, entropy, imbalance, and a composite difficulty score. This meta-feature vector is the main handoff artifact for downstream components.

4. **Meta-learning brain**
   `brain.py` is the learned decision layer. It maps dataset DNA to hyperparameters using prior run history stored in `knowledge_base.csv`. The architecture is explicitly stateful: cold-start heuristics are used when history is sparse, then trained predictions take over once enough elite runs exist. The brain also appends new experiences back into the knowledge base.

5. **Algorithm selection boundary**
   `algorithm_recommender.py` sits between dataset analysis and training. It turns DNA plus hyperparameter context into a ranked algorithm recommendation and a trial budget. This module is important because it defines the contract for what the engines must support; the tests enforce that every algorithm ID produced here is recognized by the execution layer.

6. **Training engines are split by runtime target**
   There are two execution engines: `sklearn_engine.py` for deployable classical ML packages and `engine.py` / `engine_stream.py` for PyTorch training. The sklearn path is designed to return a serializable package, typically for `.joblib` export, while the PyTorch path is used for training dynamics and live monitoring. `engine_stream.py` is the streaming/generator-based variant used when the UI needs epoch-by-epoch updates.

7. **Adaptive training loop**
   `engine_stream.py` contains the real-time training monitor logic. During training it can detect patterns such as plateauing, overfitting, or vanishing gradients and adjust the training process on the fly. Architecturally, this makes the training engine not just a dumb executor but a feedback-controlled subsystem that influences the optimization loop.

8. **Bilevel optimization layer**
   `bilevel.py` orchestrates the outer optimization loop. It evaluates candidate configurations, perturbs winners, and feeds results back into the system. The README frames this as a state machine with phases like initialize, tune, and use best. This layer is the bridge between “predict a good config once” and “iteratively improve across generations.”

9. **Vizier-compatible local search abstraction**
   `vizier_stub.py` and `designer_brain.py` form the search-management boundary. The stub provides local replacements for concepts like Study, Trial, Measurement, and SearchSpace, while `designer_brain.py` adapts the meta-learner into a designer/suggester interface. This gives the repo a Vizier-like control plane without depending on Google Cloud Vizier.

10. **Persistent feedback loop**
   `knowledge_base.csv` is the main persistence artifact and acts as the system’s memory. Every run can append dataset DNA, chosen parameters, and outcome metrics. That history is then reused by the brain for later predictions, making the architecture an online-learning loop rather than a stateless optimizer.

11. **Packaging and output artifacts**
   The system produces deployable model bundles as part of the training path. The sklearn route emits serialized package artifacts, while the PyTorch route works with state dictionaries. The CLI workflow also produces run reports and charts, which makes the repo useful both as an optimizer and as a training/reporting tool.

12. **Infrastructure and external dependencies**
   The main runtime is Python 3.9+ with Streamlit, PyTorch, scikit-learn, and optional XGBoost. W&B is mentioned for experiment tracking, but it does not appear to be a hard architectural dependency in the file tree. The knowledge base is file-based rather than database-backed, so the repo is intentionally low-infrastructure and session-persistent through CSV files.

13. **Testing and contract boundaries**
   The `tests/` directory is not incidental; it enforces the repo’s architecture. `test_integration.py` protects the recommender-to-engine contract, `test_vizier_stub.py` validates the local study/trial abstraction, `test_brain.py` checks meta-learner outputs, and `test_pipeline.py` exercises the end-to-end orchestration. This test structure mirrors the system boundaries and is a good guide for diagramming the architecture accurately.

14. **Supporting utilities and dataset generators**
   `car.py` and `generate_audit_data.py` are not core runtime modules, but they support validation and demos by generating realistic and stress-test datasets. `verify_metatune.py` and `verify_evolution.py` are verification scripts for system-wide and evolutionary behavior. These sit outside the main optimization pipeline but are useful to show as auxiliary tooling around the core system.

15. **Recommended diagram framing**
   A clear architecture diagram for this repo should show four main layers: input/UI, dataset analysis and recommendation, optimization/training execution, and persistent feedback memory. The most important data artifact flowing through the system is the dataset DNA dictionary, and the most important long-lived boundary is the `knowledge_base.csv` feedback store that closes the learning loop.

Main entry points: [`algorithm_recommender.py`](https://github.com/muzammilck/metatune/blob/master/algorithm_recommender.py) (Recommender), [`data_analyzer.py`](https://github.com/muzammilck/metatune/blob/master/data_analyzer.py) (Data DNA), [`bilevel.py`](https://github.com/muzammilck/metatune/blob/master/bilevel.py) (Bilevel), [`brain.py`](https://github.com/muzammilck/metatune/blob/master/brain.py) (Brain)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interfaces["Entry points"]
  node_app_py["App<br/>Streamlit UI<br/>[app.py]"]
  node_pipeline_py["Pipeline<br/>CLI runner<br/>[pipeline.py]"]
end

subgraph group_analysis["Analysis & recommendation"]
  node_data_analyzer_py["Data DNA<br/>dataset analysis<br/>[data_analyzer.py]"]
  node_algorithm_recommender_py["Recommender<br/>ranker"]
end

subgraph group_optimization["Search & meta-learning"]
  node_brain_py["Brain<br/>meta-learner<br/>[brain.py]"]
  node_designer_brain_py["Designer<br/>suggestion adapter<br/>[designer_brain.py]"]
  node_bilevel_py["Bilevel<br/>outer loop<br/>[bilevel.py]"]
  node_vizier_stub_py["Vizier stub<br/>study model<br/>[vizier_stub.py]"]
end

subgraph group_execution["Training engines"]
  node_engine_py["Torch engine<br/>trainer<br/>[engine.py]"]
  node_engine_stream_py["Stream engine<br/>live trainer<br/>[engine_stream.py]"]
  node_sklearn_engine_py["Sklearn engine<br/>package trainer<br/>[sklearn_engine.py]"]
end

subgraph group_memory["Persistent memory"]
  node_knowledge_base_csv[("Knowledge base<br/>run history<br/>[knowledge_base.csv]")]
end

subgraph group_validation["Tooling & checks"]
  node_tests_dir["Tests"]
  node_car_py["CAR data<br/>dataset helper<br/>[car.py]"]
  node_generate_audit_data_py["Audit data<br/>data generator"]
  node_verify_metatune_py["Verify core<br/>check script<br/>[verify_metatune.py]"]
  node_verify_evolution_py["Verify evolution<br/>check script"]
end

node_app_py -->|"load/analyze"| node_data_analyzer_py
node_app_py -->|"optimize"| node_bilevel_py
node_app_py -->|"live train"| node_engine_stream_py
node_pipeline_py -->|"load/analyze"| node_data_analyzer_py
node_pipeline_py -->|"optimize"| node_bilevel_py
node_pipeline_py -->|"train"| node_engine_py
node_data_analyzer_py -->|"DNA"| node_algorithm_recommender_py
node_data_analyzer_py -->|"meta-features"| node_brain_py
node_data_analyzer_py -->|"task context"| node_bilevel_py
node_algorithm_recommender_py -->|"suggest"| node_designer_brain_py
node_algorithm_recommender_py -->|"classical path"| node_sklearn_engine_py
node_algorithm_recommender_py -->|"torch path"| node_engine_py
node_algorithm_recommender_py -->|"live path"| node_engine_stream_py
node_designer_brain_py -->|"study/trial"| node_vizier_stub_py
node_designer_brain_py -->|"delegate"| node_brain_py
node_bilevel_py -->|"select"| node_algorithm_recommender_py
node_bilevel_py -->|"evaluate"| node_engine_stream_py
node_bilevel_py -->|"feedback"| node_brain_py
node_engine_stream_py -->|"metrics"| node_brain_py
node_engine_py -->|"results"| node_knowledge_base_csv
node_sklearn_engine_py -->|"package/run"| node_knowledge_base_csv
node_brain_py -->|"append/history"| node_knowledge_base_csv
node_tests_dir -->|"check"| node_data_analyzer_py
node_tests_dir -->|"contract"| node_algorithm_recommender_py
node_tests_dir -->|"check"| node_brain_py
node_tests_dir -->|"check"| node_vizier_stub_py
node_tests_dir -->|"end-to-end"| node_pipeline_py
node_car_py -->|"support"| node_generate_audit_data_py
node_generate_audit_data_py -->|"sample data"| node_data_analyzer_py
node_verify_metatune_py -->|"verify"| node_pipeline_py
node_verify_evolution_py -->|"verify"| node_bilevel_py

click node_app_py "https://github.com/muzammilck/metatune/blob/master/app.py"
click node_pipeline_py "https://github.com/muzammilck/metatune/blob/master/pipeline.py"
click node_data_analyzer_py "https://github.com/muzammilck/metatune/blob/master/data_analyzer.py"
click node_algorithm_recommender_py "https://github.com/muzammilck/metatune/blob/master/algorithm_recommender.py"
click node_brain_py "https://github.com/muzammilck/metatune/blob/master/brain.py"
click node_designer_brain_py "https://github.com/muzammilck/metatune/blob/master/designer_brain.py"
click node_bilevel_py "https://github.com/muzammilck/metatune/blob/master/bilevel.py"
click node_vizier_stub_py "https://github.com/muzammilck/metatune/blob/master/vizier_stub.py"
click node_engine_py "https://github.com/muzammilck/metatune/blob/master/engine.py"
click node_engine_stream_py "https://github.com/muzammilck/metatune/blob/master/engine_stream.py"
click node_sklearn_engine_py "https://github.com/muzammilck/metatune/blob/master/sklearn_engine.py"
click node_knowledge_base_csv "https://github.com/muzammilck/metatune/blob/master/knowledge_base.csv"
click node_tests_dir "https://github.com/muzammilck/metatune/tree/master/tests"
click node_car_py "https://github.com/muzammilck/metatune/blob/master/car.py"
click node_generate_audit_data_py "https://github.com/muzammilck/metatune/blob/master/generate_audit_data.py"
click node_verify_metatune_py "https://github.com/muzammilck/metatune/blob/master/verify_metatune.py"
click node_verify_evolution_py "https://github.com/muzammilck/metatune/blob/master/verify_evolution.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_app_py,node_pipeline_py toneBlue
class node_data_analyzer_py,node_algorithm_recommender_py toneAmber
class node_brain_py,node_designer_brain_py,node_bilevel_py,node_vizier_stub_py toneMint
class node_engine_py,node_engine_stream_py,node_sklearn_engine_py toneRose
class node_knowledge_base_csv toneIndigo
class node_tests_dir,node_car_py,node_generate_audit_data_py,node_verify_metatune_py,node_verify_evolution_py toneTeal
```

## Components

### Entry points

User-facing shells for interactive and batch runs.

- [App](https://github.com/muzammilck/metatune/blob/master/app.py): Streamlit UI · Interactive dashboard entry point. · `app.py`
- [Pipeline](https://github.com/muzammilck/metatune/blob/master/pipeline.py): CLI runner · Headless orchestration entry point. · `pipeline.py`

### Analysis & recommendation

Dataset characterization and algorithm selection.

- [Data DNA](https://github.com/muzammilck/metatune/blob/master/data_analyzer.py): dataset analysis · Loads CSVs, detects target/task, and computes meta-features. · `data_analyzer.py`
- [Recommender](https://github.com/muzammilck/metatune/blob/master/algorithm_recommender.py): ranker · Ranks algorithms and sets trial budget from dataset DNA. · `algorithm_recommender.py`

### Search & meta-learning

Stateful decision layers and outer-loop orchestration.

- [Brain](https://github.com/muzammilck/metatune/blob/master/brain.py): meta-learner · Maps dataset DNA to hyperparameters using prior run history. · `brain.py`
- [Designer](https://github.com/muzammilck/metatune/blob/master/designer_brain.py): suggestion adapter · Adapts the meta-learner to a Vizier-like suggester interface. · `designer_brain.py`
- [Bilevel](https://github.com/muzammilck/metatune/blob/master/bilevel.py): outer loop · Coordinates candidate evaluation, mutation, and best-config reuse. · `bilevel.py`
- [Vizier stub](https://github.com/muzammilck/metatune/blob/master/vizier_stub.py): study model · Local Study/Trial/SearchSpace abstraction for search control. · `vizier_stub.py`

### Training engines

Classical and PyTorch execution paths.

- [Torch engine](https://github.com/muzammilck/metatune/blob/master/engine.py): trainer · PyTorch training runtime for model fitting. · `engine.py`
- [Stream engine](https://github.com/muzammilck/metatune/blob/master/engine_stream.py): live trainer · Generator-based training with epoch-by-epoch monitoring and adaptation. · `engine_stream.py`
- [Sklearn engine](https://github.com/muzammilck/metatune/blob/master/sklearn_engine.py): package trainer · Classical ML execution path that returns serializable artifacts. · `sklearn_engine.py`

### Persistent memory

Run history used for online learning.

- [Knowledge base](https://github.com/muzammilck/metatune/blob/master/knowledge_base.csv): run history · Persistent CSV memory of dataset DNA, params, and outcomes. · `knowledge_base.csv`

### Tooling & checks

Auxiliary generators and verification scripts.

- [Tests](https://github.com/muzammilck/metatune/tree/master/tests): contract tests · Integration and unit checks that protect subsystem boundaries. · `tests`
- [CAR data](https://github.com/muzammilck/metatune/blob/master/car.py): dataset helper · Auxiliary dataset/support script for demos and validation. · `car.py`
- [Audit data](https://github.com/muzammilck/metatune/blob/master/generate_audit_data.py): data generator · Creates realistic and stress-test datasets. · `generate_audit_data.py`
- [Verify core](https://github.com/muzammilck/metatune/blob/master/verify_metatune.py): check script · System-level verification for end-to-end behavior. · `verify_metatune.py`
- [Verify evolution](https://github.com/muzammilck/metatune/blob/master/verify_evolution.py): check script · Checks evolutionary and iterative optimization behavior. · `verify_evolution.py`

## Connections

- App → Data DNA: load/analyze · Interactive runs begin with dataset inspection.
- App → Bilevel: optimize · The UI drives the shared optimization loop.
- App → Stream engine: live train · Streamlit consumes epoch-level updates from the streaming engine.
- Pipeline → Data DNA: load/analyze · Batch runs begin with the same dataset analysis stage.
- Pipeline → Bilevel: optimize · CLI execution invokes the outer optimization loop.
- Pipeline → Torch engine: train · Headless runs can use the non-streaming torch trainer.
- Data DNA → Recommender: DNA · Meta-features are handed to algorithm ranking.
- Data DNA → Brain: meta-features · Dataset DNA also informs hyperparameter priors.
- Data DNA → Bilevel: task context · Outer-loop orchestration uses the task and difficulty context.
- Recommender → Designer: suggest · Recommendations are adapted into search suggestions.
- Recommender → Sklearn engine: classical path · Ranks candidates supported by the sklearn execution layer.
- Recommender → Torch engine: torch path · Ranks candidates supported by the torch execution layer.
- Recommender → Stream engine: live path · Streaming training is one of the executable options.
- Designer → Vizier stub: study/trial · The designer uses the local Vizier-like search abstractions.
- Designer → Brain: delegate · Suggestion logic ultimately relies on the meta-learner.
- Bilevel → Recommender: select · The outer loop asks for ranked candidates and budgets.
- Bilevel → Stream engine: evaluate · Candidates are evaluated with adaptive live training.
- Bilevel → Brain: feedback · Evaluation outcomes feed back into the learned decision layer.
- Stream engine → Brain: metrics · Training dynamics and outcomes update meta-learning.
- Torch engine → Knowledge base: results · Training outcomes ultimately become persistent run history.
- Sklearn engine → Knowledge base: package/run · Classical runs contribute results to the persistent memory loop.
- Brain → Knowledge base: append/history · The brain reads and appends to the CSV knowledge base.
- Tests → Data DNA: check · Tests cover analysis behavior and metadata extraction.
- Tests → Recommender: contract · Tests ensure recommender outputs are executable downstream.
- Tests → Brain: check · Tests validate meta-learner predictions and updates.
- Tests → Vizier stub: check · Tests validate the study/trial abstraction boundary.
- Tests → Pipeline: end-to-end · Integration tests exercise the orchestration path.
- CAR data → Audit data: support · Auxiliary dataset support is used for generation and demos.
- Audit data → Data DNA: sample data · Generated data can be analyzed by the core pipeline.
- Verify core → Pipeline: verify · Core verification script exercises the batch workflow.
- Verify evolution → Bilevel: verify · Evolution checks focus on the outer optimization loop.
