# ayushpani/feature_selector architecture

> Architecture diagram of the GitHub repository ayushpani/feature_selector, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-23). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ayushpani/feature_selector
- Repository: https://github.com/ayushpani/feature_selector

## Overview

##### Project type and purpose
- This is an open-source Python library (feature_selector) for automated feature engineering and selection.
- It provides a pipeline to ingest raw tabular data, apply various feature transformers and selectors, evaluate feature subsets, and generate reports.

##### File structure
- Top-level modules:
- feature_engine_pro/
– engine.py         (pipeline orchestrator)
– data_processor.py (raw data ingestion & preprocessing)
– evaluator.py      (feature subset scoring)
– reporter.py       (result reporting)
- feature_engine_pro/selectors/ (pluggable feature-selection strategies)
- feature_engine_pro/transformers/ (pluggable feature-transformation strategies)
- examples/demo_usage.py shows how a user instantiates and runs the pipeline.
- tests/ verifies each component (selectors, transformers, data processor).

##### README
- Minimal, but project name “feature_selector” confirms focus on feature engineering.

a. Main Components
1. Client / Example Script (demo_usage.py)
2. Engine (orchestrates the pipeline)
3. DataProcessor (loads & cleans raw data)
4. Transformers (categorical_encoder, datetime_extractor, etc.)
5. Selectors (correlation, RFE, select_k_best, variance_threshold)
6. Evaluator (scores candidate feature sets)
7. Reporter (generates summary outputs)
8. External Dependencies (e.g., pandas, scikit-learn)
b. Relationships & Interactions
- The Client calls Engine.run() with configuration.
- Engine invokes DataProcessor to load and prepare data.
- Prepared data flows into a Transformer chain (ordered list of transformers).
- Transformed data passes to each Selector (either sequentially or in parallel).
- Selector outputs candidate feature subsets to Evaluator.
- Evaluator computes metrics (e.g., cross-validation scores) for each subset.
- Reporter consumes evaluation results to produce final reports (tables, plots).
c. Architectural Patterns & Design Principles
- Pipeline / Chain of Responsibility for transformers and selectors.
- Strategy pattern for selector and transformer plugins (base classes with interchangeable implementations).
- Single Responsibility: each module focuses on one stage of the pipeline.
- Testable components: each selector/transformer has its own test.
d. Key Technologies & Libraries
- Python 3
- pandas for data handling
- scikit-learn for algorithms (e.g., RFE, SelectKBest)
- pytest (tests)
- CLI or scripting (example usage)

##### Essential details to capture
- The plugin architecture: highlight base classes and concrete implementations.
- Data formats exchanged between stages (e.g., pandas DataFrame).
- Configuration injection point (how the Engine knows which steps to run).
- Test harness separate from production code.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Client Layer
    Client["Client / Example Script"]:::client
    click Client "https://github.com/ayushpani/feature_selector/blob/main/examples/demo_usage.py"

    %% Orchestrator
    Engine["Engine (Pipeline Orchestrator)"]:::orchestrator
    click Engine "https://github.com/ayushpani/feature_selector/blob/main/feature_engine_pro/engine.py"

    %% Data Processing
    DataProcessor["DataProcessor (Load & Clean)"]:::processor
    click DataProcessor "https://github.com/ayushpani/feature_selector/blob/main/feature_engine_pro/data_processor.py"

    %% Transformers Subpipeline
    subgraph "Transformers"
        direction TB
        BaseTransformer["BaseTransformer"]:::transformer
        click BaseTransformer "https://github.com/ayushpani/feature_selector/blob/main/feature_engine_pro/transformers/base_transformer.py"
        CategoricalEncoder["CategoricalEncoder"]:::transformer
        click CategoricalEncoder "https://github.com/ayushpani/feature_selector/blob/main/feature_engine_pro/transformers/categorical_encoder.py"
        DateTimeExtractor["DateTimeExtractor"]:::transformer
        click DateTimeExtractor "https://github.com/ayushpani/feature_selector/blob/main/feature_engine_pro/transformers/datetime_extractor.py"
        GroupAggregator["GroupAggregator"]:::transformer
        click GroupAggregator "https://github.com/ayushpani/feature_selector/blob/main/feature_engine_pro/transformers/group_aggregator.py"
        LogTransform["LogTransform"]:::transformer
        click LogTransform "https://github.com/ayushpani/feature_selector/blob/main/feature_engine_pro/transformers/log_transform.py"
        PolynomialFeatures["PolynomialFeatures"]:::transformer
        click PolynomialFeatures "https://github.com/ayushpani/feature_selector/blob/main/feature_engine_pro/transformers/polynomial_features.py"
    end

    %% Selectors Subpipeline
    subgraph "Selectors"
        direction TB
        BaseSelector["BaseSelector"]:::selector
        click BaseSelector "https://github.com/ayushpani/feature_selector/blob/main/feature_engine_pro/selectors/base_selector.py"
        CorrelationSelector["CorrelationSelector"]:::selector
        click CorrelationSelector "https://github.com/ayushpani/feature_selector/blob/main/feature_engine_pro/selectors/correlation.py"
        RFESelector["RFESelector"]:::selector
        click RFESelector "https://github.com/ayushpani/feature_selector/blob/main/feature_engine_pro/selectors/rfe.py"
        SelectKBest["SelectKBest"]:::selector
        click SelectKBest "https://github.com/ayushpani/feature_selector/blob/main/feature_engine_pro/selectors/select_k_best.py"
        VarianceThreshold["VarianceThreshold"]:::selector
        click VarianceThreshold "https://github.com/ayushpani/feature_selector/blob/main/feature_engine_pro/selectors/variance_threshold.py"
    end

    %% Evaluation and Reporting
    Evaluator["Evaluator (Subset Scoring)"]:::evaluation
    click Evaluator "https://github.com/ayushpani/feature_selector/blob/main/feature_engine_pro/evaluator.py"
    Reporter["Reporter (Result Reporting)"]:::reporter
    click Reporter "https://github.com/ayushpani/feature_selector/blob/main/feature_engine_pro/reporter.py"

    %% External Dependencies
    subgraph "External Libraries"
        direction TB
        Pandas["pandas"]:::external
        Sklearn["scikit-learn"]:::external
        PyTest["pytest"]:::external
    end

    %% Data Flow
    Client -->|"run(config)"| Engine
    Engine -->|"prepare data"| DataProcessor
    DataProcessor -->|"DataFrame"| BaseTransformer
    BaseTransformer --> CategoricalEncoder
    CategoricalEncoder --> DateTimeExtractor
    DateTimeExtractor --> GroupAggregator
    GroupAggregator --> LogTransform
    LogTransform --> PolynomialFeatures
    PolynomialFeatures -->|"transformed DataFrame"| BaseSelector
    BaseSelector --> CorrelationSelector
    CorrelationSelector --> RFESelector
    RFESelector --> SelectKBest
    SelectKBest --> VarianceThreshold
    VarianceThreshold -->|"feature subsets"| Evaluator
    Evaluator -->|"evaluation results"| Reporter
    Reporter -->|"reports"| Client

    %% External Links
    DataProcessor --> Pandas
    BaseTransformer --> Pandas
    CategoricalEncoder --> Sklearn
    RFESelector --> Sklearn
    SelectKBest --> Sklearn
    Evaluator --> Sklearn
    Reporter --> PyTest

    %% Styles
    classDef client fill:#e0f7fa,stroke:#006064;
    classDef orchestrator fill:#ffecb3,stroke:#ff6f00;
    classDef processor fill:#c8e6c9,stroke:#2e7d32;
    classDef transformer fill:#d1c4e9,stroke:#512da8;
    classDef selector fill:#f8bbd0,stroke:#c2185b;
    classDef evaluation fill:#b3e5fc,stroke:#0288d1;
    classDef reporter fill:#ffe0b2,stroke:#ef6c00;
    classDef external fill:#eeeeee,stroke:#777;
```
