# sedjro2002/autoclean architecture

> Architecture diagram of the GitHub repository sedjro2002/autoclean, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sedjro2002/autoclean
- Repository: https://github.com/sedjro2002/autoclean

## Overview

##### Main Components

A. Core Processing Layer
- Data Processor Module
- Data Cleaner Module
- Normalizer Module
- Dimension Reduction Module
- Feature Engineering Module

B. Fairness & Analysis Layer
- Fairness Analyzer
- AI Code Analyzer
- Audit System

C. Interface Layer
- CLI Interface
- Python Package Interface
- Configuration Management

D. Output & Reporting Layer
- Report Generator
- Profile Generator
- Audit Trail Generator

##### Component Relationships

**Input Layer → Core Processing Layer → Fairness Layer → Output Layer**

**With parallel systems:**
- Configuration System (affecting all layers)
- AI Agent System (interacting with Fairness Layer)
- Audit System (monitoring all operations)

##### Specific Design Elements

Use these visual elements:
- Hexagons for AI components
- Double lines for configuration flow

##### Key Architectural Patterns to Highlight

A. Pipeline Architecture:
Main data flow from input through processing to output

B. Plugin System:
Demonstrate how different cleaning and analysis modules can be configured

C. Observer Pattern:
How the audit system monitors operations

##### Technical Details

A. External Dependencies:
- AI Model Integration (OpenAI/Local LLM)
- Data Processing Libraries

B. Configuration Points:
- Environment Variables
- JSON Configuration
- Runtime Parameters

Top Section:
- Input Interface (CLI/Python Package)
- Configuration Management

Middle Section:
- Core Processing Pipeline
- Fairness Analysis System
- AI Agent Integration

Bottom Section:
- Output Generation
- Report Creation
- Data Storage

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph Input
        IN[Data Input]
        CFG[Configuration Input]:::config
    end

    subgraph Core[Core Processing Layer]
        DP[Data Processor]:::core
        DC[Data Cleaner]:::core
        NM[Normalizer]:::core
        DR["Dimension Reduction"]:::core
        FE["Feature Engineering"]:::core
    end

    subgraph Fair[Fairness & Analysis Layer]
        FA[Fairness Analyzer]:::fairness
        CA[Code Analyzer]:::fairness
        AS[Audit System]:::fairness
    end

    subgraph Interface[Interface Layer]
        CLI[CLI Interface]:::interface
        PKG[Python Package]:::interface
        CM[Configuration Management]:::interface
    end

    subgraph AI[AI Agent System]
        LLM[LLM Integration]:::ai
        SPH[System Prompt Handler]:::ai
    end

    subgraph Output[Output & Reporting]
        RG[Report Generator]:::output
        PG[Profile Generator]:::output
        AT[Audit Trail Generator]:::output
        
        subgraph Files[Output Files]
            AD[Audit Trail]:::file
            FR[Fairness Results]:::file
            CD[Cleaned Data]:::file
            PR[Profile Report]:::file
        end
    end

    IN --> CLI
    IN --> PKG
    CFG --> CM

    CLI --> DP
    PKG --> DP
    CM --> DP

    DP --> DC
    DC --> NM
    NM --> DR
    DR --> FE

    FE --> FA
    FA --> CA
    CA --> AS

    LLM --> FA
    LLM --> CA
    SPH --> LLM

    FA --> RG
    CA --> RG
    AS --> AT

    RG --> PG
    RG --> Files
    AT --> AD
    PG --> PR
    FA --> FR
    DC --> CD

    %% Click Events
    click DP "https://github.com/Sedjro2002/AutoClean/blob/main/AutoClean/data_processor.py"
    click DC "https://github.com/Sedjro2002/AutoClean/blob/main/AutoClean/datacleaner.py"
    click NM "https://github.com/Sedjro2002/AutoClean/blob/main/AutoClean/normalizer.py"
    click DR "https://github.com/Sedjro2002/AutoClean/blob/main/AutoClean/dim_reduction.py"
    click FE "https://github.com/Sedjro2002/AutoClean/blob/main/preprocessing/feature_engineering.py"
    click FA "https://github.com/Sedjro2002/AutoClean/blob/main/AutoClean/fairness_analyzer.py"
    click CA "https://github.com/Sedjro2002/AutoClean/blob/main/AutoClean/ai_code_analyzer.py"
    click AS "https://github.com/Sedjro2002/AutoClean/blob/main/AutoClean/audit.py"
    click CLI "https://github.com/Sedjro2002/AutoClean/blob/main/AutoClean/cli.py"
    click CM "https://github.com/Sedjro2002/AutoClean/blob/main/AutoClean/config.py"
    click LLM "https://github.com/Sedjro2002/AutoClean/blob/main/AutoClean/ai_agent/main.py"
    click SPH "https://github.com/Sedjro2002/AutoClean/blob/main/AutoClean/ai_agent/system_prompt.txt"
    click RG "https://github.com/Sedjro2002/AutoClean/blob/main/AutoClean/report_generator.py"
    click Files "https://github.com/Sedjro2002/AutoClean/tree/main/AutoClean/output"
    click AD "https://github.com/Sedjro2002/AutoClean/blob/main/output/audit_trail.json"
    click FR "https://github.com/Sedjro2002/AutoClean/blob/main/output/fairness_results.json"
    click CD "https://github.com/Sedjro2002/AutoClean/blob/main/output/cleaned_data.csv"
    click PR "https://github.com/Sedjro2002/AutoClean/blob/main/AutoClean/output/comparative_profile.html"

    %% Styling
    classDef core fill:#a8d1f0,stroke:#333
    classDef fairness fill:#90EE90,stroke:#333
    classDef interface fill:#FFE5B4,stroke:#333
    classDef ai fill:#FFB6C1,stroke:#333
    classDef output fill:#DDA0DD,stroke:#333
    classDef config fill:#F0E68C,stroke:#333
    classDef file fill:#E6E6FA,stroke:#333

    %% Legend
    subgraph Legend
        L1[Core Processing]:::core
        L2[Fairness Components]:::fairness
        L3[Interface Components]:::interface
        L4[AI Components]:::ai
        L5[Output Components]:::output
        L6[Configuration]:::config
        L7[Files]:::file
    end
```
