How enesmanan/llmautoeda works
What this project is (type + purpose)
This is a Python package/library that provides automated exploratory data analysis (EDA). It combines:
• Deterministic/statistical EDA (summaries, correlations, distributions, etc.)
• Visualization generation
• LLM-powered narrative/insights (via an OpenAI API key)
• Report generation (interactive HTML report)
From the README usage (AutoEDA(...).analyze(...).save_report(...)), the primary “product surface” is a library API (not a web service), with optional notebook/script examples.
How to read the file tree architecturally
Key directories/files:
• llm_auto_eda/ — package root
• llm_auto_eda/main.py — likely CLI or entrypoint glue
• data_loader.py — ingestion + parsing of data sources (CSV, maybe DataFrame)
• analyzer.py — orchestrator / façade that coordinates the pipeline (very likely where AutoEDA lives or is used)
• statistical_analyzer.py — computes numeric summaries, correlations, missingness, etc.
• visualizer.py — creates plots/figures (likely matplotlib/plotly/seaborn)
• llm_analyzer.py — sends curated stats + context to LLM and receives insights
• report_generator.py — composes HTML report from stats + visuals + LLM insights
• examples/ — consumer-side usage patterns
• setup.py, MANIFEST.in — packaging/distribution boundaries
This strongly suggests a pipeline-style layered design:
What the README implies about runtime interactions
Runtime calls look like:
Receives an analysis object exposing
• analysis.data_summary
• analysis.correlations
• analysis.insights
Optionally persists via analysis.save_report('...html')
• A public API surface (AutoEDA, analysis result model)
• A core pipeline in core/*
• An external dependency (OpenAI/LLM provider)
• Artifacts produced (plots, HTML report)
Public API layer
• AutoEDA (from llm_auto_eda/__init__.py and/or core/analyzer.py)
Core pipeline (llm_auto_eda/core)
• DataLoader (data_loader.py)
• StatisticalAnalyzer (statistical_analyzer.py)
• Visualizer (visualizer.py)
• LLMAnalyzer (llm_analyzer.py)
• ReportGenerator (report_generator.py)
• Analyzer/Orchestrator (analyzer.py) — this coordinates calls to the above and assembles the result
External systems
• LLM Provider API (OpenAI or compatible)
• (Optional to include if relevant) “Local file system” for reading CSV and writing HTML report
Inputs/outputs
• Inputs: CSV / DataFrame / data_source
• Outputs:
• In-memory: summaries/correlations/insights
• On-disk: analysis_report.html (and possibly embedded images)
• User → AutoEDA.analyze()
• AutoEDA/analyzer.py → DataLoader (reads CSV / returns DataFrame)
• Analyzer → StatisticalAnalyzer (returns summary tables, missingness, correlations)
• Analyzer → Visualizer (consumes DataFrame + stats → returns plots/figures)
• Analyzer → LLMAnalyzer
• Inputs to LLMAnalyzer: domain context, exclude_columns, selected stats, maybe schema/profile
• LLMAnalyzer → External “LLM API” (network call)
• External LLM API → LLMAnalyzer (insights text / structured suggestions)
• Analyzer → ReportGenerator (consumes stats + plots + LLM insights → returns HTML content)
• User / analysis.save_report() → File system (writes HTML)
Important: make the LLM call *explicit* as a network boundary; it matters for latency, cost, failure modes.
Architectural patterns/design principles used
• Façade / Orchestrator pattern: AutoEDA / Analyzer presents a simple API over multiple subsystems.
• Separation of concerns: each core module has a single responsibility.
• Python package distribution (setup.py)
• LLM provider (OpenAI mentioned explicitly in README)
For plotting/report tech, only label generically unless you confirm in code:
• “Plotting library (matplotlib/plotly/seaborn)” as a generic sub-label inside Visualizer
• “HTML templating/report composition” inside ReportGenerator
What
• Actor: “Data Scientist / Analyst”
• System: “LLM Auto EDA (Python library)”
• External system: “OpenAI API”
• External resource: “Dataset (CSV/Parquet/DF)” + “File system (report output)”
Purpose: shows boundaries and dependencies at a glance.
• AutoEDA API
• Analyzer/Orchestrator
• DataLoader
• StatisticalAnalyzer
• Visualizer
• LLMAnalyzer
• ReportGenerator
• Analysis Result Model (data summary, correlations, insights, plots, report html)
• AutoEDA / Public API → llm_auto_eda/__init__.py (+ implementation likely in core/analyzer.py)
• Orchestrator → llm_auto_eda/core/analyzer.py
• Loaders → llm_auto_eda/core/data_loader.py
• Stats → llm_auto_eda/core/statistical_analyzer.py
• LLM → llm_auto_eda/core/llm_analyzer.py
• Viz → llm_auto_eda/core/visualizer.py
• Report → llm_auto_eda/core/report_generator.py
• CLI/Entry (if used) → llm_auto_eda/main.py
• Examples/Consumers → examples/example_usage.py, examples/example_notebook.ipynb
Final guidance: don’t overthink, just decompose
For “best and most accurate,” prefer more components, not fewer:
• Separate “Stats computation” from “LLM prompt building”
• Separate “Plot creation” from “Report assembly”
• Explicitly show “Domain/exclude_columns config” flowing into analyzers and LLM prompt
• Explicitly show “Analysis object” as the aggregation point returned to the user