How rudra-narayan-shaw/eda_dashboard works
Project type and purpose
• It uses Streamlit as the front-end UI framework and DuckDB as an embedded analytical database.
• The primary goal is exploratory data analysis (EDA) on some data sources, with data ingestion, transformation, storage, and visualization all handled within a single process.
File structure
• Top-level scripts:
• app.py → Streamlit application entry point (UI + orchestration)
• data_mani.py → Data transformation and preprocessing logic
• database.py → DuckDB connection, schema setup, query abstractions
• eda.py → EDA functions, charts/plots, analytics routines
• Ancillary files:
• requirements.txt → Dependencies (streamlit, duckdb, pandas, plotly, etc.)
• license and README for documentation
• No separate “frontend/” or “backend/” folders—everything lives in one codebase, but logical separation is in modules.
README
• Very brief: “EDA Dashboard using Streamlit and DuckDB.”
• Confirms technologies and high-level use case (dashboard for exploratory analysis).
a. Main components
1. Streamlit UI (app.py)
2. Data Transformation Module (data_mani.py)
3. Database Module (database.py)
4. EDA Module (eda.py)
5. DuckDB Embedded Database (file‐based)
b. Relationships & interactions
• User interacts with Streamlit UI → triggers callback in app.py
• app.py orchestrates:
– Calls database.py to ingest/load raw data into DuckDB
– Invokes data_mani.py to clean/transform data
– Queries transformed data via database.py
– Passes query results to eda.py for visualization computations
• eda.py returns chart objects (e.g., Plotly or Streamlit charts) back to app.py
• Streamlit renders charts in the browser
c. Architectural patterns & design principles
• Modular single-process architecture (no microservices)
• Separation of concerns:
– UI layer (app.py)
– Persistence layer (database.py + DuckDB)
– Business logic layers (data_mani.py / eda.py)
• Embedded analytical database (DuckDB) for fast in-memory ingestion and SQL queries
• Function-based modules rather than classes—focus on script-style orchestration
d. Key technologies/frameworks
• Python 3.x
• Streamlit (UI)
• DuckDB (embedded database / SQL engine)
• pandas or pyarrow (likely for data frames)
• Plotly or Streamlit built‐in charts for visualization