How udayb97/retail-vendor-analytics works
Project type and purpose
• This is a data-pipeline and analytics project (“Retail Vendor Performance Analysis”) that automates ingestion, transformation, summary computation and exploratory analysis of retail/vendor data.
• It is not a web-app but a scripted ETL + analysis workflow producing both Jupyter Notebook outputs and a Power BI dashboard.
File structure
• ingestion_db.py: Python script using SQLAlchemy to read raw CSV files, define tables and load data into SQLite.
• get_vendor_summary.py: Python script using pandas/SQL to compute vendor-level metrics and write summary tables or CSVs.
• *.ipynb (Exploratory Data Analysis.ipynb & Vendor Performance Analysis.ipynb): Jupyter notebooks for interactive validation, incremental EDA and charting.
• Vendor_Perforamance_Analysis.pbix: Power BI dashboard file connecting to the summary data.
• images/: holds static assets (e.g., dashboard snapshots).
• README.md: project overview, technologies and workflow.
Extract architecture insights from the README
• Data Collection & Structuring → ingestion_db.py + SQLite
• Summary Generation → get_vendor_summary.py (pandas + SQLAlchemy)
• EDA & Validation → Jupyter Notebooks
• Visualization → Power BI (.pbix) fed by processed summary tables
• Logging is enabled in scripts for traceability
Main components and interactions
a. Data Sources (raw CSV files)
b. Ingestion Layer
• “ingestion_db.py” process
• Ingestion Layer writes to “SQLite Database”
c. Data Storage
• “SQLite Relational Database”
d. Transformation Layer
• “get_vendor_summary.py”
• Outputs vendor_summary table (and/or CSV export)
e. Analysis & Validation
• Two Jupyter Notebooks
• Notebooks use pandas, SQLAlchemy, SQL
f. Reporting & Dashboard
• Power BI (.pbix)
g. Logging & Monitoring
• Logging component integrated in both ingestion and transformation scripts
Architectural patterns and design principles
• Clear separation of concerns: Ingestion, Transformation, Analysis, Visualization
• Modularity: discrete scripts for each stage
• Single source of truth: relational DB (SQLite)
• Reproducibility: scripted steps + notebooks
• Lightweight embedded database for easy distribution
Technologies & frameworks
• Python 3.x
• pandas (data manipulation)
• SQLAlchemy + SQL (DB interface + complex joins)
• SQLite (storage)
• Jupyter Notebook (interactive analysis)
• Power BI (dashboard)
• Logging module (traceability)