# sanjay-krishna-mv/product-analytics-project architecture

> Architecture diagram of the GitHub repository sanjay-krishna-mv/product-analytics-project, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sanjay-krishna-mv/product-analytics-project
- Repository: https://github.com/sanjay-krishna-mv/product-analytics-project

## Overview

1. **Project type and overall shape**
   - This repository is a **product analytics pipeline** rather than an application service.
   - The architecture is organized as a **data flow from raw session data → warehouse transformation → analytics notebooks → business insights and prediction**.

2. **Primary data source**
   - The main input data lives in `data/raw/cleaned_speakers_data.csv`.
   - Per the README, this CSV is uploaded into **Google BigQuery** as a `raw_sessions` table.
   - The `data/processed/events.csv` file appears to be a local processed/exported artifact of the engineered event stream.

3. **Warehouse and storage boundary**
   - BigQuery is the central system boundary for structured analytics data.
   - The repository models two key warehouse layers:
     - `raw_sessions`: the ingested source table
     - `events`: the transformed, granular event stream used for analysis
   - This separation is important for architecture because raw ingestion and analytical modeling are intentionally decoupled.

4. **Transformation layer**
   - `sql/01_feature_engineering_events.sql` is the main transformation script.
   - It converts session-level raw data into an **event-level model** that resembles product analytics platforms like Amplitude or Pendo.
   - The SQL directory also contains downstream analytical queries, so SQL is both the transformation and metric-computation layer.

5. **Core analytical workflows**
   - `sql/02_product_metrics_dau_wau_mau.sql` computes product health metrics such as DAU/WAU/MAU.
   - `sql/03_funnel_analysis_query.sql` performs funnel analysis to locate drop-off points.
   - `sql/04_deep_dive_dropped_user_segmentation.sql` and `sql/05_deep_dive_dropped_users_by_device_traffic.sql` support segmentation and root-cause analysis around funnel abandonment.
   - These queries represent the business-intelligence layer built on top of the engineered event table.

6. **Python notebook layer**
   - The `notebooks/` directory is the primary consumer-facing analysis layer.
   - `exploratory_data_analysis.ipynb` inspects data quality, distributions, and relationships.
   - `product_metrics_analysis.ipynb` operationalizes metric analysis and visualization.
   - `conversion_prediction_model.ipynb` builds the machine learning layer for purchase prediction.

7. **Machine learning boundary**
   - The predictive model is trained in Python using the engineered dataset and product-behavior features.
   - The README indicates use of **scikit-learn** and a **Random Forest Classifier** to predict conversion for signed-in users.
   - This is a separate subsystem from reporting: it consumes analytic features and produces a scoring/prediction outcome.

8. **Documentation as architecture map**
   - The `documentation/` folder contains phase-by-phase design and analysis notes.
   - `DATA_SETUP.md` defines environment and BigQuery setup, while `phase1` through `phase6` document the end-to-end analytical lifecycle.
   - In architecture terms, documentation captures the operating model, assumptions, and interpretation of each layer.

9. **Execution environment and dependencies**
   - The repo is designed to run in a local **Python + Jupyter** environment.
   - `requirements.txt` includes data science and BigQuery integration libraries such as `pandas`, `numpy`, `matplotlib`, `seaborn`, `scikit-learn`, `google-cloud-bigquery`, and `db-dtypes`.
   - BigQuery is the external cloud service that materially affects the architecture; Python notebooks are the local computation and visualization runtime.

10. **Output and artifact layer**
   - `screenshots/` stores visual evidence of schemas, charts, funnel results, and model evaluation outputs.
   - These are presentation artifacts, not system inputs, but they help document the analytical results and are useful when diagramming the pipeline’s end states.

11. **End-to-end data flow**
   - The practical flow is: **CSV source → BigQuery raw table → SQL feature engineering → BigQuery events table → SQL metrics/funnel queries and Python notebooks → insights, recommendations, and conversion model**.
   - The main architectural boundary is between **data warehouse processing** and **analysis/modeling in notebooks**.

12. **What to emphasize in a diagram**
   - Show **BigQuery** as the central hub.
   - Distinguish **raw ingestion**, **event transformation**, **metric/funnel analysis**, and **predictive modeling** as separate stages.
   - Treat notebooks and SQL scripts as distinct consumers of the engineered data, with documentation and screenshots as supporting layers around the core pipeline.

Main entry points: [`notebooks/product_metrics_analysis.ipynb`](https://github.com/sanjay-krishna-mv/product-analytics-project/blob/main/notebooks/product_metrics_analysis.ipynb) (Metrics Notebook), [`screenshots`](https://github.com/sanjay-krishna-mv/product-analytics-project/tree/main/screenshots) (Screenshots), [`sql/01_feature_engineering_events.sql`](https://github.com/sanjay-krishna-mv/product-analytics-project/blob/main/sql/01_feature_engineering_events.sql) (Feature SQL), [`sql/02_product_metrics_dau_wau_mau.sql`](https://github.com/sanjay-krishna-mv/product-analytics-project/blob/main/sql/02_product_metrics_dau_wau_mau.sql) (Metrics SQL)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_group_sources["Sources & Storage"]
  node_node_raw_csv["Raw CSV<br/>source data"]
  node_node_events_csv["Events CSV<br/>processed data<br/>[events.csv]"]
end

subgraph group_group_warehouse["Warehouse & SQL"]
  node_node_bigquery_raw[("raw_sessions<br/>BigQuery table")]
  node_node_bigquery_events[("events<br/>BigQuery table")]
  node_node_feature_sql["Feature SQL<br/>transform script"]
  node_node_metrics_sql["Metrics SQL<br/>analytics query"]
  node_node_funnel_sql["Funnel SQL<br/>analytics query"]
  node_node_segmentation_sql["Segmentation SQL<br/>deep-dive query"]
  node_node_device_sql["Device SQL<br/>deep-dive query"]
end

subgraph group_group_analysis["Analysis & ML"]
  node_node_eda_nb["EDA Notebook<br/>jupyter analysis"]
  node_node_metrics_nb["Metrics Notebook<br/>jupyter analysis"]
  node_node_model_nb["Prediction Notebook<br/>ml notebook"]
end

subgraph group_group_support["Docs & Artifacts"]
  node_node_readme["README<br/>project overview<br/>[README.md]"]
  node_node_setup_md["Data Setup<br/>environment guide<br/>[DATA_SETUP.md]"]
  node_node_phase_docs["Phase Notes<br/>analysis docs<br/>[documentation.md]"]
  node_node_screenshots["Screenshots<br/>artifacts"]
  node_node_requirements["Dependencies<br/>python deps<br/>[requirements.txt]"]
end

node_node_raw_csv -->|"load"| node_node_bigquery_raw
node_node_bigquery_raw -->|"transform"| node_node_feature_sql
node_node_feature_sql -->|"build"| node_node_bigquery_events
node_node_bigquery_events -.->|"export"| node_node_events_csv
node_node_bigquery_events -->|"analyze"| node_node_metrics_sql
node_node_bigquery_events -->|"analyze"| node_node_funnel_sql
node_node_bigquery_events -->|"analyze"| node_node_segmentation_sql
node_node_bigquery_events -->|"analyze"| node_node_device_sql
node_node_bigquery_events -->|"explore"| node_node_eda_nb
node_node_bigquery_events -->|"visualize"| node_node_metrics_nb
node_node_bigquery_events -->|"train"| node_node_model_nb
node_node_metrics_sql -.->|"support"| node_node_metrics_nb
node_node_funnel_sql -.->|"inform"| node_node_metrics_nb
node_node_segmentation_sql -.->|"capture"| node_node_screenshots
node_node_device_sql -.->|"capture"| node_node_screenshots
node_node_model_nb -.->|"evaluate"| node_node_screenshots
node_node_readme -.->|"references"| node_node_setup_md
node_node_setup_md -.->|"configures"| node_node_requirements
node_node_phase_docs -.->|"documents"| node_node_readme

click node_node_raw_csv "https://github.com/sanjay-krishna-mv/product-analytics-project/blob/main/data/raw/cleaned_speakers_data.csv"
click node_node_events_csv "https://github.com/sanjay-krishna-mv/product-analytics-project/blob/main/data/processed/events.csv"
click node_node_feature_sql "https://github.com/sanjay-krishna-mv/product-analytics-project/blob/main/sql/01_feature_engineering_events.sql"
click node_node_metrics_sql "https://github.com/sanjay-krishna-mv/product-analytics-project/blob/main/sql/02_product_metrics_dau_wau_mau.sql"
click node_node_funnel_sql "https://github.com/sanjay-krishna-mv/product-analytics-project/blob/main/sql/03_funnel_analysis_query.sql"
click node_node_segmentation_sql "https://github.com/sanjay-krishna-mv/product-analytics-project/blob/main/sql/04_deep_dive_dropped_user_segmentation.sql"
click node_node_device_sql "https://github.com/sanjay-krishna-mv/product-analytics-project/blob/main/sql/05_deep_dive_dropped_users_by_device_traffic.sql"
click node_node_eda_nb "https://github.com/sanjay-krishna-mv/product-analytics-project/blob/main/notebooks/exploratory_data_analysis.ipynb"
click node_node_metrics_nb "https://github.com/sanjay-krishna-mv/product-analytics-project/blob/main/notebooks/product_metrics_analysis.ipynb"
click node_node_model_nb "https://github.com/sanjay-krishna-mv/product-analytics-project/blob/main/notebooks/conversion_prediction_model.ipynb"
click node_node_readme "https://github.com/sanjay-krishna-mv/product-analytics-project/blob/main/README.md"
click node_node_setup_md "https://github.com/sanjay-krishna-mv/product-analytics-project/blob/main/documentation/DATA_SETUP.md"
click node_node_phase_docs "https://github.com/sanjay-krishna-mv/product-analytics-project/blob/main/documentation/documentation.md"
click node_node_screenshots "https://github.com/sanjay-krishna-mv/product-analytics-project/tree/main/screenshots"
click node_node_requirements "https://github.com/sanjay-krishna-mv/product-analytics-project/blob/main/requirements.txt"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_node_raw_csv,node_node_events_csv toneBlue
class node_node_bigquery_raw,node_node_bigquery_events,node_node_feature_sql,node_node_metrics_sql,node_node_funnel_sql,node_node_segmentation_sql,node_node_device_sql toneAmber
class node_node_eda_nb,node_node_metrics_nb,node_node_model_nb toneMint
class node_node_readme,node_node_setup_md,node_node_phase_docs,node_node_screenshots,node_node_requirements toneRose
```

## Components

### Sources & Storage

Raw and processed data boundaries

- [Raw CSV](https://github.com/sanjay-krishna-mv/product-analytics-project/blob/main/data/raw/cleaned_speakers_data.csv): source data · Primary session-level input file · `data/raw/cleaned_speakers_data.csv`
- [Events CSV](https://github.com/sanjay-krishna-mv/product-analytics-project/blob/main/data/processed/events.csv): processed data · Local engineered event-stream artifact · `data/processed/events.csv`

### Warehouse & SQL

BigQuery tables and analytic SQL layers

- **raw_sessions**: BigQuery table · Ingested source table in BigQuery
- **events**: BigQuery table · Transformed event-level analytics table
- [Feature SQL](https://github.com/sanjay-krishna-mv/product-analytics-project/blob/main/sql/01_feature_engineering_events.sql): transform script · Session-to-event feature engineering · `sql/01_feature_engineering_events.sql`
- [Metrics SQL](https://github.com/sanjay-krishna-mv/product-analytics-project/blob/main/sql/02_product_metrics_dau_wau_mau.sql): analytics query · DAU/WAU/MAU product health metrics · `sql/02_product_metrics_dau_wau_mau.sql`
- [Funnel SQL](https://github.com/sanjay-krishna-mv/product-analytics-project/blob/main/sql/03_funnel_analysis_query.sql): analytics query · Conversion funnel analysis and drop-off · `sql/03_funnel_analysis_query.sql`
- [Segmentation SQL](https://github.com/sanjay-krishna-mv/product-analytics-project/blob/main/sql/04_deep_dive_dropped_user_segmentation.sql): deep-dive query · Dropped-user segmentation analysis · `sql/04_deep_dive_dropped_user_segmentation.sql`
- [Device SQL](https://github.com/sanjay-krishna-mv/product-analytics-project/blob/main/sql/05_deep_dive_dropped_users_by_device_traffic.sql): deep-dive query · Dropped users by device and traffic source · `sql/05_deep_dive_dropped_users_by_device_traffic.sql`

### Analysis & ML

Notebook-based exploration, reporting, and prediction

- [EDA Notebook](https://github.com/sanjay-krishna-mv/product-analytics-project/blob/main/notebooks/exploratory_data_analysis.ipynb): jupyter analysis · Exploration of distributions and data quality · `notebooks/exploratory_data_analysis.ipynb`
- [Metrics Notebook](https://github.com/sanjay-krishna-mv/product-analytics-project/blob/main/notebooks/product_metrics_analysis.ipynb): jupyter analysis · Visual product metrics analysis · `notebooks/product_metrics_analysis.ipynb`
- [Prediction Notebook](https://github.com/sanjay-krishna-mv/product-analytics-project/blob/main/notebooks/conversion_prediction_model.ipynb): ml notebook · Conversion prediction with Random Forest · `notebooks/conversion_prediction_model.ipynb`

### Docs & Artifacts

Documentation and presentation outputs

- [README](https://github.com/sanjay-krishna-mv/product-analytics-project/blob/main/README.md): project overview · Top-level project narrative and usage · `README.md`
- [Data Setup](https://github.com/sanjay-krishna-mv/product-analytics-project/blob/main/documentation/DATA_SETUP.md): environment guide · BigQuery and local environment setup · `documentation/DATA_SETUP.md`
- [Phase Notes](https://github.com/sanjay-krishna-mv/product-analytics-project/blob/main/documentation/documentation.md): analysis docs · Phase-by-phase documentation set · `documentation/documentation.md`
- [Screenshots](https://github.com/sanjay-krishna-mv/product-analytics-project/tree/main/screenshots): artifacts · Charts, schemas, and result captures · `screenshots`
- [Dependencies](https://github.com/sanjay-krishna-mv/product-analytics-project/blob/main/requirements.txt): python deps · Runtime libraries for notebooks and BigQuery · `requirements.txt`

## Connections

- Raw CSV → raw_sessions: load · Upload raw sessions into BigQuery
- raw_sessions → Feature SQL: transform · Use raw warehouse data as SQL input
- Feature SQL → events: build · Materialize engineered event stream
- events → Events CSV: export · Local processed artifact mirrors event table
- events → Metrics SQL: analyze · Compute product health metrics
- events → Funnel SQL: analyze · Run funnel drop-off analysis
- events → Segmentation SQL: analyze · Segment users for root-cause analysis
- events → Device SQL: analyze · Break down dropped users by device/traffic
- events → EDA Notebook: explore · Notebook exploration on engineered data
- events → Metrics Notebook: visualize · Notebook-driven metric analysis
- events → Prediction Notebook: train · Use engineered features for prediction
- Metrics SQL → Metrics Notebook: support · Notebook narrative may reference SQL metrics
- Funnel SQL → Metrics Notebook: inform · Funnel results feed analysis and recommendations
- Segmentation SQL → Screenshots: capture · Save result visuals and summaries
- Device SQL → Screenshots: capture · Save supporting charts and outputs
- Prediction Notebook → Screenshots: evaluate · Store model evaluation artifacts
- README → Data Setup: references · Project overview points to setup guidance
- Data Setup → Dependencies: configures · Describes local runtime dependencies
- Phase Notes → README: documents · Phase notes explain the end-to-end workflow