How becoollll/nyctaxi-visualization works
Project Type and Purpose
• This is a full‐stack, web‐based data visualization and analysis dashboard built with the Dash framework (a Flask wrapper for Python).
• It ingests NYC taxi trip data (Parquet + GeoJSON), performs data cleaning, transformations, statistical tests, PCA, outlier detection, and renders interactive Plotly charts in the browser.
File Structure
Top-level:
• app.py – Dash app entrypoint, layout definition, callback registration
• data_manager.py – Data loading, caching, preprocessing utilities
• Dockerfile – Containerization instructions
• requirements.txt – Python dependencies
• datasets/ – Raw data files (Parquet, GeoJSON)
• assets/ – Static files (CSS, images)
• components/ – Self-contained feature modules, each exposing layout + callbacks
– home.py
– data_cleaning.py
– outlier.py
– pca.py
– normality_test.py
– data_transform.py
– numerical_plots.py
– categorical_plots.py
– statistics.py
Key Architecture Insights from README
• Single‐process Dash server hosting both “frontend” (React‐powered UI) and “backend” (Python callbacks and data logic).
• Modular component pattern: each feature is a plug-in under /components.
• Data flow: Browser ↔ Dash server ↔ data_manager ↔ raw datasets.
Main Components
A. Client Layer
• Web Browser (User Interface)
• Static Assets (CSS, images served from assets/)
B. Application Server (Dash)
• app.py (Dash app factory, page routing, global layout)
• Dash Callbacks (registered in app.py and component modules)
C. Feature Modules (components/)
• home, data_cleaning, outlier, pca, normality_test, data_transform, numerical_plots, categorical_plots, statistics
– Each exports a layout block and associated callbacks.
D. Data Layer
• data_manager.py (data loading, caching, preprocessing, chunking)
• datasets/ (yellow_tripdata_2024-07.parquet, taxi_zones.geojson)
E. Infrastructure
• Docker Container (Dockerfile build)
• Python runtime & Dependencies (requirements.txt)
Relationships & Data Flow
1. User opens browser → requests Dash app UI.
2. Dash serves HTML/CSS/JS + layout definitions → browser renders interactive components.
3. User interacts with UI (dropdowns, buttons) → triggers Dash callbacks on server.
4. Callback functions (in component modules) invoke data_manager to:
a. Read raw files (Parquet, GeoJSON)
b. Perform cleaning, transformation, statistical tests, PCA, outlier detection
c. Build Plotly figures or tables
5. Dash returns JSON payload → browser updates corresponding charts.
6. Docker container wraps the whole environment for deployment.
Architectural Patterns & Principles
• Modular, component-based design (separation of concerns: UI + logic per feature).
• Single monolithic service (no microservices); leverages Dash’s request‐callback model.
• Data access centralized in data_manager (DRY principle).
• Static vs. dynamic separation: assets/ for static, components/ + data_manager for dynamic logic.