dmehta2396/segmentation_analysis_v2

How dmehta2396/segmentation_analysis_v2 works

1. This repository is a Python batch-analysis pipeline for segmenting entities by month and comparing how they move over time. It is not an application server; the main execution path is from command-line scripts into analysis modules that produce HTML and Excel deliverables.

2. The architecture is organized around three analysis phases:

- Phase 00: data understanding and quality checks

- Phase 01: base-month vs current-month comparison

- Phase 02: movement analysis across segments

Each phase is isolated under analysis/phase_XX/, which makes the phases the main functional boundaries in the system.

3. The scripts/ directory is the orchestration layer. run_phase_00.py, run_phase_01.py, and run_phase_02.py act as entry points that select a phase, load data, run the corresponding analysis steps, and trigger output generation. generate_data.py is a supporting utility for creating sample input data.

4. Core data handling lives in src/. data_loader.py is the ingestion boundary for reading source files, data_generator.py supports synthetic or sample data creation, validators.py handles integrity checks, and utils.py contains shared helpers. This layer is the primary bridge between raw CSV inputs and the phase-specific analytics.

5. Configuration is centralized in config/settings.py, which likely defines column mappings, month selections, file naming conventions, and display colors. This matters architecturally because the rest of the pipeline appears to be schema-driven rather than hardcoded to one dataset layout.

6. analysis/phase_00/ focuses on exploratory and diagnostic outputs: dataset overview, base month distribution, selected month inspection, data quality, metrics overview, and reference data. This phase establishes the baseline context and validates the inputs before comparison or movement analysis.

7. analysis/phase_01/ contains the base-vs-current comparison logic, with base_vs_current.py as the main module. This phase compares entity counts and segment composition between two time points, producing growth/decline summaries and ranking changes. It acts as the bridge between descriptive profiling and movement-focused analysis.

8. analysis/phase_02/ is the most specialized subsystem. movement_core.py appears to hold shared transition logic used by modules for movement matrix, new entities, lost entities, movement analysis, Sankey diagram generation, and summary reporting. This suggests a shared computation kernel feeding multiple visual/report outputs.

9. Visualization concerns are separated into visualizations/. charts.py likely contains reusable Plotly chart builders, while styles.py centralizes consistent formatting and color themes. That separation keeps analysis code focused on business logic and reduces duplication across phases.

10. Exporting is handled independently in exports/excel_exporter.py, which indicates that Excel generation is a first-class output path rather than an afterthought. The separation of HTML visualization generation and XLSX export means each analysis can emit both interactive and tabular artifacts from the same computed result set.

11. The data flow is straightforward: raw input data is loaded and validated, normalized using config mappings, processed by a phase module, rendered into Plotly-based visual outputs, and exported to formatted spreadsheets. The README’s data/raw, data/processed, and outputs/phase_XX/{html,xlsx} conventions describe the persistence boundaries of that flow.

12. Overall, the architecture is a layered offline analytics pipeline: configuration at the top, shared ingestion/validation in src/, phase-specific analytical computation in analysis/, reusable presentation logic in visualizations/, and output generation in exports/. An architecture diagram should emphasize the three analysis phases as parallel consumers of the same core data layer and config, with outputs split into HTML and Excel artifacts.

Main entry points: run_phase_02.py Run phase 02 · run_phase_00.py Run phase 00 · settings.py Settings · data_loader.py Data loader

Components 22

Scripts

Command-line entry points and orchestration

Run phase 00 · Starts the exploratory and quality-check workflow scripts/run_phase_00.py
Run phase 01 · Starts the base-vs-current comparison workflow scripts/run_phase_01.py
Run phase 02 · Starts the movement analysis workflow scripts/run_phase_02.py

Config

Centralized settings and schema mappings

Settings · Shared mappings, month selection, naming, and display settings config/settings.py

Core data

Ingestion, validation, shared helpers, and sample data support

Data loader · Reads source files into the analysis pipeline src/data_loader.py
Validators · Validates input data quality and required structure src/validators.py
Utils · Common utility functions used across phases src/utils.py
Data generator · Creates synthetic or example inputs for the pipeline src/data_generator.py

Phase 00

Dataset understanding and quality checks

Overview · Summarizes dataset structure and baseline context analysis/phase_00/a_dataset_overview.py
Quality checks · Checks data completeness and consistency analysis/phase_00/d_data_quality.py
Reference data · Shows supporting reference tables and baseline slices analysis/phase_00/f_reference_data.py

Phase 01

Base-month vs current-month comparison

Base vs current · Compares entity counts and segment composition across two months analysis/phase_01/base_vs_current.py

Phase 02

Movement analysis across segments

Movement core · Shared movement logic used by all phase 02 reports analysis/phase_02/movement_core.py
Movement matrix · Computes segment-to-segment movement counts analysis/phase_02/a_movement_matrix.py
New entities · Identifies entities entering the system or segment analysis/phase_02/b_new_entities.py
Lost entities · Identifies entities leaving the system or segment analysis/phase_02/c_lost_entities.py
Movement analysis · Explains movement patterns across segments analysis/phase_02/d_movement_analysis.py
Sankey diagram · Builds the flow diagram for segment transitions analysis/phase_02/e_sankey_diagram.py
Summary · Consolidates the phase 02 findings analysis/phase_02/f_movement_summary.py

Visuals

Reusable charting and styling

Charts · Reusable Plotly chart constructors visualizations/charts.py
Styles · Shared colors and formatting rules visualizations/styles.py

Exports

Spreadsheet output generation

Excel export · Writes formatted spreadsheet deliverables exports/excel_exporter.py

Connections 35

Run phase 00 → Settings: loads settings · Uses centralized mappings and output conventions
Run phase 00 → Data loader: loads data · Reads source inputs for analysis
Run phase 00 → Validators: validates · Checks input integrity before reporting
Run phase 00 → Overview: runs · Executes the phase 00 analysis modules
Run phase 00 → Quality checks: runs · Executes data quality diagnostics
Run phase 00 → Reference data: runs · Produces reference and baseline views
Run phase 01 → Settings: loads settings · Uses shared comparison configuration
Run phase 01 → Data loader: loads data · Reads source inputs for comparison
Run phase 01 → Base vs current: runs · Executes base-vs-current comparison
Run phase 02 → Settings: loads settings · Uses shared movement configuration
Run phase 02 → Data loader: loads data · Reads source inputs for movement analysis
Run phase 02 → Movement core: uses core · Delegates transition logic to the shared kernel
Run phase 02 → Movement matrix: runs · Generates movement matrix output
Run phase 02 → New entities: runs · Generates new-entity output
Run phase 02 → Lost entities: runs · Generates lost-entity output
Run phase 02 → Movement analysis: runs · Produces analytical movement commentary
Run phase 02 → Sankey diagram: runs · Produces transition visualization
Run phase 02 → Summary: runs · Compiles final phase summary
Overview → Charts: renders with · Uses shared chart primitives for HTML output
Quality checks → Charts: renders with · Uses charts for diagnostic presentation
Base vs current → Charts: renders with · Uses shared chart primitives for comparison views
Movement matrix → Charts: renders with · Uses chart builders for matrix display
Sankey diagram → Charts: renders with · Uses chart builders for Sankey output
Overview → Styles: applies · Uses shared theme and formatting
Base vs current → Styles: applies · Uses shared theme and formatting
Summary → Styles: applies · Uses shared theme and formatting
Overview → Excel export: exports · Writes spreadsheet deliverables
Base vs current → Excel export: exports · Writes spreadsheet deliverables
Summary → Excel export: exports · Writes spreadsheet deliverables
Data loader → Validators: passes through · Loaded records are validated before analysis
Data loader → Utils: uses · Relies on common helper functions
Data generator → Data loader: feeds · Generates sample inputs for ingestion
Settings → Overview: configures · Shapes month selection and labeling
Settings → Base vs current: configures · Shapes base/current comparison rules
Settings → Movement core: configures · Defines segment and transition behavior

Drawn by GitDiagram on June 17, 2026 from the repository's file tree, README and source files. Read as Markdown ·