# shapniljoy/eda--and--inferential--analysis architecture

> Architecture diagram of the GitHub repository shapniljoy/eda--and--inferential--analysis, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/shapniljoy/eda--and--inferential--analysis
- Repository: https://github.com/shapniljoy/eda--and--inferential--analysis

## Overview

1. **Repository purpose**
   - This is a **data analysis repo**, not an application or service.
   - Its job is to turn the **Global Cancer Patients 2015–2024** Kaggle dataset into an exploratory report with statistical findings, visualizations, and a small predictive modeling component.

2. **Primary artifacts**
   - `eda-and-inferential-analysis.ipynb` is the **main working notebook** and likely contains the full analysis workflow.
   - `EDA Report.pdf` is the **published/exported report**.
   - `README.md` is the **narrative summary** of the analysis and findings.

3. **Core data flow**
   - The workflow starts with a patient-level dataset containing demographics, risk factors, cancer characteristics, costs, survival, and a target severity score.
   - Data is then explored univariately, tested statistically in pairwise/multivariate views, and finally fed into a Random Forest feature-importance analysis.
   - Outputs are visualizations, statistical test results, model scores, and the final report narrative.

4. **Analytical pipeline boundary**
   - The repo is organized around a **single notebook-driven pipeline** rather than separated modules or services.
   - The main boundary is between:
     - **raw dataset / source data**
     - **analysis notebook**
     - **reporting artifacts** (`README.md`, PDF)
   - There is no evidence of a database, API, UI, or scheduled job layer.

5. **Descriptive statistics subsystem**
   - The first analysis phase covers distributions of age, gender, country, risk factors, cancer type, stage, treatment cost, and severity.
   - This subsystem is responsible for histograms, KDE plots, boxplots, Q-Q plots, bar charts, and pie charts.
   - Its output is baseline characterization of the dataset and outlier/normality assessment.

6. **Inferential statistics subsystem**
   - The second phase evaluates relationships using **Pearson/Spearman correlations**, **t-tests**, **one-way ANOVA**, **chi-square tests**, and **OLS regression**.
   - This part of the notebook answers questions like:
     - whether risk factors relate to severity
     - whether treatment cost differs across countries or genders
     - whether stage and cancer type are associated
   - These tests form the statistical backbone of the report.

7. **Predictive modeling subsystem**
   - A **Random Forest Regressor** is used as the only explicit machine-learning model.
   - It is trained with an **80/20 split** and used for two targets:
     - `Target_Severity_Score`
     - `Survival_Years`
   - Feature importance is used as a ranking mechanism to identify the strongest predictors of severity.

8. **Feature engineering / variable handling**
   - The notebook distinguishes between:
     - **numerical standardized risk factors** such as genetic risk, air pollution, alcohol use, smoking, obesity
     - **categorical variables** such as gender, country, year, cancer type, and stage
   - It also constructs at least one **interaction term**: `Genetic_Risk × Smoking`, plus a transformed version for regression testing.
   - This is the main modeling boundary where raw fields are turned into analytical features.

9. **Outcome-focused analytical themes**
   - The report is organized around seven thematic questions:
     - distributions of demographics and clinical variables
     - risk factors vs severity
     - early-stage detection by cancer type
     - predictors of severity and survival
     - treatment cost burden
     - stage effects on cost and survival
     - genetic risk × smoking interaction
   - These themes define the repository’s logical structure more than code modules do.

10. **Visualization and reporting layer**
   - Visual output is generated with **Seaborn, Matplotlib, and Plotly**.
   - The notebook likely serves both as computation and presentation, with charts embedded inline.
   - The PDF is the final consumable artifact for sharing findings outside the notebook environment.

11. **Key architectural constraint**
   - The dataset appears to be **prestructured and standardized**: several risk factors share the same scale and distribution shape.
   - Because of that, the notebook is focused on **analysis and interpretation**, not data ingestion, cleansing pipelines, or heavy transformation logic.

12. **Practical diagram takeaway**
   - If you were drawing this repo, it would be best represented as:
     - one source dataset entering a notebook-centric analysis pipeline,
     - branching into descriptive stats, inferential tests, and Random Forest modeling,
     - converging into written conclusions and exported report artifacts.
   - The repo is essentially a **single analytical workflow with reporting outputs**, not a multi-component system.

Main entry points: [`eda-and-inferential-analysis.ipynb`](https://github.com/shapniljoy/eda--and--inferential--analysis/blob/master/eda-and-inferential-analysis.ipynb) (Main notebook), [`README.md`](https://github.com/shapniljoy/eda--and--inferential--analysis/blob/master/README.md) (README), [`EDA Report.pdf`](https://github.com/shapniljoy/eda--and--inferential--analysis/blob/HEAD/EDA%20Report.pdf) (PDF report)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_g_source["Source Data"]
  node_source_dataset[("Cancer dataset<br/>source data")]
end

subgraph group_g_analysis["Notebook Pipeline"]
  node_notebook["Main notebook<br/>analysis notebook"]
  node_feature_engineering["Feature setup<br/>preprocessing"]
  node_descriptive_stats["Descriptive stats<br/>eda"]
  node_dist_plots(("Distribution plots<br/>visualization"))
  node_inferential_stats["Inferential tests<br/>statistics"]
  node_risk_severity(("Risk vs severity<br/>hypothesis"))
  node_group_effects(("Group effects<br/>hypothesis"))
  node_association_tests(("Association tests<br/>hypothesis"))
  node_interaction_term["Risk interaction<br/>feature"]
  node_rf_model["Random forest<br/>modeling"]
  node_feature_importance(("Feature importance<br/>ranking"))
  node_model_scores(("Model scores<br/>evaluation"))
end

subgraph group_g_output["Reporting"]
  node_notebook_outputs["Plots and tables<br/>analysis output"]
  node_readme["README<br/>narrative<br/>[README.md]"]
  node_pdf_report["PDF report<br/>published report<br/>[EDA Report.pdf]"]
end

node_source_dataset -->|"analyze"| node_notebook
node_notebook -->|"prepare"| node_feature_engineering
node_feature_engineering -->|"branch"| node_descriptive_stats
node_descriptive_stats -->|"render"| node_dist_plots
node_feature_engineering -->|"test"| node_inferential_stats
node_inferential_stats -->|"answer"| node_risk_severity
node_inferential_stats -->|"answer"| node_group_effects
node_inferential_stats -->|"answer"| node_association_tests
node_feature_engineering -->|"derive"| node_interaction_term
node_feature_engineering -->|"fit"| node_rf_model
node_interaction_term -->|"include"| node_rf_model
node_rf_model -->|"evaluate"| node_model_scores
node_rf_model -->|"rank"| node_feature_importance
node_notebook -->|"emit"| node_notebook_outputs
node_notebook_outputs -->|"summarize"| node_readme
node_notebook_outputs -->|"publish"| node_pdf_report

click node_notebook "https://github.com/shapniljoy/eda--and--inferential--analysis/blob/master/eda-and-inferential-analysis.ipynb"
click node_readme "https://github.com/shapniljoy/eda--and--inferential--analysis/blob/master/README.md"
click node_pdf_report "https://github.com/shapniljoy/eda--and--inferential--analysis/blob/master/EDA Report.pdf"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_source_dataset toneBlue
class node_notebook,node_feature_engineering,node_descriptive_stats,node_dist_plots,node_inferential_stats,node_risk_severity,node_group_effects,node_association_tests,node_interaction_term,node_rf_model,node_feature_importance,node_model_scores toneAmber
class node_notebook_outputs,node_readme,node_pdf_report toneMint
```

## Components

### Source Data

Patient-level cancer dataset and derived analytical fields

- **Cancer dataset**: source data · Global Cancer Patients 2015–2024 patient-level dataset

### Notebook Pipeline

Exploratory analysis, statistical testing, and modeling in the main notebook

- [Main notebook](https://github.com/shapniljoy/eda--and--inferential--analysis/blob/master/eda-and-inferential-analysis.ipynb): analysis notebook · Primary workflow for exploration, tests, modeling, and visualization · `eda-and-inferential-analysis.ipynb`
- **Feature setup**: preprocessing · Standardized numeric risks, categorical fields, and interaction terms
- **Descriptive stats**: eda · Distributions, outliers, normality checks, and baseline summaries
- **Distribution plots**: visualization · Histograms, KDEs, boxplots, Q-Q plots, bar charts, and pie charts
- **Inferential tests**: statistics · Correlation, t-test, ANOVA, chi-square, and OLS analysis
- **Risk vs severity**: hypothesis · Tests how risk factors relate to target severity
- **Group effects**: hypothesis · Compares cost, survival, and severity across categories
- **Association tests**: hypothesis · Checks stage, cancer type, and other categorical relationships
- **Risk interaction**: feature · Genetic risk × smoking interaction and transformed variant
- **Random forest**: modeling · Random Forest regressor used for severity and survival prediction
- **Feature importance**: ranking · Ranks predictors of severity from the fitted forest model
- **Model scores**: evaluation · 80/20 split evaluation for severity and survival targets

### Reporting

Narrative and exported deliverables

- **Plots and tables**: analysis output · Inline figures, statistical results, and model summaries
- [README](https://github.com/shapniljoy/eda--and--inferential--analysis/blob/master/README.md): narrative · Written summary of findings and interpretation · `README.md`
- [PDF report](https://github.com/shapniljoy/eda--and--inferential--analysis/blob/HEAD/EDA%20Report.pdf): published report · Exported report for sharing outside the notebook · `EDA Report.pdf`

## Connections

- Cancer dataset → Main notebook: analyze · Notebook consumes the structured patient dataset
- Main notebook → Feature setup: prepare · Numeric, categorical, and interaction features are defined
- Feature setup → Descriptive stats: branch · Prepared variables feed exploratory summaries
- Descriptive stats → Distribution plots: render · Summaries are visualized as distribution charts
- Feature setup → Inferential tests: test · Engineered variables support statistical testing
- Inferential tests → Risk vs severity: answer · Evaluates relationships between risk factors and severity
- Inferential tests → Group effects: answer · Compares outcomes across countries, genders, and stages
- Inferential tests → Association tests: answer · Tests categorical associations such as stage and cancer type
- Feature setup → Risk interaction: derive · Creates the genetic risk and smoking interaction feature
- Feature setup → Random forest: fit · Model uses the prepared analytical feature set
- Risk interaction → Random forest: include · Interaction feature is available to the model and regression tests
- Random forest → Model scores: evaluate · Produces train-test scores for severity and survival targets
- Random forest → Feature importance: rank · Outputs predictor importance for severity interpretation
- Main notebook → Plots and tables: emit · Notebook generates figures, tables, and statistical summaries
- Plots and tables → README: summarize · Key findings are written into the repository summary
- Plots and tables → PDF report: publish · Final report is exported as a PDF artifact