# ebrahim-tambuwala/sofnexis_data_analysis_with_python architecture

> Architecture diagram of the GitHub repository ebrahim-tambuwala/sofnexis_data_analysis_with_python, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-02-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ebrahim-tambuwala/sofnexis_data_analysis_with_python
- Repository: https://github.com/ebrahim-tambuwala/sofnexis_data_analysis_with_python

## Overview

##### What this project is (type + purpose)
This repository is not a deployed “system” in the traditional sense; it’s a **data analysis / data science notebook workspace** for an internship. The primary “runtime” is a **local (or hosted) Jupyter environment** executing **Python notebooks** against **CSV datasets**, producing **cleaned CSV outputs** and analytical results (EDA, time-series analysis, hypothesis testing, ML modeling).

##### How the file tree maps to architecture
Top-level structure:
- `Task_1/` … `Task_5/`: five independent analysis modules (each is effectively a “component”).
- Each task is centered around one or more **.ipynb notebooks** and **CSV datasets**.
- Task 1 is special: it contains **three sub-datasets** (Customers, Organizations, People) and produces **cleaned CSV outputs**.

Key artifacts:
- **Inputs**:
- `Task_1/*/customers-100.csv`, `organizations-100.csv`, `people-100.csv`
- `Task_2/titanic.csv`
- `Task_3/airline-passengers.csv`
- `Task_4/ab_data.csv`
- `Task_5/titanic.csv`
- **Processing**:
- Jupyter notebooks (`Task*_*.ipynb`) contain the transformation/analysis logic.
- **Outputs**:
- `Task_1/*/cleaned_*_data.csv` are explicit persisted outputs.
- Other tasks likely generate plots, metrics, model results, but they are not persisted in the repo (so represent them as “in-notebook outputs” unless you confirm they write files).

##### Actor / Operator
- “Analyst / Intern” running notebooks.

##### Execution Environment
- “Jupyter Notebook (Python)” as the primary runtime.

##### Data Sources (CSV files)
- Task 1 raw CSVs (customers/org/people)
- Titanic dataset (used in Task 2 + Task 5; show it reused)
- Airline passengers dataset (Task 3)
- A/B dataset (Task 4)

##### Processing Modules (Notebooks)
- Notebook as a component:
- `Task1_Data_Cleaning_Customers.ipynb`
- `Task1_Data_Cleaning_Organization.ipynb`
- `Task1_Data_Cleaning_People.ipynb`
- `Task2_EDA.ipynb`
- `Task3_Time_Series_Analysis.ipynb`
- `Task4_Hypothesis_Testing.ipynb`
- `Task_5_Machine_Learning_Modeling.ipynb`

##### Outputs / Artifacts
- Explicit persisted outputs (Task 1 cleaned CSVs) should be drawn as **artifact storage**:
- `cleaned_customers_data.csv`
- `cleaned_organizations_data.csv`
- `cleaned_people_data.csv`
- “EDA Visualizations + Summary Stats”
- “Time-series plots + forecast results (if any)”
- “Hypothesis test metrics (p-values, confidence intervals)”
- “Trained model + evaluation metrics” (even if not saved, it’s still an output of the notebook execution)

##### Task 1 (Data Cleaning pipeline)
- `customers-100.csv` → `Task1_Data_Cleaning_Customers.ipynb` → `cleaned_customers_data.csv`
- `organizations-100.csv` → `Task1_Data_Cleaning_Organization.ipynb` → `cleaned_organizations_data.csv`
- `people-100.csv` → `Task1_Data_Cleaning_People.ipynb` → `cleaned_people_data.csv`

##### Task 2 (EDA)
- `titanic.csv` → `Task2_EDA.ipynb` → “EDA outputs (charts/tables)”

##### Task 3 (Time Series)
- `airline-passengers.csv` → `Task3_Time_Series_Analysis.ipynb` → “Time-series outputs”

##### Task 4 (Hypothesis Testing)
- `ab_data.csv` → `Task4_Hypothesis_Testing.ipynb` → “Test results (p-value, effect size, CI)”

##### Task 5 (ML Modeling)
- `titanic.csv` → `Task_5_Machine_Learning_Modeling.ipynb` → “Model + metrics”
- Important: show `titanic.csv` as a **shared input** reused by Task 2 and Task 5.

##### Orchestration
- “Analyst” → triggers execution of each notebook in Jupyter.
- Jupyter → reads/writes CSVs on local filesystem (this repo).

##### Architectural patterns present (and how to reflect them)
This repo follows a **task-oriented, notebook-per-problem** pattern:
- Each `Task_*` directory is effectively a **self-contained analysis module**.
- Inputs are local flat files (CSV), not databases or APIs.
- Outputs are either persisted files (Task 1) or ephemeral notebook results (Tasks 2–5).

- Inside each task boundary: show notebook + its dataset(s) + outputs.

##### What not to overcomplicate
- Don’t invent services (APIs, DBs, queues) that aren’t present.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Global / Workspace Architecture (top-down)
    Analyst["Analyst / Intern"]:::actor
    Jupyter["Jupyter Notebook Environment\n(Python Runtime)"]:::runtime
    PyLibs["Analytical Stack\n(pandas, numpy, matplotlib/seaborn,\nscipy/stats, statsmodels, scikit-learn)"]:::deps
    RepoFS["Repo Workspace / Local File System"]:::storage

    Analyst -->|"runs_notebooks"| Jupyter
    Jupyter -.->|"depends_on"| PyLibs
    Jupyter -->|"reads_writes_CSVs"| RepoFS

    %% Shared datasets (avoid duplication)
    Titanic["titanic.csv\n(Shared Input)"]:::csvShared

    %% Task Modules (bounded by subgraphs, arranged vertically)
    subgraph "Task 1: Data Cleaning"
        direction TB
        T1Dir["Task_1/"]:::module

        subgraph "Customer Dataset"
            direction TB
            T1CustIn["customers-100.csv"]:::csvIn
            T1CustNB["Task1_Data_Cleaning_Customers.ipynb"]:::notebook
            T1CustOut["cleaned_customers_data.csv"]:::csvOut

            T1CustIn -->|"reads"| T1CustNB
            T1CustNB -->|"writes"| T1CustOut
        end

        subgraph "Organization Dataset"
            direction TB
            T1OrgIn["organizations-100.csv"]:::csvIn
            T1OrgNB["Task1_Data_Cleaning_Organization.ipynb"]:::notebook
            T1OrgOut["cleaned_organizations_data.csv"]:::csvOut

            T1OrgIn -->|"reads"| T1OrgNB
            T1OrgNB -->|"writes"| T1OrgOut
        end

        subgraph "People Dataset"
            direction TB
            T1PeopleIn["people-100.csv"]:::csvIn
            T1PeopleNB["Task1_Data_Cleaning_People.ipynb"]:::notebook
            T1PeopleOut["cleaned_people_data.csv"]:::csvOut

            T1PeopleIn -->|"reads"| T1PeopleNB
            T1PeopleNB -->|"writes"| T1PeopleOut
        end

        RepoFS -->|"stores"| T1Dir
        T1Dir -->|"contains"| T1CustIn
        T1Dir -->|"contains"| T1OrgIn
        T1Dir -->|"contains"| T1PeopleIn
    end

    subgraph "Task 2: Exploratory Data Analysis (EDA)"
        direction TB
        T2Dir["Task_2/"]:::module
        T2NB["Task2_EDA.ipynb"]:::notebook
        T2Out["EDA Outputs\n(charts, tables, summary statistics)\n(in-notebook)"]:::result

        Titanic -->|"reads"| T2NB
        T2NB -->|"produces"| T2Out

        RepoFS -->|"stores"| T2Dir
        T2Dir -->|"contains"| T2NB
    end

    subgraph "Task 3: Time Series Analysis"
        direction TB
        T3Dir["Task_3/"]:::module
        T3In["airline-passengers.csv"]:::csvIn
        T3NB["Task3_Time_Series_Analysis.ipynb"]:::notebook
        T3Out["Time-series Outputs\n(plots, decomposition, forecast results if any)\n(in-notebook)"]:::result

        T3In -->|"reads"| T3NB
        T3NB -->|"produces"| T3Out

        RepoFS -->|"stores"| T3Dir
        T3Dir -->|"contains"| T3In
        T3Dir -->|"contains"| T3NB
    end

    subgraph "Task 4: Hypothesis Testing"
        direction TB
        T4Dir["Task_4/"]:::module
        T4In["ab_data.csv"]:::csvIn
        T4NB["Task4_Hypothesis_Testing.ipynb"]:::notebook
        T4Out["Hypothesis Test Outputs\n(p-values, effect size, confidence intervals)\n(in-notebook)"]:::result

        T4In -->|"reads"| T4NB
        T4NB -->|"produces"| T4Out

        RepoFS -->|"stores"| T4Dir
        T4Dir -->|"contains"| T4In
        T4Dir -->|"contains"| T4NB
    end

    subgraph "Task 5: Machine Learning Modeling"
        direction TB
        T5Dir["Task_5/"]:::module
        T5NB["Task_5_Machine_Learning_Modeling.ipynb"]:::notebook
        T5Out["ML Outputs\n(trained model, feature importance, evaluation metrics)\n(in-notebook)"]:::result

        Titanic -->|"reads"| T5NB
        T5NB -->|"produces"| T5Out

        RepoFS -->|"stores"| T5Dir
        T5Dir -->|"contains"| T5NB
    end

    %% Orchestration: Jupyter triggers notebook execution (per-task)
    Jupyter -->|"executes"| T1CustNB
    Jupyter -->|"executes"| T1OrgNB
    Jupyter -->|"executes"| T1PeopleNB
    Jupyter -->|"executes"| T2NB
    Jupyter -->|"executes"| T3NB
    Jupyter -->|"executes"| T4NB
    Jupyter -->|"executes"| T5NB

    %% Shared dataset storage relationship
    RepoFS -->|"stores"| Titanic

    %% Click Events (from component_mapping)
    click T1Dir "https://github.com/ebrahim-tambuwala/sofnexis_data_analysis_with_python/tree/main/Task_1/"
    click T1CustIn "https://github.com/ebrahim-tambuwala/sofnexis_data_analysis_with_python/blob/main/Task_1/Customer_Dataset/customers-100.csv"
    click T1CustNB "https://github.com/ebrahim-tambuwala/sofnexis_data_analysis_with_python/blob/main/Task_1/Customer_Dataset/Task1_Data_Cleaning_Customers.ipynb"
    click T1CustOut "https://github.com/ebrahim-tambuwala/sofnexis_data_analysis_with_python/blob/main/Task_1/Customer_Dataset/cleaned_customers_data.csv"
    click T1OrgIn "https://github.com/ebrahim-tambuwala/sofnexis_data_analysis_with_python/blob/main/Task_1/Organization_Dataset/organizations-100.csv"
    click T1OrgNB "https://github.com/ebrahim-tambuwala/sofnexis_data_analysis_with_python/blob/main/Task_1/Organization_Dataset/Task1_Data_Cleaning_Organization.ipynb"
    click T1OrgOut "https://github.com/ebrahim-tambuwala/sofnexis_data_analysis_with_python/blob/main/Task_1/Organization_Dataset/cleaned_organizations_data.csv"
    click T1PeopleIn "https://github.com/ebrahim-tambuwala/sofnexis_data_analysis_with_python/blob/main/Task_1/People_Dataset/people-100.csv"
    click T1PeopleNB "https://github.com/ebrahim-tambuwala/sofnexis_data_analysis_with_python/blob/main/Task_1/People_Dataset/Task1_Data_Cleaning_People.ipynb"
    click T1PeopleOut "https://github.com/ebrahim-tambuwala/sofnexis_data_analysis_with_python/blob/main/Task_1/People_Dataset/cleaned_people_data.csv"

    click T2Dir "https://github.com/ebrahim-tambuwala/sofnexis_data_analysis_with_python/tree/main/Task_2/"
    click Titanic "https://github.com/ebrahim-tambuwala/sofnexis_data_analysis_with_python/blob/main/Task_2/titanic.csv"
    click T2NB "https://github.com/ebrahim-tambuwala/sofnexis_data_analysis_with_python/blob/main/Task_2/Task2_EDA.ipynb"

    click T3Dir "https://github.com/ebrahim-tambuwala/sofnexis_data_analysis_with_python/tree/main/Task_3/"
    click T3In "https://github.com/ebrahim-tambuwala/sofnexis_data_analysis_with_python/blob/main/Task_3/airline-passengers.csv"
    click T3NB "https://github.com/ebrahim-tambuwala/sofnexis_data_analysis_with_python/blob/main/Task_3/Task3_Time_Series_Analysis.ipynb"

    click T4Dir "https://github.com/ebrahim-tambuwala/sofnexis_data_analysis_with_python/tree/main/Task_4/"
    click T4In "https://github.com/ebrahim-tambuwala/sofnexis_data_analysis_with_python/blob/main/Task_4/ab_data.csv"
    click T4NB "https://github.com/ebrahim-tambuwala/sofnexis_data_analysis_with_python/blob/main/Task_4/Task4_Hypothesis_Testing.ipynb"

    click T5Dir "https://github.com/ebrahim-tambuwala/sofnexis_data_analysis_with_python/tree/main/Task_5/"
    click T5NB "https://github.com/ebrahim-tambuwala/sofnexis_data_analysis_with_python/blob/main/Task_5/Task_5_Machine_Learning_Modeling.ipynb"

    %% Styles
    classDef actor fill:#FDE68A,stroke:#B45309,stroke-width:2,color:#111827
    classDef runtime fill:#BFDBFE,stroke:#1D4ED8,stroke-width:2,color:#0B1220
    classDef deps fill:#E0E7FF,stroke:#4338CA,stroke-width:1.5,color:#0B1220,stroke-dasharray: 4 3
    classDef storage fill:#E5E7EB,stroke:#374151,stroke-width:1.5,color:#111827
    classDef module fill:#DBEAFE,stroke:#2563EB,stroke-width:1.5,color:#0B1220
    classDef notebook fill:#93C5FD,stroke:#1D4ED8,stroke-width:2,color:#0B1220
    classDef csvIn fill:#F3F4F6,stroke:#4B5563,stroke-width:1.5,color:#111827
    classDef csvOut fill:#BBF7D0,stroke:#15803D,stroke-width:2,color:#052e16
    classDef result fill:#A7F3D0,stroke:#047857,stroke-width:1.5,color:#052e16
    classDef csvShared fill:#FFF7ED,stroke:#C2410C,stroke-width:3,color:#111827
```
