# sandeepgundeboina/preppin-data architecture

> Architecture diagram of the GitHub repository sandeepgundeboina/preppin-data, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sandeepgundeboina/preppin-data
- Repository: https://github.com/sandeepgundeboina/preppin-data

## Overview

##### Main Components
- Notebook Environment: Since every solution is in a Jupyter Notebook (.ipynb), include a central “Data Analysis & Transformation” component that represents this notebook environment powered by Python.
- Data Processing Modules: Even though each notebook tackles a separate challenge (moderated by week, e.g., preppindata_2023W1.ipynb, preppindata_2023W2.ipynb, etc.), group them into a module that handles extraction of data, transformation logic, and potentially any computation routines. You might also mention a “DeltaLake.ipynb” as representing a more complex or different processing approach.
- Output & Visualization: Given the connection to Tableau Prep challenges, you may want to include a component that indicates output visualization or export – even if the notebooks simply produce outputs. This component could be “Visualization/Reporting” which might interface with Tableau or simply produce charts/graphs in Python.

##### Relationships and Interactions
- Inside the “Data Analysis & Transformation” block, illustrate separate subcomponents or modules (e.g., individual notebooks by week or grouped by year). Show that they are part of the same Python environment.
- Optionally, if any shared libraries, external modules, or configuration steps are used, include a “Dependencies & Libraries” component connected to the Data Processing module.

##### Architectural Patterns
- That the design is similar to an ETL pipeline (Extract, Transform, Load) even though it is implemented through standalone notebooks.
- Modularity: Each notebook (or set grouped by year or challenge grouping) represents an independent module that processes its data while following a similar processing pattern.
- Mention any design principles like reproducibility, iterative exploration, and data-driven analysis.

##### Additional Notes for the Principal Software Engineer
- Be very detailed when drawing out how each notebook (or group of notebooks) fits into the overall processing pipeline – although they are individual files, they adhere to a common challenge-solving workflow.
- It’s perfectly acceptable—even helpful—to over-document the pipeline since the educational and explorative nature of these notebooks benefits from clarity in structure.

## Diagram (Mermaid)

```mermaid
flowchart TD
    DS["Preppin Data Blog/Challenge Data"]:::external
    DEP["Dependencies & Libraries (Python,Pandas,NumPy)"]:::processing
    NB["Jupyter Notebook (Python)"]:::notebook
    subgraph "Data Analysis & Transformation"
        CN["Individual Challenge Notebooks"]:::processing
        DL["DeltaLake Notebook"]:::processing
    end
    OV["Visualization/Reporting"]:::output

    DS --> NB
    DEP --> NB
    NB --> CN
    NB --> DL
    CN --> OV
    DL --> OV

    click NB "https://github.com/sandeepgundeboina/preppin-data/blob/main/preppindata_2023W1.ipynb"
    click CN "https://github.com/sandeepgundeboina/preppin-data/blob/main/preppindata_2022W1.ipynb"
    click DL "https://github.com/sandeepgundeboina/preppin-data/blob/main/DeltaLake.ipynb"

    classDef external fill:#B3E5FC,stroke:#0277BD,stroke-width:2px;
    classDef notebook fill:#C8E6C9,stroke:#2E7D32,stroke-width:2px;
    classDef processing fill:#FFF9C4,stroke:#F9A825,stroke-width:2px;
    classDef output fill:#FFCCBC,stroke:#D84315,stroke-width:2px;
```
