How sandeepgundeboina/preppin-data works
Main Components
• Notebook Environment: Since every solution is in a Jupyter Notebook (.ipynb), include a central “Data Analysis & Transformation” component that represents this notebook environment powered by Python.
• Data Processing Modules: Even though each notebook tackles a separate challenge (moderated by week, e.g., preppindata_2023W1.ipynb, preppindata_2023W2.ipynb, etc.), group them into a module that handles extraction of data, transformation logic, and potentially any computation routines. You might also mention a “DeltaLake.ipynb” as representing a more complex or different processing approach.
• Output & Visualization: Given the connection to Tableau Prep challenges, you may want to include a component that indicates output visualization or export – even if the notebooks simply produce outputs. This component could be “Visualization/Reporting” which might interface with Tableau or simply produce charts/graphs in Python.
Relationships and Interactions
• Inside the “Data Analysis & Transformation” block, illustrate separate subcomponents or modules (e.g., individual notebooks by week or grouped by year). Show that they are part of the same Python environment.
• Optionally, if any shared libraries, external modules, or configuration steps are used, include a “Dependencies & Libraries” component connected to the Data Processing module.
Architectural Patterns
• That the design is similar to an ETL pipeline (Extract, Transform, Load) even though it is implemented through standalone notebooks.
• Modularity: Each notebook (or set grouped by year or challenge grouping) represents an independent module that processes its data while following a similar processing pattern.
• Mention any design principles like reproducibility, iterative exploration, and data-driven analysis.
Additional Notes for the Principal Software Engineer
• Be very detailed when drawing out how each notebook (or group of notebooks) fits into the overall processing pipeline – although they are individual files, they adhere to a common challenge-solving workflow.
• It’s perfectly acceptable—even helpful—to over-document the pipeline since the educational and explorative nature of these notebooks benefits from clarity in structure.