How aymenezz/data-sceince works
Project type and purpose
• This is not a traditional full-stack application or microservice. It is a modular data-science/analysis repository, composed of multiple independent data‐processing pipelines. Each top-level folder (book_data, github, homes_data, netflix, etc.) contains raw datasets, Jupyter notebooks that clean and analyze those datasets, and derived output (plots, CSVs, figures). The overall purpose is to ingest public data dumps, perform exploratory data analysis, and produce visualizations and cleaned artifacts.
File structure
• Top‐level components:
• README.md: project overview (currently centered on Netflix visuals)
• data/: a miscellaneous Excel file (us_avg_tuition.xlsx)
• book_data/, github/, homes_data/, most_pupler_prog_lang/, netflix/, spotify/, student/, youtube/: each folder is one analytical “service” or pipeline
• Within each dataset folder:
• Raw data files (.csv, .zip, .xlsx)
• Jupyter notebooks (.ipynb) that read in the raw data, transform/clean it, and generate outputs
• Optional output folders (pictures/, plot/, cleaned CSVs)
• Checkpoints (.ipynb_checkpoints)
• No build scripts, no backend or frontend code—pure data pipelines.
README for additional insights
• The README currently shows example visual outputs for the Netflix analysis pipeline. It does not describe cross-cutting architecture but illustrates the end artifacts (charts for movies vs. TV shows by country).
a. Main components:
1. Data Sources – External CSVs, Excel, ZIP archives in each dataset folder
2. Data Storage – Raw-data folder structure under each module
3. Processing Engines – Jupyter Notebook kernels executing Python scripts (pandas, numpy, matplotlib/seaborn)
4. Output Artifacts – Generated plots, cleaned CSV files, image files under each module’s output subdirectory
5. Version Control – Git repository hosting all modules and notebooks
b. Relationships & interactions:
• Git integration overlay connecting all components (every code/notebook change is tracked)
c. Architectural patterns & design principles:
• Modular pipeline pattern: each dataset folder is a self-contained ETL/EDA pipeline
• Separation of concerns: raw data, transform logic (notebooks), and final outputs remain in distinct folders
• Repeatable & reproducible workflows: Jupyter notebooks with version-controlled code
d. Key technologies:
• Python, Jupyter Notebook
• pandas, numpy for data manipulation
• matplotlib / seaborn for plotting
• CSV, XLSX, ZIP for data storage
• Git for version control