How hkuennemann/msc_ba_projects works
Project Type and Purpose
• This is a monolithic “portfolio” repository structured as folders by course, each containing one or more projects.
• Purpose: showcase data-analysis, ML, programming and visualization workflows.
File Structure
• Top-level directories = course names (e.g., Advanced Data Analysis, Machine Learning, Web & Cloud Computing).
• Inside each: subfolders per project, containing:
• Raw/input data (CSV, XLSX, images, XML, etc.)
• Processing code (Jupyter notebooks, Python/R scripts, JS/HTML/CSS)
• Documentation (README.md, LICENSE, Sphinx docs, tests)
• Outputs (dashboards, HTML exports, notebooks, model artifacts).
1. Data Sources
• Raw files: CSVs, Excel, images, XML, API-fetched JSON.
• External API service (API-Football) in “League Lense” web project.
2. Processing Engines
• Python notebooks & scripts (Pandas, NumPy, Spark, Dask).
• R scripts/notebooks for Network Analytics.
• Jupyter front-end as interactive orchestration.
3. Analytical Workflows
• ETL pipelines in Data Curation (Extract → Transform → Load).
• ML pipelines in Machine Learning (pre-processing → training → evaluation → tuning).
• Graph analytics in Advanced Data Analysis (Bitcoin network) and Network Analytics (igraph).
4. Documentation & Testing
• Sphinx docs & unit tests in Advanced Programming (Project Icaras).
• README and LICENSE across modules.
5. Visualizations & Outputs
• Tableau dashboards (Tableau TWBX files) in Data Visualization.
• Static HTML/JS front-end in Web & Cloud Computing.
• Notebook HTML/exports (Rmd → HTML) in Network Analytics.
• From “Data Sources” → into each project’s “Processing Engine.”
• In Data Visualization: raw CSV → data-cleaning script → Tableau workbook → interactive dashboard.
• In Advanced Programming: code modules → tests → Sphinx → built HTML docs.
c. Architectural Patterns & Principles:
• Monorepo with modular, folder-based separation by course/project.
• Reproducible workflows: each project isolated with its own data folder, environment file (environment.yml).
• ETL pattern in Data Curation.
• ML pipeline pattern: Data → Feature Engineering → Model Training → Validation → Deployment artifacts.
• MVC-like separation in Web & Cloud Computing: static HTML/CSS/JS view, fetch calls as controller logic, API as model.
• Documentation-first in Advanced Programming: docs in Sphinx, tests ensure code quality.
d. Key Technologies & Frameworks (annotate next to each component):
• Python ecosystem: Pandas, NumPy, Matplotlib, scikit-learn, TensorFlow/Keras, cuDF/cuGraph (GPU).
• Big data: Spark, Dask.
• R ecosystem: igraph, ggplot2.
• JS ecosystem: fetch API, Promises, async/await.
• Visualization: Tableau.
• Documentation: Sphinx, pytest.
• Environments: Conda (environment.yml).