# matcraft94/ds-projects architecture

> Architecture diagram of the GitHub repository matcraft94/ds-projects, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/matcraft94/ds-projects
- Repository: https://github.com/matcraft94/ds-projects

## Overview

##### Project Type and Purpose
- Recognize that this repository is not a monolithic application but rather a multi-project portfolio.
- Each folder (such as “academic-performance-prediction”, “actuarial-loss-prediction”, “market-risk-analysis”, “neural-pdes-solver”, and “pdes-simulations”) represents an independent analysis pipeline or research tool.
- The README further identifies additional projects (educational-assessment, geospatial-analysis, math-research-tools) that follow the same pattern—a set of data-driven, research, or predictive modeling applications.

##### Analyzing the File Structure
- Top-Level Directories: Each project directory contains its own set of notebooks, scripts, data folders, and sometimes pre-trained models or processed results.
- File Types: Notice the use of Jupyter Notebooks (.ipynb), Python scripts (.py), R Markdown files (.Rmd), and data files (CSV, Excel, etc.), indicating that each project is self-contained focusing on experimentations and modeling rather than a multi-tier production application.
- Configuration Elements: The presence of a requirements.txt supports dependency management for Python-based projects.
- Consistency: Many projects include data processing, model development, and visualization which can be abstracted as common sub-modules across the projects.

##### Key Components to Highlight
- Individual Projects:
- Actuarial Loss Prediction: Show a component for preprocessing, XGBoost model training (emphasizing hyperparameter optimization), and statistical analysis.
- Neural PDEs Solver: Illustrate a component that encompasses physics-informed neural networks (PINNs), different experiment notebooks (e.g., COVID modeling, double pendulum), and GPU memory management.
- Academic Performance Prediction: Highlight the use of LightGBM (with GPU acceleration), feature engineering, and predictive analysis.
- Shared Supporting Modules:
- Data Handling: Represent a common data ingestion/preprocessing layer that is tailored for different data formats (CSV, Excel).
- Visualization: A component dedicated to generating plots, dashboards (as seen in market-risk-analysis and educational assessment references in the README).
- Experimentation & Evaluation: Separate out model evaluation, backtesting, and metric analysis steps.

##### Relationships and Interactions
- Use connection lines if some projects share libraries or similar dependencies (e.g., Pandas, NumPy, scikit-learn), underlining the common ecosystem.

##### Architectural Patterns and Design Principles
- Modularity: Reinforce that each project is autonomous with its own processing pipeline.
- Separation of Concerns: Break down the components into clear stages (data ingestion, processing, modeling, visualization).
- Reusability: Although projects are isolated, similar patterns across projects (e.g., notebook-based experimentation) can be highlighted as reusable patterns across the portfolio.
- Scalability: Some projects (like Neural PDEs, Market Risk Analysis) represent production-level complexity, which might be shown with components like GPU optimization for scaling compute or iterative training loops.

## Diagram (Mermaid)

```mermaid
graph TD
    Portfolio["Portfolio: Data Science Projects"]:::portfolio

    %% Academic Performance Prediction Subgraph
    subgraph "Academic Performance Prediction"
        A_Data["Academic Data"]:::data
        A_Notebook["Academic Notebook"]:::notebook
    end
    Portfolio -->|"ingests"| A_Data
    A_Data -->|"analyzed by"| A_Notebook

    %% Actuarial Loss Prediction Subgraph
    subgraph "Actuarial Loss Prediction"
        AL_Rmd["Analysis (Rmd)"]:::notebook
        AL_HTML["Reporting (HTML)"]:::report
    end
    Portfolio -->|"starts"| AL_Rmd
    AL_Rmd -->|"generates"| AL_HTML

    %% Market Risk Analysis Subgraph
    subgraph "Market Risk Analysis"
        MR_Data["Market Risk Data"]:::data
        MR_Entry["Entry Point (main.py)"]:::entry
        MR_Pipeline["Modeling Pipeline (src)"]:::pipeline
        MR_Artifacts["Model Artifacts"]:::report
    end
    Portfolio -->|"feeds"| MR_Data
    MR_Data -->|"triggers"| MR_Entry
    MR_Entry -->|"processes via"| MR_Pipeline
    MR_Pipeline -->|"produces"| MR_Artifacts
    MR_Pipeline -- "backtesting" --> MR_Data

    %% Neural PDEs Solver Subgraph
    subgraph "Neural PDEs Solver"
        NPD_Notebooks["Experiment Notebooks"]:::notebook
        NPD_Data["Data Folder"]:::data
    end
    Portfolio -->|"initiates"| NPD_Notebooks
    NPD_Notebooks -->|"utilizes"| NPD_Data

    %% PDES Simulations Subgraph
    subgraph "PDES Simulations"
        PDES_Notebooks["Simulation Notebooks"]:::notebook
    end
    Portfolio -->|"includes"| PDES_Notebooks

    %% Shared Supporting Modules Subgraph
    subgraph "Common Modules"
        Req["requirements.txt"]:::module
        Readme["README.md"]:::module
    end
    Portfolio ---|"dependency"| Req
    Portfolio ---|"documentation"| Readme

    %% Click Events
    click A_Notebook "https://github.com/matcraft94/ds-projects/blob/main/academic-performance-prediction/academic_performance_prediction.ipynb"
    click A_Data "https://github.com/matcraft94/ds-projects/tree/main/academic-performance-prediction/data"
    
    click AL_Rmd "https://github.com/matcraft94/ds-projects/blob/main/actuarial-loss-prediction/trabajo_final_E_ARIAS.Rmd"
    click AL_HTML "https://github.com/matcraft94/ds-projects/blob/main/actuarial-loss-prediction/trabajo_final_E_ARIAS.html"
    
    click MR_Entry "https://github.com/matcraft94/ds-projects/blob/main/market-risk-analysis/main.py"
    click MR_Pipeline "https://github.com/matcraft94/ds-projects/tree/main/market-risk-analysis/src"
    click MR_Data "https://github.com/matcraft94/ds-projects/tree/main/market-risk-analysis/data"
    click MR_Artifacts "https://github.com/matcraft94/ds-projects/blob/main/market-risk-analysis/best_model.pth"
    
    click NPD_Notebooks "https://github.com/matcraft94/ds-projects/tree/main/neural-pdes-solver"
    click NPD_Data "https://github.com/matcraft94/ds-projects/tree/main/neural-pdes-solver/data"
    
    click PDES_Notebooks "https://github.com/matcraft94/ds-projects/tree/main/pdes-simulations"
    
    click Req "https://github.com/matcraft94/ds-projects/blob/main/requirements.txt"
    click Readme "https://github.com/matcraft94/ds-projects/blob/main/README.md"

    %% Styles
    classDef portfolio fill:#FFD700,stroke:#B8860B,stroke-width:2px;
    classDef data fill:#FFCCCB,stroke:#FF0000,stroke-width:2px;
    classDef notebook fill:#C1FFC1,stroke:#008000,stroke-width:2px;
    classDef pipeline fill:#ADD8E6,stroke:#0000FF,stroke-width:2px;
    classDef report fill:#FFB6C1,stroke:#DB7093,stroke-width:2px;
    classDef entry fill:#DDA0DD,stroke:#800080,stroke-width:2px;
    classDef module fill:#E6E6FA,stroke:#4B0082,stroke-width:2px;
```
