# davidecallegaro/big_data_in_finance_2020 architecture

> Architecture diagram of the GitHub repository davidecallegaro/big_data_in_finance_2020, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/davidecallegaro/big_data_in_finance_2020
- Repository: https://github.com/davidecallegaro/big_data_in_finance_2020

## Overview

##### Project type and purpose
- This is a data-science / network-simulation project implemented entirely in Jupyter notebooks.
- Purpose: build a financial network from bilateral claims and credit‐rating data, then run an SIR (Susceptible‐Infected‐Recovered) style default‐cascade simulation to understand the spread of systemic risk.

##### File structure
- Top‐level notebooks:
- BDF – First Submission.ipynb, BDF – Final Notebook.ipynb, Data Exploration.ipynb
- “Big Data in Finance – Presentation.pdf” and other static deliverables
- Data directory (data/_bin and data/comtrade): raw CSV/XLSX exports of reserves, GDP series, investment, trade (COMTRADE), credit scores, foreign claims
- README.md describes the end-to-end workflow
- No application code beyond analysis notebooks and raw data

##### Key insights from the README
- Four major steps:
1) Build the network from bilateral claims
2) Model parameters (β from claims, γ from credit ratings)
3) Run per‐country default‐cascade simulations
4) And compare epidemic curves and derive policy insights

a. Main components
1. Data Sources
– BIS foreign claims CSVs (data/_bin)
– COMTRADE bilateral trade CSVs (data/comtrade)
– Credit rating/credit_score.csv
– Macroeconomic indicators (GDP, reserves)
2. Data Ingestion & Cleaning
– “Data Exploration.ipynb”: load, clean, standardize CSV/Excel
– Outputs: cleaned data tables
3. Network Builder
– Code cells in notebooks that take cleaned bilateral claims → weighted graph edges
4. Parameter Calculator
– Compute β (transmission rate) from normalized claims
– Compute γ (recovery rate) from credit scores
5. Simulation Engine
– SIR module in BDF notebooks: iteratively propagate defaults through network
– Iterates simulation per country origin
6. Results Aggregator & Visualizer
– Generate time‐series of Susceptible/Infected/Recovered counts
– Produce plots, summary tables
7. Reporting & Presentation
– Final notebook compiles figures and conclusions
– Exports to PDF (Big Data in Finance – Presentation.pdf)

b. Relationships & interactions
– Annotation: “Iterate simulation for each origin country” between Simulation Engine and Results Aggregator

c. Architectural patterns & design principles
– Modular pipeline: separate data ingestion, preprocessing, simulation, and reporting
– Notebook‐centric “orchestration” rather than script/function libraries
– Emphasis on reproducibility: raw data → cleaned data → simulation → plots

d. Key technologies & libraries
– Python, Jupyter notebooks
– pandas (CSV/Excel I/O, dataframes)
– networkx (graph construction)
– numpy (numerical computation)
– matplotlib / seaborn (plotting)
– comtrade API or local CSVs for trade data

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Preparation Layer
    subgraph "Data Preparation"
        direction TB
        RAW1(["BIS Foreign Claims"]):::datastore
        RAW2(["Credit Ratings"]):::datastore
        RAW3(["Macro Indicators"]):::datastore
        RAW4(["COMTRADE Bilateral Trade Data"]):::datastore
        INGEST(["Data Ingestion Notebook"]):::notebook
        CLEAN(["Cleaned Data Tables"]):::datastore
    end

    %% Modeling Layer
    subgraph "Modeling"
        direction TB
        NETBUILD(["Network Builder"]):::process
        PARAM(["Parameter Calculator"]):::process
        SIM(["Simulation Engine"]):::process
        AGG(["Results Aggregator & Visualizer"]):::process
    end

    %% Reporting Layer
    subgraph "Reporting"
        direction TB
        FINALNB(["Reporting Notebook"]):::notebook
        PRESENT(["Big Data in Finance – Presentation.pdf"]):::report
    end

    %% Orchestration / Workflow Guide
    README(["README.md"]):::report

    %% Data Flow
    README -->|describes workflow| RAW1
    README --> RAW2
    README --> RAW3
    README --> RAW4

    RAW1 --> INGEST
    RAW2 --> INGEST
    RAW3 --> INGEST
    RAW4 --> INGEST
    INGEST --> CLEAN

    CLEAN --> NETBUILD
    NETBUILD --> PARAM
    PARAM --> SIM
    SIM -->|for_each_country| AGG
    AGG --> FINALNB
    FINALNB --> PRESENT

    %% Click Events
    click RAW1 "https://github.com/davidecallegaro/big_data_in_finance_2020/blob/master/data/foreign_claims.csv"
    click RAW2 "https://github.com/davidecallegaro/big_data_in_finance_2020/blob/master/data/credit_score.csv"
    click RAW3 "https://github.com/davidecallegaro/big_data_in_finance_2020/tree/master/data/_bin/"
    click RAW4 "https://github.com/davidecallegaro/big_data_in_finance_2020/tree/master/data/comtrade/"
    click INGEST "https://github.com/davidecallegaro/big_data_in_finance_2020/blob/master/Data Exploration.ipynb"
    click NETBUILD "https://github.com/davidecallegaro/big_data_in_finance_2020/blob/master/BDF – First Submission.ipynb"
    click PARAM "https://github.com/davidecallegaro/big_data_in_finance_2020/blob/master/BDF – Final Notebook.ipynb"
    click SIM "https://github.com/davidecallegaro/big_data_in_finance_2020/blob/master/BDF – Final Notebook.ipynb"
    click AGG "https://github.com/davidecallegaro/big_data_in_finance_2020/blob/master/BDF – Final Notebook.ipynb"
    click FINALNB "https://github.com/davidecallegaro/big_data_in_finance_2020/blob/master/BDF – Final Notebook.ipynb"
    click PRESENT "https://github.com/davidecallegaro/big_data_in_finance_2020/blob/master/Big Data in Finance – Presentation.pdf"
    click README "https://github.com/davidecallegaro/big_data_in_finance_2020/blob/master/README.md"

    %% Styles
    classDef datastore fill:#f9f,stroke:#333,stroke-width:1px;
    classDef notebook fill:#bbf,stroke:#333,stroke-width:1px;
    classDef process fill:#bfb,stroke:#333,stroke-width:1px;
    classDef report fill:#fbf,stroke:#333,stroke-width:1px;
```
