# pablopaezsheridan/tfm architecture

> Architecture diagram of the GitHub repository pablopaezsheridan/tfm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pablopaezsheridan/tfm
- Repository: https://github.com/pablopaezsheridan/tfm

## Overview

This repository appears to implement a notebook-based market-data pipeline: it prepares static reference data and fetches prices and news, derives silver-layer datasets, then builds a gold ticker-prediction training dataset. The tree also shows shared symbol and Spark utilities. The README is empty and only the two utility files were excerpted; therefore, the layer transitions and external-source connections below are inferred from notebook names and are marked unverified.

Main entry points: [`pdt/notebooks/bronze/01a-bronze_prices_yfinance.py.ipynb`](https://github.com/pablopaezsheridan/tfm/blob/main/pdt/notebooks/bronze/01a-bronze_prices_yfinance.py.ipynb) (Price ingestion), [`pdt/notebooks/bronze/01b-bronze_gdelt_fetch_to_landing.py.ipynb`](https://github.com/pablopaezsheridan/tfm/blob/main/pdt/notebooks/bronze/01b-bronze_gdelt_fetch_to_landing.py.ipynb) (News ingestion), [`pdt/notebooks/silver/00-silver_prices_metrics.py.ipynb`](https://github.com/pablopaezsheridan/tfm/blob/main/pdt/notebooks/silver/00-silver_prices_metrics.py.ipynb) (Price metrics), [`pdt/notebooks/silver/01-silver_news_from_gdelt.py.ipynb`](https://github.com/pablopaezsheridan/tfm/blob/main/pdt/notebooks/silver/01-silver_news_from_gdelt.py.ipynb) (News processing)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ingestion["Bronze ingestion"]
  node_static_tables[("Static tables")]
  node_prices_fetch["Price ingestion"]
  node_news_fetch["News ingestion"]
  node_static_injection["Static data injection"]
end

subgraph group_transform["Silver and gold"]
  node_price_metrics["Price metrics"]
  node_news_processing["News processing"]
  node_training_dataset[("Training dataset")]
end

subgraph group_shared["Shared data tools"]
  node_spark_utils["Spark utilities<br/>[utils.py]"]
  node_symbol_mapping["Vendor symbols<br/>[vendor_mapping.py]"]
  node_universe_snapshot["Universe snapshot"]
end

node_yfinance["Yahoo Finance"]
node_gdelt["GDELT"]

node_yfinance -.->|"provides prices"| node_prices_fetch
node_gdelt -.->|"provides news"| node_news_fetch
node_prices_fetch -.->|"feeds metrics"| node_price_metrics
node_news_fetch -.->|"feeds processing"| node_news_processing
node_price_metrics -.->|"feeds dataset"| node_training_dataset
node_news_processing -.->|"feeds dataset"| node_training_dataset

click node_static_tables "https://github.com/pablopaezsheridan/tfm/blob/main/pdt/notebooks/bronze/00-create_bronze_static_tables.ipynb"
click node_prices_fetch "https://github.com/pablopaezsheridan/tfm/blob/main/pdt/notebooks/bronze/01a-bronze_prices_yfinance.py.ipynb"
click node_news_fetch "https://github.com/pablopaezsheridan/tfm/blob/main/pdt/notebooks/bronze/01b-bronze_gdelt_fetch_to_landing.py.ipynb"
click node_static_injection "https://github.com/pablopaezsheridan/tfm/blob/main/pdt/notebooks/bronze/one%20time%20static%20data%20injection.ipynb"
click node_price_metrics "https://github.com/pablopaezsheridan/tfm/blob/main/pdt/notebooks/silver/00-silver_prices_metrics.py.ipynb"
click node_news_processing "https://github.com/pablopaezsheridan/tfm/blob/main/pdt/notebooks/silver/01-silver_news_from_gdelt.py.ipynb"
click node_training_dataset "https://github.com/pablopaezsheridan/tfm/blob/main/pdt/notebooks/gold/00-gold_ticker_predictor_training_dataset.py.ipynb"
click node_spark_utils "https://github.com/pablopaezsheridan/tfm/blob/main/pdt/notebooks/common/utils.py"
click node_symbol_mapping "https://github.com/pablopaezsheridan/tfm/blob/main/pdt/notebooks/common/vendor_mapping.py"
click node_universe_snapshot "https://github.com/pablopaezsheridan/tfm/blob/main/pdt/notebooks/common/universe_sp100_snapshot.csv"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_static_tables,node_prices_fetch,node_news_fetch,node_static_injection toneBlue
class node_price_metrics,node_news_processing,node_training_dataset toneAmber
class node_spark_utils,node_symbol_mapping,node_universe_snapshot toneMint
class node_yfinance,node_gdelt toneIndigo
```

## Components

### Bronze ingestion

- [Static tables](https://github.com/pablopaezsheridan/tfm/blob/main/pdt/notebooks/bronze/00-create_bronze_static_tables.ipynb): `pdt/notebooks/bronze/00-create_bronze_static_tables.ipynb`
- [Price ingestion](https://github.com/pablopaezsheridan/tfm/blob/main/pdt/notebooks/bronze/01a-bronze_prices_yfinance.py.ipynb): `pdt/notebooks/bronze/01a-bronze_prices_yfinance.py.ipynb`
- [News ingestion](https://github.com/pablopaezsheridan/tfm/blob/main/pdt/notebooks/bronze/01b-bronze_gdelt_fetch_to_landing.py.ipynb): `pdt/notebooks/bronze/01b-bronze_gdelt_fetch_to_landing.py.ipynb`
- [Static data injection](https://github.com/pablopaezsheridan/tfm/blob/main/pdt/notebooks/bronze/one%20time%20static%20data%20injection.ipynb): `pdt/notebooks/bronze/one time static data injection.ipynb`

### Silver and gold

- [Price metrics](https://github.com/pablopaezsheridan/tfm/blob/main/pdt/notebooks/silver/00-silver_prices_metrics.py.ipynb): `pdt/notebooks/silver/00-silver_prices_metrics.py.ipynb`
- [News processing](https://github.com/pablopaezsheridan/tfm/blob/main/pdt/notebooks/silver/01-silver_news_from_gdelt.py.ipynb): `pdt/notebooks/silver/01-silver_news_from_gdelt.py.ipynb`
- [Training dataset](https://github.com/pablopaezsheridan/tfm/blob/main/pdt/notebooks/gold/00-gold_ticker_predictor_training_dataset.py.ipynb): `pdt/notebooks/gold/00-gold_ticker_predictor_training_dataset.py.ipynb`

### Shared data tools

- [Spark utilities](https://github.com/pablopaezsheridan/tfm/blob/main/pdt/notebooks/common/utils.py): `pdt/notebooks/common/utils.py`
- [Vendor symbols](https://github.com/pablopaezsheridan/tfm/blob/main/pdt/notebooks/common/vendor_mapping.py): `pdt/notebooks/common/vendor_mapping.py`
- [Universe snapshot](https://github.com/pablopaezsheridan/tfm/blob/main/pdt/notebooks/common/universe_sp100_snapshot.csv): `pdt/notebooks/common/universe_sp100_snapshot.csv`

### Other components

- **Yahoo Finance**
- **GDELT**

## Connections

- Yahoo Finance → Price ingestion: provides prices
- GDELT → News ingestion: provides news
- Price ingestion → Price metrics: feeds metrics
- News ingestion → News processing: feeds processing
- Price metrics → Training dataset: feeds dataset
- News processing → Training dataset: feeds dataset
