How pablopaezsheridan/tfm works
This repository appears to implement a notebook-based market-data pipeline: it prepares static reference data and fetches prices and news, derives silver-layer datasets, then builds a gold ticker-prediction training dataset. The tree also shows shared symbol and Spark utilities. The README is empty and only the two utility files were excerpted; therefore, the layer transitions and external-source connections below are inferred from notebook names and are marked unverified.
Main entry points: 01a-bronze_prices_yfinance.py.ipynb Price ingestion · 01b-bronze_gdelt_fetch_to_landing.py.ipynb News ingestion · 00-silver_prices_metrics.py.ipynb Price metrics · 01-silver_news_from_gdelt.py.ipynb News processing
Components 12
Bronze ingestion
- Static tables
pdt/notebooks/bronze/00-create_bronze_static_tables.ipynb - Price ingestion
pdt/notebooks/bronze/01a-bronze_prices_yfinance.py.ipynb - News ingestion
pdt/notebooks/bronze/01b-bronze_gdelt_fetch_to_landing.py.ipynb - Static data injection
pdt/notebooks/bronze/one time static data injection.ipynb
Silver and gold
- Price metrics
pdt/notebooks/silver/00-silver_prices_metrics.py.ipynb - News processing
pdt/notebooks/silver/01-silver_news_from_gdelt.py.ipynb - Training dataset
pdt/notebooks/gold/00-gold_ticker_predictor_training_dataset.py.ipynb
Shared data tools
- Spark utilities
pdt/notebooks/common/utils.py - Vendor symbols
pdt/notebooks/common/vendor_mapping.py - Universe snapshot
pdt/notebooks/common/universe_sp100_snapshot.csv
Other components
- Yahoo Finance
- GDELT
Connections 6
- Yahoo Finance → Price ingestion: provides prices
- GDELT → News ingestion: provides news
- Price ingestion → Price metrics: feeds metrics
- News ingestion → News processing: feeds processing
- Price metrics → Training dataset: feeds dataset
- News processing → Training dataset: feeds dataset