How muhammad-ahmad2511/finsight-pk works
FinSight PK is a research pipeline for regime-aware, sentiment-augmented KSE-100 forecasts. The workflow cleans index prices, derives technical indicators and regimes, gathers news and macro features, labels news sentiment, assembles model inputs, and trains a multi-horizon LSTM. Documented product branches include per-regime SHAP explainability and a web app with dashboard, forecasts, sentiment feed, and REST API. Edges reflect explicit script inputs/outputs or README statements; the web and SHAP implementations are not present in the supplied tree, so their nodes point to the README.
Main entry points: README.md Web Application · 08_build_lstm_dataset.py Forecast Dataset Builder · 07_label_using_distilbert.py News Labeling · 01_kse100_clean.py Price Cleaner
Components 20
Market Features
- Raw KSE Prices
Data/Datasets/Karachi 100 Historical Data.csv - Price Cleaner
Data/Data Scripts/01_kse100_clean.py - Technical Indicators
Data/Data Scripts/02_technical_indicators.py - Regime Detection
Data/Data Scripts/03_regime_detection.py - EPU Features
Data/Data Scripts/04_epu_clean.py - External Features
Data/Data Scripts/06_fetch_external.py
News Sentiment
- Dawn Scraper
Data/Data Scripts/05_wayback_scraper.py - Dawn News Corpus
Data/Datasets/dawn_business_2013_to_2026.csv - Sentiment Fine-tuning
model/Distilbert/Script/finetune_distilbert_kse100.py - News Labeling
Data/Data Scripts/07_label_using_distilbert.py - Labeled News
Data/Datasets/dawn_business_2013_to_2026_labeled.csv
Forecasting
- Forecast Dataset Builder
Data/Data Scripts/08_build_lstm_dataset.py - LSTM Training & Forecast
Data/Data Scripts/09_train_lstm.py
Explainability & App
- Per-Regime SHAP
README.md - Web Application
README.md - Live Dashboard
README.md - Forecast Views
README.md - Sentiment Feed
README.md - REST API
README.md
Other components
- Market Participant
Connections 19
- Raw KSE Prices → Price Cleaner: cleans
Data/Data Scripts/01_kse100_clean.py - Price Cleaner → Technical Indicators: provides prices
Data/Data Scripts/02_technical_indicators.py - Technical Indicators → Regime Detection: provides indicators
Data/Data Scripts/03_regime_detection.py - Regime Detection → Forecast Dataset Builder: provides regimes
Data/Data Scripts/08_build_lstm_dataset.py - Dawn Scraper → Dawn News Corpus: writes corpus
README.md - Dawn News Corpus → News Labeling: labels headlines
Data/Data Scripts/07_label_using_distilbert.py - Sentiment Fine-tuning → News Labeling: supplies model
Data/Data Scripts/07_label_using_distilbert.py - News Labeling → Labeled News: writes labels
Data/Data Scripts/07_label_using_distilbert.py - Labeled News → Forecast Dataset Builder: adds sentiment
Data/Data Scripts/08_build_lstm_dataset.py - EPU Features → Forecast Dataset Builder: provides EPU
Data/Data Scripts/08_build_lstm_dataset.py - External Features → Forecast Dataset Builder: provides features
Data/Data Scripts/08_build_lstm_dataset.py - Forecast Dataset Builder → LSTM Training & Forecast: provides inputs
Data/Data Scripts/09_train_lstm.py - LSTM Training & Forecast → Per-Regime SHAP: explained by
README.md - Market Participant → Web Application: uses
README.md - Web Application → Live Dashboard: serves
README.md - Web Application → Forecast Views: serves
README.md - Web Application → Sentiment Feed: serves
README.md - Web Application → REST API: exposes
README.md - Per-Regime SHAP → Forecast Views: explains forecasts
README.md