How firaold/news-sentiment-stock-prediction works
Project Type and Purpose
• This is a data-science analytics pipeline (not a web app). It ingests financial news and stock-price data, performs sentiment and technical-indicator analysis, and uncovers correlations between sentiment and returns.
Main Components
a. Data Sources
• External News API (e.g., RSS feeds, scraped headlines)
• Stock Price API (e.g., yfinance)
b. Data Ingestion Layer
• scripts/download_stock_data.py
• (Implicit) news-download module inside NewsAnalyzer or separate script
c. Data Storage
• Local “data/” folder (gitignored) containing raw CSV/JSON
d. Processing & Analysis Modules (src/)
• BaseAnalyzer (common I/O, error handling)
• NewsAnalyzer (sentiment, EDA, topic modeling)
• StockAnalyzer (RSI, MACD, moving averages via TA-Lib/PyNance)
• CorrelationAnalyzer (combines sentiment & returns)
e. Notebooks & Visualization
• Jupyter notebooks under notebooks/ for EDA, technical indicators, correlation analysis
• “Plot Visualizations” folder with saved charts
f. Testing & CI
• tests/test_news_analyzer.py (unit tests for NewsAnalyzer)
• GitHub Actions workflows (.github/workflows/ci.yml, unittests.yml)
g. External Libraries & Frameworks
• TextBlob / NLTK for NLP
• TA-Lib / PyNance for technical indicators
• Pandas, NumPy for data manipulation
• Matplotlib, Seaborn for plotting
Relationships & Data Flow
1. Data Sources → Data Ingestion Layer (download scripts)
2. Ingestion writes Raw Data → Data Storage
3. Processing Modules read Raw Data:
• NewsAnalyzer reads headlines → performs sentiment & topic modeling → outputs cleaned sentiment data
• StockAnalyzer reads price data → computes indicators → outputs indicator data
4. CorrelationAnalyzer reads sentiment + indicators → computes correlation results
5. Notebooks read processed outputs → generate interactive EDA, visualizations → save figures to “Plot Visualizations”
6. Tests validate NewsAnalyzer; CI pipelines run on GitHub on each push/pull
Architectural Patterns & Principles
• Modular OOP: BaseAnalyzer parent, specialized child analyzers (follows DRY)
• Data-Pipeline pattern: clear extract → transform → load & analyze stages
• Layered separation: ingestion, processing, visualization, testing
Must-Include Elements
• Component names & roles
• Data sources vs. storage vs. compute vs. visualization vs. CI
• Callouts for key technologies (TextBlob, TA-Lib, Pandas, Matplotlib)