# sandeepsandy1912/financial-market-prediction-using-machine-learning architecture

> Architecture diagram of the GitHub repository sandeepsandy1912/financial-market-prediction-using-machine-learning, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-23). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sandeepsandy1912/financial-market-prediction-using-machine-learning
- Repository: https://github.com/sandeepsandy1912/financial-market-prediction-using-machine-learning

## Overview

##### Project Type & Purpose
- This is primarily a **data-science pipeline** with:
- Data ingestion (from Yahoo Finance API via `yfinance`)
- Data preprocessing & feature engineering
- Model training & hyperparameter tuning (ML + Deep Learning)
- Model evaluation & metrics
- A **Streamlit** dashboard for visualization

##### File Structure
- Top-level files:
- `Model Building.py`, `Simple EDA for understanding.ipynb`, `Steamlit_for_Visualising_output.py` – exploratory & pipeline scripts
- `best_models_summary.pkl`, `consolidated_evaluation_results.pkl` – serialized outputs
- README suggests an intended structure under `data/`, `models/`, `notebooks/`, `src/`, `results/` (even if not physically present in your file tree, adopt it conceptually).

##### Main Components & Their Roles
a. Data Ingestion
– Script pulls raw time-series via `yfinance`.
b. Data Preprocessing
– Cleans data, imputes missing, normalizes, feature engineers (`data_preprocessing.py`).
c. Model Training
– Implements Random Forest, GBM, SVR, Ridge, LSTM; uses GridSearchCV, Keras Tuner (`model_training.py`).
d. Model Evaluation
– Calculates MSE, RMSE, R², Directional Accuracy (`evaluation.py`); writes to `consolidated_evaluation_results.pkl`.
e. Persistence / Storage
– Saves pickled models (`models/`), summary (`best_models_summary.pkl`).
f. Visualization / UI
– Streamlit dashboard (`streamlit_dashboard.py` or `Steamlit_for_Visualising_output.py`) reads results & models, exposes an interactive UI.

##### Interactions & Data Flow
- Raw → Preprocess → Training → Evaluation → Storage → Dashboard → End-User

##### Architectural Patterns & Principles
- **Modular scripts**: each stage in its own file under `src/`
- **Separation of concerns**: EDA vs. preprocessing vs. training vs. evaluation vs. UI
- **Artifact versioning**: pickled models & results kept separately
- **Interactive front end**: Streamlit decoupled from batch pipeline

##### Technologies & Libraries
- Data: pandas, yfinance
- ML: scikit-learn (RF, GB, SVR, Ridge), keras/TensorFlow (LSTM)
- Hyperparameter Tuning: GridSearchCV, RandomizedSearchCV, Keras Tuner
- UI: Streamlit

## Diagram (Mermaid)

```mermaid
flowchart TD
    ExternalAPI["External API<br>(Yahoo Finance)"]:::external
    DataIngestion["Data Ingestion"]:::process
    DataPreprocessing["Data Preprocessing"]:::process
    ModelTraining["Model Training"]:::process
    ModelEvaluation["Model Evaluation"]:::process
    ModelsStore["Models Store<br>(pickles)"]:::store
    ResultsStore["Results Store<br>(pickle summaries)"]:::store
    Dashboard["Streamlit Dashboard"]:::ui
    EDA["Exploratory Data Analysis"]:::process

    ExternalAPI --> DataIngestion
    DataIngestion --> DataPreprocessing
    DataPreprocessing --> ModelTraining
    ModelTraining --> ModelEvaluation
    ModelTraining -->|saves models| ModelsStore
    ModelEvaluation -->|writes results| ResultsStore
    ResultsStore --> Dashboard
    EDA -.-> DataPreprocessing

    click DataIngestion "https://github.com/sandeepsandy1912/financial-market-prediction-using-machine-learning/blob/main/Model Building.py"
    click DataPreprocessing "https://github.com/sandeepsandy1912/financial-market-prediction-using-machine-learning/blob/main/Model Building.py"
    click ModelTraining "https://github.com/sandeepsandy1912/financial-market-prediction-using-machine-learning/blob/main/Model Building.py"
    click ModelEvaluation "https://github.com/sandeepsandy1912/financial-market-prediction-using-machine-learning/blob/main/Model Building.py"
    click ModelsStore "https://github.com/sandeepsandy1912/financial-market-prediction-using-machine-learning/blob/main/best_models_summary.pkl"
    click ResultsStore "https://github.com/sandeepsandy1912/financial-market-prediction-using-machine-learning/blob/main/consolidated_evaluation_results.pkl"
    click EDA "https://github.com/sandeepsandy1912/financial-market-prediction-using-machine-learning/blob/main/Simple EDA for understanding.ipynb"
    click Dashboard "https://github.com/sandeepsandy1912/financial-market-prediction-using-machine-learning/blob/main/Steamlit_for_Visualising_output.py"

    classDef external fill:#FFA500,stroke:#333,stroke-width:1px;
    classDef process fill:#ADD8E6,stroke:#333,stroke-width:1px;
    classDef store fill:#90EE90,stroke:#333,stroke-width:1px;
    classDef ui fill:#DDA0DD,stroke:#333,stroke-width:1px;
```
