How sandeepsandy1912/financial-market-prediction-using-machine-learning works
Project Type & Purpose
• This is primarily a data-science pipeline with:
• Data ingestion (from Yahoo Finance API via yfinance)
• Data preprocessing & feature engineering
• Model training & hyperparameter tuning (ML + Deep Learning)
• Model evaluation & metrics
• A Streamlit dashboard for visualization
File Structure
• Top-level files:
• Model Building.py, Simple EDA for understanding.ipynb, Steamlit_for_Visualising_output.py – exploratory & pipeline scripts
• best_models_summary.pkl, consolidated_evaluation_results.pkl – serialized outputs
• README suggests an intended structure under data/, models/, notebooks/, src/, results/ (even if not physically present in your file tree, adopt it conceptually).
Main Components & Their Roles
a. Data Ingestion
– Script pulls raw time-series via yfinance.
b. Data Preprocessing
– Cleans data, imputes missing, normalizes, feature engineers (data_preprocessing.py).
c. Model Training
– Implements Random Forest, GBM, SVR, Ridge, LSTM; uses GridSearchCV, Keras Tuner (model_training.py).
d. Model Evaluation
– Calculates MSE, RMSE, R², Directional Accuracy (evaluation.py); writes to consolidated_evaluation_results.pkl.
e. Persistence / Storage
– Saves pickled models (models/), summary (best_models_summary.pkl).
f. Visualization / UI
– Streamlit dashboard (streamlit_dashboard.py or Steamlit_for_Visualising_output.py) reads results & models, exposes an interactive UI.
Interactions & Data Flow
• Raw → Preprocess → Training → Evaluation → Storage → Dashboard → End-User
Architectural Patterns & Principles
• Modular scripts: each stage in its own file under src/
• Separation of concerns: EDA vs. preprocessing vs. training vs. evaluation vs. UI
• Artifact versioning: pickled models & results kept separately
• Interactive front end: Streamlit decoupled from batch pipeline
Technologies & Libraries
• Data: pandas, yfinance
• ML: scikit-learn (RF, GB, SVR, Ridge), keras/TensorFlow (LSTM)
• Hyperparameter Tuning: GridSearchCV, RandomizedSearchCV, Keras Tuner
• UI: Streamlit