How prasunray477/nfm-hyb works
1. System shape
- This repo is a two-tier stock forecasting application: a FastAPI backend that performs data fetching and model inference/training, and a Streamlit frontend that presents predictions, comparisons, and analysis to users.
- The core boundary is between the presentation layer (frontend/) and the forecasting engine (app/ + ml/).
2. Backend entry point and API surface
- app/main.py is the FastAPI application entrypoint.
- API routes are split by responsibility in app/api/routes/:
- health.py for service checks
- predict.py for synchronous forecasting
- train.py for background training runs
- Request/response contracts are centralized in app/models/schemas.py, which keeps the API boundary explicit.
3. Backend orchestration layer
- The backend is not just a thin API wrapper; it likely orchestrates the full forecasting workflow.
- ml/pipeline.py appears to be the main coordinator for the end-to-end “analysis engine” described in the README: data preparation, stationarity handling, neutrosophic transform, model fitting, prediction, weighting, and evaluation.
- The /api/predict and /api/train routes are the main entry points into that pipeline.
4. Data acquisition and service boundary
- app/services/data_service.py is the repository’s data ingestion layer.
- The README indicates live market data comes from Yahoo Finance via yfinance, so this service is the external-data boundary for OHLCV retrieval.
- This layer isolates the rest of the system from raw market API calls and likely normalizes downloaded data before it enters the ML pipeline.
5. Preprocessing and signal transformation
- ml/neutrosophic/ implements the domain-specific preprocessing:
- normalizer.py for transforming raw series into neutrosophic components
- entropy.py for indeterminacy/uncertainty calculation
- app/utils/stationarity.py handles stationarity checks and differencing logic, which is especially important for the ARIMA branch.
- This is the main feature-engineering boundary: raw time series becomes model-ready inputs.
6. Modeling subsystem: two parallel predictors
- The forecasting engine combines two model families:
- ml/arima/model.py for the classical time-series branch
- ml/bilstm/architecture.py, trainer.py, and predictor.py for the TensorFlow Bi-LSTM branch
- These are deliberately separate because they solve different subproblems:
- ARIMA captures linear autocorrelation
- Bi-LSTM captures nonlinear temporal patterns in neutrosophic sequences
7. Ensemble and weighting logic
- ml/aggregator/static_weights.py and adaptive_weights.py implement the hybrid fusion strategy.
- The repository supports both fixed weighting and validation-driven optimization, matching the README’s “weighted ensemble” design.
- This subsystem is the decision point where the two model outputs are combined into a final forecast.
8. Evaluation and comparative analysis
- ml/evaluation/metrics.py centralizes forecast quality metrics such as RMSE, MAE, MAPE, and Theil’s U.
- The frontend’s “analysis” page aligns with the backend’s ability to compare multiple configurations, from baseline ARIMA to the full hybrid model.
- This makes evaluation a first-class output, not just an internal training artifact.
9. Frontend architecture
- frontend/streamlit_app.py is the Streamlit entry point.
- frontend/pages/ defines the user workflow:
- 01_predict.py for running forecasts
- 02_analysis.py for model comparison and diagnostics
- 03_about.py for explanation/documentation
- frontend/components/ contains reusable UI pieces for charts, metrics, and neutrosophic visualizations, which keeps the page logic thin.
10. Persistence and local artifacts
- The data/ directory is the local artifact store:
- data/models/arima/ and data/models/bilstm/ store saved model outputs/checkpoints
- data/results/ stores forecast outputs and analysis results
- This indicates the system is partly stateful: training and prediction can produce persisted artifacts that later runs may reuse.
11. Configuration, logging, and runtime concerns
- app/core/config.py centralizes environment-driven settings, with .env.example showing the app expects runtime configuration outside source control.
- app/core/logger.py provides structured logging, which is important for observability across the API and ML workflow.
- The repo targets Python 3.11+, with FastAPI, Streamlit, and TensorFlow 2.15 as the key runtime technologies.
12. Deployment and delivery
- Dockerfile, render.yaml, and .github/workflows/deploy.yml show the project is designed for containerized deployment with CI/CD automation.
- The architecture therefore includes deployment concerns directly in-repo: reproducible packaging, environment configuration, and cloud deployment hooks.
13. Testing boundary
- tests/ covers the major subsystems separately:
- aggregation
- ARIMA
- Bi-LSTM
- neutrosophic logic
- the pipeline
- This reflects the system’s modular design: each major architectural slice can be validated independently.
14. High-level data flow for an inference request
- A user submits a ticker and forecast options in the frontend or via /api/predict.
- The backend fetches market data, applies stationarity and neutrosophic preprocessing, runs ARIMA and Bi-LSTM prediction paths, combines them via the aggregator, evaluates outputs, and returns the final result.
- The frontend then renders charts, metrics, and comparative analysis from that response.
Main entry points: pipeline.py Pipeline · predict.py Predict route · main.py API app · train.py Train route
Components 23
Backend
FastAPI API, services, and runtime configuration.
- API app · Creates and hosts the FastAPI application.
app/main.py - Predict route · Synchronous forecasting endpoint.
app/api/routes/predict.py - Train route · Background training endpoint.
app/api/routes/train.py - Health route · Service status endpoint.
app/api/routes/health.py - Schemas · Request and response models for the API boundary.
app/models/schemas.py - Data service · Fetches and normalizes live market data.
app/services/data_service.py - Stationarity · Checks and transforms series for modeling.
app/utils/stationarity.py - Config & logs · Environment settings and structured logging.
app/core/config.py
Forecast Engine
Data prep, models, ensemble logic, and evaluation.
- Pipeline · Coordinates preparation, training, prediction, weighting, and evaluation.
ml/pipeline.py - Normalizer · Transforms series into neutrosophic components.
ml/neutrosophic/normalizer.py - Entropy · Computes indeterminacy and uncertainty measures.
ml/neutrosophic/entropy.py - ARIMA model · Linear time-series forecasting branch.
ml/arima/model.py - Bi-LSTM · Neural forecasting architecture for temporal patterns.
ml/bilstm/architecture.py - Trainer · Fits and updates the Bi-LSTM model.
ml/bilstm/trainer.py - Predictor · Runs Bi-LSTM inference on prepared inputs.
ml/bilstm/predictor.py - Ensemble weights · Combines ARIMA and Bi-LSTM outputs using fixed or adaptive weighting.
ml/aggregator/adaptive_weights.py - Metrics · Forecast quality measures and comparisons.
ml/evaluation/metrics.py
Frontend
Streamlit user interface and reusable components.
- Streamlit app · Main Streamlit application shell.
frontend/streamlit_app.py - Predict page · User workflow for requesting forecasts.
frontend/pages/01_predict.py - Analysis page · Model comparison and diagnostics.
frontend/pages/02_analysis.py - UI components · Reusable charts, metrics, and neutrosophic visualizations.
frontend/components/charts.py
Ops
Deployment and local stateful artifacts.
- Artifacts · Saved models and stored forecast results.
data/results/.gitkeep - Deploy config · Container and CI/CD deployment configuration.
Dockerfile
Connections 31
- API app → Health route: mounts · Registers the health route.
- API app → Predict route: mounts · Registers prediction endpoints.
- API app → Train route: mounts · Registers training endpoints.
- Predict route → Schemas: uses · Validates request and response payloads.
- Train route → Schemas: uses · Shares API contracts.
- Predict route → Pipeline: invokes · Starts the forecasting workflow.
- Train route → Pipeline: invokes · Starts training and optimization runs.
- Pipeline → Data service: loads · Fetches market data for analysis.
- Pipeline → Stationarity: preprocesses · Prepares time series for modeling.
- Pipeline → Normalizer: transforms · Builds neutrosophic inputs.
- Pipeline → Entropy: derives · Computes uncertainty features.
- Pipeline → ARIMA model: runs · Executes the classical forecast branch.
- Pipeline → Trainer: trains · Fits the neural branch when needed.
- Pipeline → Predictor: infers · Runs neural inference for forecasts.
- Pipeline → Ensemble weights: fuses · Combines branch outputs into the final prediction.
- Pipeline → Metrics: evaluates · Scores forecasts and comparisons.
- Trainer → Bi-LSTM: builds · Uses the neural architecture for fitting.
- Predictor → Bi-LSTM: loads · Uses the trained architecture for inference.
- Ensemble weights → Metrics: compares · Supports model comparison and selection.
- Streamlit app → Predict page: routes · Exposes the forecast workflow.
- Streamlit app → Analysis page: routes · Exposes analysis and diagnostics.
- Predict page → UI components: renders · Uses shared charts and metric displays.
- Analysis page → UI components: renders · Uses shared visualization components.
- Predict page → Predict route: calls · Requests forecasts from the backend.
- Analysis page → Metrics: shows · Presents evaluation results and comparisons.
- Predict route → Artifacts: reads/writes · May reuse saved models and persist outputs.
- Train route → Artifacts: writes · Stores trained models and run outputs.
- Pipeline → Artifacts: persists · Emits reusable results and checkpoints.
- Config & logs → API app: supports · Provides runtime settings and logging.
- Deploy config → API app: packages · Defines how the backend is built and run.
- Deploy config → Streamlit app: packages · Defines how the UI is built and run.