How rushi873/anomaly-detection-and-clustering-for-stock-indices works
Main Components
• Data Collection & Preprocessing: Represent the “Data collection” directory where raw time series (e.g., IBOV.csv and NIFTY.csv) is ingested and cleaned.
• Exploratory Analysis: A distinct module that includes the “Analysis” folder. This component covers initial data manipulation, exploration, and the computation of essential attributes needed for later stages.
• Modeling Modules:
• Deep Learning Model: Illustrate the notebooks (e.g., LSTM_encoder, Data_manipulation in that folder) that implement advanced anomaly detection using deep learning techniques.
• Statistical Model: Represent notebooks such as ARMA, EWMA, and VAR, which handle forecasting, cointegration, and other econometric tests.
• Unsupervised Model: Include the unsupervised techniques (Isolation Forest, KNN, Local Outlier Factor, SVM) for anomaly detection.
• Comparative & Strategy Analysis: Indicate the part of the system that performs comparative studies, handles the trading strategy development (peak/trough analysis), and computes performance metrics (e.g., Sharpe Ratio).
• (Optionally) Data Storage/Logging: Though not explicitly a separate service, you may also want to show how results – like missing_data_log.csv and other outputs – are stored and perhaps fed back into the analysis pipeline.
Relationships & Data Flow
• From Preprocessing/Exploratory Analysis, branch out to the three main modeling modules:
• Direct the cleaned/processed data into Statistical Model and Deep Learning Model pipelines.
• Also feed the same data into the Unsupervised Model for outlier detection.
• That the outputs from all these modeling streams are funneled into a Comparative Analysis component which integrates and analyzes anomalies alongside trading signal extraction.
Architectural Patterns & Design Principles
• Separation of Concerns: Clearly delineate data ingestion, preprocessing, modeling, and strategy analysis as separate components.
• Modular Design: Emphasize that each folder (Deep Learning, Statistic, Unsupervised) represents a self-contained module that can be adjusted or extended independently.
• Notebook-Driven Prototyping: Note that the system is prototype-focused using Jupyter notebooks, so these modules can be visualized as iterative and exploratory rather than static microservices.
Incorporate Relevant Technologies & Frameworks
• Jupyter Notebooks: Indicate that each module (found in the various directories) leverages notebooks for rapid prototyping.