# rushi873/anomaly-detection-and-clustering-for-stock-indices architecture

> Architecture diagram of the GitHub repository rushi873/anomaly-detection-and-clustering-for-stock-indices, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rushi873/anomaly-detection-and-clustering-for-stock-indices
- Repository: https://github.com/rushi873/anomaly-detection-and-clustering-for-stock-indices

## Overview

##### Main Components
- Data Collection & Preprocessing: Represent the “Data collection” directory where raw time series (e.g., IBOV.csv and NIFTY.csv) is ingested and cleaned.
- Exploratory Analysis: A distinct module that includes the “Analysis” folder. This component covers initial data manipulation, exploration, and the computation of essential attributes needed for later stages.
- Modeling Modules:
- Deep Learning Model: Illustrate the notebooks (e.g., LSTM_encoder, Data_manipulation in that folder) that implement advanced anomaly detection using deep learning techniques.
- Statistical Model: Represent notebooks such as ARMA, EWMA, and VAR, which handle forecasting, cointegration, and other econometric tests.
- Unsupervised Model: Include the unsupervised techniques (Isolation Forest, KNN, Local Outlier Factor, SVM) for anomaly detection.
- Comparative & Strategy Analysis: Indicate the part of the system that performs comparative studies, handles the trading strategy development (peak/trough analysis), and computes performance metrics (e.g., Sharpe Ratio).
- (Optionally) Data Storage/Logging: Though not explicitly a separate service, you may also want to show how results – like missing_data_log.csv and other outputs – are stored and perhaps fed back into the analysis pipeline.

##### Relationships & Data Flow
- From Preprocessing/Exploratory Analysis, branch out to the three main modeling modules:
- Direct the cleaned/processed data into Statistical Model and Deep Learning Model pipelines.
- Also feed the same data into the Unsupervised Model for outlier detection.
- That the outputs from all these modeling streams are funneled into a Comparative Analysis component which integrates and analyzes anomalies alongside trading signal extraction.

##### Architectural Patterns & Design Principles
- Separation of Concerns: Clearly delineate data ingestion, preprocessing, modeling, and strategy analysis as separate components.
- Modular Design: Emphasize that each folder (Deep Learning, Statistic, Unsupervised) represents a self-contained module that can be adjusted or extended independently.
- Notebook-Driven Prototyping: Note that the system is prototype-focused using Jupyter notebooks, so these modules can be visualized as iterative and exploratory rather than static microservices.

##### Incorporate Relevant Technologies & Frameworks
- Jupyter Notebooks: Indicate that each module (found in the various directories) leverages notebooks for rapid prototyping.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Data Ingestion & Preprocessing"
        DC["Data Collection & Preprocessing"]:::data
    end

    subgraph "Exploratory Analysis"
        EA["Exploratory Analysis"]:::data
    end

    subgraph "Modeling Modules"
        DL["Deep Learning Model"]:::model
        STAT["Statistical Model"]:::model
        UNS["Unsupervised Model"]:::model
    end

    subgraph "Comparative & Trading Strategy Analysis"
        COMP["Comparative & Trading Strategy Analysis"]:::strategy
    end

    DC -->|"transforms"| EA
    EA -->|"feeds"| DL
    EA -->|"feeds"| STAT
    EA -->|"feeds"| UNS
    DL -->|"integrates"| COMP
    STAT -->|"integrates"| COMP
    UNS -->|"integrates"| COMP

    %% Click Events for Data Collection & Preprocessing
    click DC "https://github.com/rushi873/anomaly-detection-and-clustering-for-stock-indices/blob/main/Data collection/IBOV.csv"
    click DC "https://github.com/rushi873/anomaly-detection-and-clustering-for-stock-indices/blob/main/Data collection/NIFTY.csv"
    click DC "https://github.com/rushi873/anomaly-detection-and-clustering-for-stock-indices/blob/main/Data collection/download.ipynb"

    %% Click Events for Exploratory Analysis
    click EA "https://github.com/rushi873/anomaly-detection-and-clustering-for-stock-indices/blob/main/Analysis/Comparsion_stationary.ipynb"
    click EA "https://github.com/rushi873/anomaly-detection-and-clustering-for-stock-indices/blob/main/Analysis/Data_manipulation.ipynb"
    click EA "https://github.com/rushi873/anomaly-detection-and-clustering-for-stock-indices/blob/main/Analysis/IBOVESPA.ipynb"
    click EA "https://github.com/rushi873/anomaly-detection-and-clustering-for-stock-indices/blob/main/Analysis/NIFTY.ipynb"

    %% Click Events for Deep Learning Model
    click DL "https://github.com/rushi873/anomaly-detection-and-clustering-for-stock-indices/blob/main/Deep learning model/LSTM_encoder.ipynb"
    click DL "https://github.com/rushi873/anomaly-detection-and-clustering-for-stock-indices/blob/main/Deep learning model/Data_manipulation.ipynb"

    %% Click Events for Statistical Model
    click STAT "https://github.com/rushi873/anomaly-detection-and-clustering-for-stock-indices/blob/main/Statistic model/ARMA.ipynb"
    click STAT "https://github.com/rushi873/anomaly-detection-and-clustering-for-stock-indices/blob/main/Statistic model/EWMA.ipynb"
    click STAT "https://github.com/rushi873/anomaly-detection-and-clustering-for-stock-indices/blob/main/Statistic model/Holt_winter.ipynb"
    click STAT "https://github.com/rushi873/anomaly-detection-and-clustering-for-stock-indices/blob/main/Statistic model/PCA.ipynb"
    click STAT "https://github.com/rushi873/anomaly-detection-and-clustering-for-stock-indices/blob/main/Statistic model/MCD.ipynb"
    click STAT "https://github.com/rushi873/anomaly-detection-and-clustering-for-stock-indices/blob/main/Statistic model/Checking.ipynb"

    %% Click Events for Unsupervised Model
    click UNS "https://github.com/rushi873/anomaly-detection-and-clustering-for-stock-indices/blob/main/Unsupervised model/Isolation_Forest.ipynb"
    click UNS "https://github.com/rushi873/anomaly-detection-and-clustering-for-stock-indices/blob/main/Unsupervised model/KNN.ipynb"
    click UNS "https://github.com/rushi873/anomaly-detection-and-clustering-for-stock-indices/blob/main/Unsupervised model/Local_outlier_Factor.ipynb"
    click UNS "https://github.com/rushi873/anomaly-detection-and-clustering-for-stock-indices/blob/main/Unsupervised model/SVM.ipynb"
    click UNS "https://github.com/rushi873/anomaly-detection-and-clustering-for-stock-indices/blob/main/Unsupervised model/Data_manipulation.ipynb"

    %% Click Events for Comparative & Trading Strategy Analysis
    click COMP "https://github.com/rushi873/anomaly-detection-and-clustering-for-stock-indices/blob/main/Analysis/Performance_comparsion.ipynb"
    click COMP "https://github.com/rushi873/anomaly-detection-and-clustering-for-stock-indices/blob/main/Analysis/comparsion_non-stationary.ipynb"
    click COMP "https://github.com/rushi873/anomaly-detection-and-clustering-for-stock-indices/blob/main/Analysis/Google_trends.ipynb"
    click COMP "https://github.com/rushi873/anomaly-detection-and-clustering-for-stock-indices/blob/main/Analysis/missing_data_log.csv"

    %% Styles
    classDef data fill:#BBDEFB,stroke:#0D47A1,stroke-width:2px;
    classDef model fill:#C8E6C9,stroke:#1B5E20,stroke-width:2px;
    classDef strategy fill:#FFE0B2,stroke:#E65100,stroke-width:2px;
```
