# shrimaliharshal/tradewise architecture

> Architecture diagram of the GitHub repository shrimaliharshal/tradewise, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/shrimaliharshal/tradewise
- Repository: https://github.com/shrimaliharshal/tradewise

## Overview

##### Project Type and Purpose
- This is a hybrid system that integrates real-time stock data ingestion, prediction, visualization, and an AI-powered news/article fetching component.
- The system features two primary workflows: one for live stock data streaming and prediction (using Kafka, external APIs, and Streamlit dashboards) and another for StockBot—which processes news article URLs using LangChain, converts content with OpenAI embeddings, and indexes vectors with FAISS, enabling interactive chat-based information retrieval.
- The project employs Docker and Docker Compose for container orchestration, which ensures scalability and ease of deployment.

##### Analyzing the File Structure
- You have several Python scripts handling different system responsibilities:
- Data Fetching & Streaming: ‘streaming_kafka.py’, ‘alpaca_kafka.py’, ‘alphavantage.py’, and possibly ‘firebaseSetup.py’ (for additional services).
- Data Prediction and Processing: ‘prediction.py’ for forecasting stock prices and ‘indicators_test.py’ along with ‘traditionalGraphs.py’ for visualization.
- Interactive Dashboards: ‘dashboard.py’ uses Streamlit to deliver real-time visual data.
- StockBot Module: Located in the “StockBot” directory with ‘main.py’, a requirements file, .env, and a vector index pickle file. This encapsulates capabilities for content processing and vector embedding using LangChain, FAISS, and OpenAI.
- Additional modules like ‘alerts.py’, ‘app.py’, and others are supporting components that likely contribute to routing, error handling, or additional service integrations.
- The Docker Compose file indicates that components are containerized, which reinforces that the deployment setup is modular and each service/component is isolated yet interconnected.

- Break the system into clear, distinct modules. Label and outline the following high-level components:

A. External Data Sources and APIs:
- External stock market APIs (e.g., Alpaca, AlphaVantage) as data providers.

B. Data Ingestion and Streaming Layer:
- Module “streaming_kafka.py” as the real-time data fetcher.
- “alpaca_kafka.py” and “alphavantage.py” as additional ingestion components.

C. Data Processing and Prediction Module:
- “prediction.py” as the component that subscribes to the Kafka topic and applies prediction models.

D. Interactive Dashboard Layer:
- “dashboard.py” (and potentially “app.py”) as the user interface layer powered by Streamlit.
- That this dashboard displays both real-time insights and prediction results.

E. StockBot Component:
. “main.py” for orchestration of content processing.
. The use of LangChain’s UnstructuredURL Loader for fetching article content.
. Conversion of content to vectors via OpenAI embeddings and subsequent indexing using FAISS (where “vector_index.pkl” is stored).
. A chat interface module (possibly integrated in the same container or via another microservice) for user interaction.

F. Deployment and Orchestration:
- Docker and Docker Compose as the underlying infrastructure. Represent each primary component (or group) as its own Docker container, indicating container interconnectivity.

##### Relationships and Communication
- Dependencies such as API calls, Kafka subscriptions, and containerized service communications within Docker.

##### Architectural Patterns and Technologies
- Stress the modular architecture which separates real-time data processing from AI-assisted information retrieval.
- Use of containerization to decouple services and streamline deployment.
- Mention that the architecture supports scalability by leveraging Apache Kafka for data streaming and Docker Compose for service orchestration.
- Technologies: Python, Kafka, Docker, Streamlit, LangChain, OpenAI Embeddings, FAISS, and any external API services.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Data Sources
    subgraph "External Data Sources"
        ext_stocks["External Stock APIs"]:::external
        ext_news["News Sources"]:::external
    end

    %% Data Ingestion & Streaming
    subgraph "Data Ingestion & Streaming"
        s_kafka["streaming_kafka.py"]:::ingestion
        a_kafka["alpaca_kafka.py"]:::ingestion
        alpha["alphavantage.py"]:::ingestion
    end

    %% Kafka as Central Hub
    kafka["Kafka Broker"]:::kafka

    %% Data Processing & Prediction Module
    subgraph "Data Processing & Prediction"
        prediction["prediction.py"]:::processing
        indicators["indicators_test.py"]:::processing
        graphs["traditionalGraphs.py"]:::processing
    end

    %% Interactive Dashboard Layer
    subgraph "Interactive Dashboard"
        dashboard["dashboard.py"]:::dashboard
        app["app.py"]:::dashboard
    end

    %% StockBot Component
    subgraph "StockBot Module"
        stockbot_main["main.py"]:::stockbot
        vector["vector_index.pkl"]:::stockbot
    end

    %% Deployment and Orchestration
    deployment["docker-compose.yml"]:::deployment

    %% Data Flow Arrows
    ext_stocks --> s_kafka
    ext_stocks --> a_kafka
    ext_stocks --> alpha

    s_kafka --> kafka
    a_kafka --> kafka
    alpha --> kafka

    kafka --> prediction

    prediction --> dashboard
    prediction --> indicators
    prediction --> graphs

    ext_news --> stockbot_main

    %% Deployment links (Container orchestration to services)
    deployment --- s_kafka
    deployment --- a_kafka
    deployment --- alpha
    deployment --- prediction
    deployment --- dashboard
    deployment --- stockbot_main

    %% Click Events
    click s_kafka "https://github.com/shrimaliharshal/tradewise/blob/main/streaming_kafka.py"
    click a_kafka "https://github.com/shrimaliharshal/tradewise/blob/main/alpaca_kafka.py"
    click alpha "https://github.com/shrimaliharshal/tradewise/blob/main/alphavantage.py"
    click prediction "https://github.com/shrimaliharshal/tradewise/blob/main/prediction.py"
    click indicators "https://github.com/shrimaliharshal/tradewise/blob/main/indicators_test.py"
    click graphs "https://github.com/shrimaliharshal/tradewise/blob/main/traditionalGraphs.py"
    click dashboard "https://github.com/shrimaliharshal/tradewise/blob/main/dashboard.py"
    click app "https://github.com/shrimaliharshal/tradewise/blob/main/app.py"
    click stockbot_main "https://github.com/shrimaliharshal/tradewise/blob/main/StockBot/main.py"
    click vector "https://github.com/shrimaliharshal/tradewise/blob/main/StockBot/vector_index.pkl"
    click deployment "https://github.com/shrimaliharshal/tradewise/blob/main/docker-compose.yml"

    %% Styles and Color Definitions
    classDef external fill:#E0E0E0,stroke:#333,stroke-width:2px;
    classDef ingestion fill:#AEDFF7,stroke:#333,stroke-width:2px;
    classDef kafka fill:#FDE2B2,stroke:#333,stroke-width:2px;
    classDef processing fill:#C2F0C2,stroke:#333,stroke-width:2px;
    classDef dashboard fill:#FFB347,stroke:#333,stroke-width:2px;
    classDef stockbot fill:#FFDEAD,stroke:#333,stroke-width:2px;
    classDef deployment fill:#D1C4E9,stroke:#333,stroke-width:2px;
```
