# leberknecht/neuralcoin architecture

> Architecture diagram of the GitHub repository leberknecht/neuralcoin, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/leberknecht/neuralcoin
- Repository: https://github.com/leberknecht/neuralcoin

## Overview

##### Main Components

a. Data Collection Layer:
- NodeJS Scrapers (for Poloniex, Bittrex, Bitstamp, Bitfinex)
- WebSocket connections to exchanges
- Order book collectors

b. Message Queue Layer:
- RabbitMQ for inter-service communication
- Multiple queues for different purposes (trades, predictions, training)

c. Core Application Layer:
- Symfony 3 Application
- Database (MySQL)
- Entity management (Doctrine)
- WebSocket server for frontend communication

d. Machine Learning Layer:
- Python services
- TensorFlow implementation
- Network creation and training
- Prediction generation

e. Frontend Layer:
- Web interface
- ReactJS components
- Real-time trading tools
- Network visualization

**[External Exchanges] -> [Scrapers Layer] -> [Message Queue] -> [Core Application] -> [Frontend]**
- ↕
- [Machine Learning Layer]

##### Detailed Component Relationships

a. Data Flow:
- Trade data flow from exchanges through scrapers
- WebSocket connections for real-time data
- Persistent storage points (MySQL)
- Prediction request/response paths

b. Service Communication:
- RabbitMQ message paths between components
- WebSocket communication to frontend
- File system interactions for network configs
- S3 storage integration points

##### Key Technical Elements to Highlight

- Docker containers and their relationships
- Database connections and migrations
- Network training data flow
- Real-time prediction pipeline
- Frontend data updates

##### Infrastructure Components

- Docker containers (show container boundaries)
- Shared volumes
- Network connections
- Cache layers

##### User Interface (right)

- Container boundaries
- Data flow directions
- Component types

Keep the abstraction level high enough to show the overall architecture while including critical implementation details that affect the system's behavior, such as the queue-based communication and the separation between network creation and training processes.

Remember to indicate the technology stack used in each component:
- NodeJS for scrapers
- Symfony/PHP for core application
- Python/TensorFlow for ML
- ReactJS for frontend
- RabbitMQ for messaging
- MySQL for storage

## Diagram (Mermaid)

```mermaid
graph TB
    %% Styles and classes
    classDef external fill:#B3E5FC,stroke:#0288D1
    classDef scraper fill:#C8E6C9,stroke:#388E3C
    classDef queue fill:#FFF9C4,stroke:#FBC02D
    classDef core fill:#FFCDD2,stroke:#D32F2F
    classDef ml fill:#E1BEE7,stroke:#7B1FA2
    classDef frontend fill:#FFE0B2,stroke:#F57C00
    classDef database fill:#B2DFDB,stroke:#00796B

    %% External Exchanges
    subgraph Exchanges
        Binance["Binance API"]:::external
        Bitfinex["Bitfinex API"]:::external
        Bitstamp["Bitstamp API"]:::external
        Bittrex["Bittrex API"]:::external
        Poloniex["Poloniex API"]:::external
    end

    %% Data Collection Layer
    subgraph DataCollection
        BinanceF["Binance Feed"]:::scraper
        BitfinexF["Bitfinex Feed"]:::scraper
        BitstampF["Bitstamp Feed"]:::scraper
        BittrexF["Bittrex Feed"]:::scraper
        PoloniexF["Poloniex Feed"]:::scraper
        OrderBook["Order Book Collector"]:::scraper
    end

    %% Message Queue Layer
    subgraph MessageQueue
        RMQ["RabbitMQ"]:::queue
        MsgDef["Message Definitions"]:::queue
        QueueSvc["Queue Services"]:::queue
        MsgHandler["Message Handlers"]:::queue
    end

    %% Core Application Layer
    subgraph CoreApp
        Symfony["Symfony Core"]:::core
        Entities["Database Entities"]:::core
        CoreSvc["Core Services"]:::core
        DB[(MySQL Database)]:::database
        WSServer["WebSocket Server"]:::core
    end

    %% Machine Learning Layer
    subgraph ML
        NetMgr["Network Manager"]:::ml
        TFImpl["TensorFlow Implementation"]:::ml
        Training["Training Service"]:::ml
        Prediction["Prediction Service"]:::ml
    end

    %% Frontend Layer
    subgraph Frontend
        React["React Components"]:::frontend
        Trading["Trading Tools"]:::frontend
        Views["Views"]:::frontend
        Controllers["Controllers"]:::frontend
    end

    %% Connections
    Binance --> BinanceF
    Bitfinex --> BitfinexF
    Bitstamp --> BitstampF
    Bittrex --> BittrexF
    Poloniex --> PoloniexF
    Bittrex --> OrderBook

    BinanceF & BitfinexF & BitstampF & BittrexF & PoloniexF & OrderBook --> RMQ
    RMQ --> MsgDef
    MsgDef --> QueueSvc
    QueueSvc --> MsgHandler
    MsgHandler --> Symfony

    Symfony --> DB
    Symfony <--> CoreSvc
    CoreSvc <--> Entities
    Symfony <--> WSServer

    Symfony <--> NetMgr
    NetMgr <--> TFImpl
    TFImpl <--> Training
    TFImpl <--> Prediction

    WSServer --> React
    React <--> Trading
    React --> Views
    Controllers --> Views

    %% Click events for component mapping
    click BinanceF "https://github.com/Leberknecht/neuralcoin/blob/master/nodejs/binance.js"
    click BitfinexF "https://github.com/Leberknecht/neuralcoin/blob/master/nodejs/bitfinex-feed.js"
    click BitstampF "https://github.com/Leberknecht/neuralcoin/blob/master/nodejs/bitstamp-feed.js"
    click BittrexF "https://github.com/Leberknecht/neuralcoin/blob/master/nodejs/bittrex-feed.js"
    click PoloniexF "https://github.com/Leberknecht/neuralcoin/blob/master/nodejs/poloniex-feed.js"
    click OrderBook "https://github.com/Leberknecht/neuralcoin/blob/master/nodejs/bittrex-orderbook.js"
    click MsgDef "https://github.com/Leberknecht/neuralcoin/tree/master/src/FeedBundle/Message"
    click QueueSvc "https://github.com/Leberknecht/neuralcoin/tree/master/src/FeedBundle/Service"
    click MsgHandler "https://github.com/Leberknecht/neuralcoin/tree/master/src/NetworkBundle/Service"
    click Entities "https://github.com/Leberknecht/neuralcoin/tree/master/src/DataModelBundle/Entity"
    click CoreSvc "https://github.com/Leberknecht/neuralcoin/tree/master/src/DataModelBundle/Service"
    click NetMgr "https://github.com/Leberknecht/neuralcoin/blob/master/python/NetworkManager.py"
    click TFImpl "https://github.com/Leberknecht/neuralcoin/blob/master/python/NetworkManagerTf.py"
    click Training "https://github.com/Leberknecht/neuralcoin/blob/master/python/consume-train-network.py"
    click Prediction "https://github.com/Leberknecht/neuralcoin/blob/master/python/consume-get-prediction.py"
    click React "https://github.com/Leberknecht/neuralcoin/tree/master/src/FrontendBundle/Resources/public/react"
    click Trading "https://github.com/Leberknecht/neuralcoin/blob/master/src/FrontendBundle/Resources/public/js/TradingTools.js"
    click Controllers "https://github.com/Leberknecht/neuralcoin/tree/master/src/FrontendBundle/Controller"
    click Views "https://github.com/Leberknecht/neuralcoin/tree/master/src/FrontendBundle/Resources/views"
    click WSServer "https://github.com/Leberknecht/neuralcoin/blob/master/nodejs/websocket-server.js"
```
