# bmoscon/cryptofeed architecture

> Architecture diagram of the GitHub repository bmoscon/cryptofeed, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bmoscon/cryptofeed
- Repository: https://github.com/bmoscon/cryptofeed

## Overview

##### Main components
- Feed Handler Core – This is the orchestrator that instantiates feeds, manages subscriptions, and starts the event loop.
- Exchange Modules – Under the “cryptofeed/exchanges” directory you have individual modules/classes that implement connection logic and data translation for each supported exchange (e.g., Binance, Coinbase, Kraken, etc.). Also note the “mixins” which provide shared REST functionality.
- Callbacks & Data Normalization – The package offers a system for user-registered callbacks (like ticker, trades, book updates) that are invoked once the data is processed.
- Connection & Messaging – Components such as “cryptofeed/connection.py” and “cryptofeed/connection_handler.py” are responsible for establishing and managing the actual communication with exchange endpoints.
- Utilities & Helpers – The “cryptofeed/util” and error-handling modules support tasks like book management, performance, timing, and format conversions.

##### Relationships and interactions
- Each exchange module (e.g., binance.py, coinbase.py) connects directly to the corresponding exchange endpoint (via websockets and/or REST) and processes incoming messages.
- Configuration components (like config.py and config.yaml) ensure that the feed handler and exchange modules use the proper settings. You might represent these as a configuration store that feeds into the system.
- Testing and tooling (e.g., “tests/”, “tools/”, “examples/”) can be loosely represented as supporting verification layers, noting that they validate and demonstrate end-to-end data flow.

##### Architectural patterns and design principles
- Modular Design – The library is structured with clear separations (feed handler, exchanges, backends, utilities) to facilitate extensibility.
- Callback-Driven Data Processing – Callbacks serve as hooks for client code, so representing these as “listener” components is essential.
- Plug-in Backends – Backends are chosen based on user needs; represent these as optional branches that the main data flow can fork to.
- Abstraction Layers – Note that exchange implementations abstract both websocket and REST interactions and that mixins handle shared functionalities.

##### Relevant technologies
- Python 3.8+ (as noted in the README)
- Asynchronous programming (likely using asyncio)
- Supported external systems (Redis, MongoDB, PostgreSQL, Kafka, ZeroMQ, etc.) should be labeled next to their respective backend components.
- You may add a “Configuration & Deployment” note that includes files like setup.py, config.yaml, and CI/CD config files from .github/workflows, Travis, etc.

##### Specific elements
- A central "FeedHandler" module that collects and initiates feeds.
- Branches for each major exchange integration (grouped under an "Exchanges" module) showing how data enters.
- A separate “Backends” branch showing various storage or messaging output options.
- A Configuration component feeding into the Feed Handler.
- Notations for support utilities (book management, error handling) that are used internally.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Configuration
    CONF["Configuration (config.py/config.yaml)"]:::config

    %% Core Feed Handling
    FEED["FeedHandler Core"]:::core

    %% Exchange Integrations
    subgraph "Exchange Integrations"
        EX_DIR["Exchange Modules"]:::exchange
        BIN["Binance"]:::exchange
        CBN["Coinbase"]:::exchange
        MIX["Mixins"]:::exchange
    end

    %% Connection & Messaging
    subgraph "Connection & Messaging"
        CONN["Connection"]:::connection
        CONNH["Connection Handler"]:::connection
    end

    %% Callbacks & Data Normalization
    CB["Callback Dispatcher"]:::callback

    %% Backends
    BACKENDS["Backend Integrations (Redis, Kafka, etc.)"]:::backend

    %% Utilities & Helpers
    UTIL["Utilities & Helpers"]:::utility

    %% Relationships
    CONF -->|"configFlow"| FEED
    FEED -->|"registerFeeds"| EX_DIR
    FEED -->|"initConnection"| CONN
    FEED -->|"manageConnHandler"| CONNH
    EX_DIR -->|"initConnection"| CONN
    EX_DIR -->|"handleConnHandler"| CONNH
    EX_DIR -->|"branchToBinance"| BIN
    EX_DIR -->|"branchToCoinbase"| CBN
    EX_DIR -->|"branchToMixins"| MIX
    BIN -->|"dataEvent"| CB
    CBN -->|"dataEvent"| CB
    MIX -->|"dataEvent"| CB
    FEED -->|"dispatchCallback"| CB
    CB -->|"forwardToBackend"| BACKENDS
    FEED -->|"utilizeHelpers"| UTIL
    EX_DIR -->|"utilizeHelpers"| UTIL

    %% Styles
    classDef config fill:#ffcc00,stroke:#333,stroke-width:2px;
    classDef core fill:#add8e6,stroke:#333,stroke-width:2px;
    classDef exchange fill:#90ee90,stroke:#333,stroke-width:2px;
    classDef connection fill:#ffb6c1,stroke:#333,stroke-width:2px;
    classDef callback fill:#dda0dd,stroke:#333,stroke-width:2px;
    classDef backend fill:#fffacd,stroke:#333,stroke-width:2px;
    classDef utility fill:#d3d3d3,stroke:#333,stroke-width:2px;

    %% Click Events
    click FEED "https://github.com/bmoscon/cryptofeed/blob/master/cryptofeed/feedhandler.py"
    click EX_DIR "https://github.com/bmoscon/cryptofeed/tree/master/cryptofeed/exchanges"
    click CONN "https://github.com/bmoscon/cryptofeed/blob/master/cryptofeed/connection.py"
    click CONNH "https://github.com/bmoscon/cryptofeed/blob/master/cryptofeed/connection_handler.py"
    click CB "https://github.com/bmoscon/cryptofeed/blob/master/cryptofeed/callback.py"
    click BACKENDS "https://github.com/bmoscon/cryptofeed/tree/master/cryptofeed/backends"
    click UTIL "https://github.com/bmoscon/cryptofeed/tree/master/cryptofeed/util"
```
