# redis/redis-py architecture

> Architecture diagram of the GitHub repository redis/redis-py, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/redis/redis-py
- Repository: https://github.com/redis/redis-py

## Overview

##### Project type and purpose
- This is an open-source Python client library for Redis (“redis-py”).
- Purpose: provide sync and async APIs to connect to a Redis server (single instance, cluster, sentinel), execute commands, pipelines, pub/sub, and support advanced features (RESP2/RESP3, clustering, authentication, backoff, retry).

##### File structure
- Top-level: configuration (.github/, Docker, CI, docs, benchmarks, tests).
- “redis/” package: core modules
- connection.py, client.py, cluster.py, asyncio/…, auth/, commands/, parsers/…, retry.py, backoff.py, lock.py, utils.py
- Command modules under redis/commands cover Redis data types and modules (core, cluster, bf, json, search, timeseries, vectorset, sentinel, etc.)
- Async support in redis/asyncio mirrors sync API but uses asyncio transports and connection pools.
- Parsers in redis/_parsers implement RESP2/3 decoding, with optional hiredis integration.
- Tests and doctests verify functionality across sync, async, clusters, commands, codecs, SSL, pub/sub.
- Benchmarks measure performance of core operations.
- Documentation and examples illustrate usage patterns.

##### Extract insights from README
- Highlights installation, basic usage, RESP3 support, connection pools, pipelines, pub/sub.
- Mentions advanced topics (MULTI/EXEC via Pipeline, PubSub, clustering).
- Emphasizes that hiredis is used if available.

a. Main components
1. Client API Layer
- Sync client (redis/client.py)
- Async client (redis/asyncio/client.py)
2. Connection & Pooling
- Connection classes (redis/connection.py, redis/asyncio/connection.py)
- ConnectionPool management
- Sentinel & Cluster managed connections (redis/sentinel.py, redis/cluster.py, their async variants)
3. Command Dispatcher
- redis/client.py dispatches user calls to command routers
- redis/commands/* define commands grouped by data type or module
4. Protocol Parsers
- RESP2/RESP3 logic (redis/_parsers/)
- Optional hiredis extension
5. Utility & Cross-cutting
- Backoff and retry (redis/backoff.py, redis/retry.py)
- Locking (redis/lock.py, redis/asyncio/lock.py)
- Authentication (redis/auth/)
- Event & PubSub (redis/event.py, redis/client.py pubsub method)
6. External Systems
- Redis server instances (standalone, Sentinel, Cluster)
- Hiredis C library (optional)

b. Relationships & interactions
- The Client API Layer calls into Connection & Pooling to obtain a live socket.
- Commands are serialized (using command packs) and sent over the connection.
- Parsers decode responses into Python objects.
- Retry/backoff wraps connections when errors occur.
- Pipeline batches commands at the client layer, wraps with MULTI/EXEC where needed.
- PubSub uses a dedicated connection stateful subscription.
- Async client follows same flow but uses asyncio transports and awaits coroutines.
- Cluster client routes commands to appropriate node based on key hashing.
- Sentinel client discovers masters/slaves via sentinel service.

c. Architectural patterns & design principles
- Layered architecture: API → Connection → Protocol → Transport
- Strategy/dispatch pattern: commands are implemented as pluggable modules in redis/commands
- Decorator/wrapper for retry and backoff
- Adapter for optional hiredis parser
- Template method for sync vs async (mirror modules)
- Factory for ConnectionPool and cluster/sentinel clients

d. Key technologies & libraries
- Python 3.7+ standard library (socket, asyncio)
- Optional hiredis C extension for faster parsing
- CI tooling (GitHub Actions), Docker for examples (docker-compose.yml)
- Sphinx for docs, pytest for tests, black/mypy for linting

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% User Application
    subgraph "User Application"
        UA["User Application"]:::external
    end

    %% redis-py Library
    subgraph "redis-py Library"
        %% API Layer
        subgraph "Client API Layer"
            SyncAPI["Sync Client API"]:::api
            AsyncAPI["Async Client API"]:::api
        end

        %% Connection & Pooling
        subgraph "Connection & Pooling"
            BackoffSync["Backoff"]:::util
            RetrySync["Retry"]:::util
            SyncConn["Sync ConnectionPool"]:::conn
            SentinelSync["Sentinel Client (Sync)"]:::conn
            ClusterSync["Cluster Client (Sync)"]:::conn

            BackoffAsync["Backoff (Async)"]:::util
            RetryAsync["Retry (Async)"]:::util
            AsyncConn["Async ConnectionPool"]:::conn
            SentinelAsync["Sentinel Client (Async)"]:::conn
            ClusterAsync["Cluster Client (Async)"]:::conn
        end

        %% Command Dispatcher & Modules
        subgraph "Command Dispatch & Modules"
            Dispatcher["Command Dispatcher"]:::api
            Commands["Command Modules"]:::api
        end

        %% Protocol Parsers
        subgraph "Protocol Parsers"
            RESP2["RESP2 Parser"]:::proto
            RESP3["RESP3 Parser"]:::proto
            ParserBase["Parser Base & Helpers"]:::proto
            HiredisAdapter["Hiredis Adapter"]:::proto
        end

        %% Utilities & Cross-cutting
        subgraph "Utilities & Cross-cutting"
            LockSync["Lock (Sync)"]:::util
            Auth["Authentication"]:::util
            Event["Pub/Sub & Events"]:::util
            UtilsSync["General Utils (Sync)"]:::util
            CRC["CRC Utilities"]:::util
            Cred["Credentials Helper"]:::util
            OCSP["OCSP Support"]:::util

            LockAsync["Lock (Async)"]:::util
            UtilsAsync["General Utils (Async)"]:::util
        end
    end

    %% External Systems
    subgraph "External Systems"
        Redis["Redis Server"]:::external
        SentinelService["Sentinel Service"]:::external
        ClusterNodes["Cluster Nodes"]:::external
        HiredisLib["Hiredis C Library"]:::external
    end

    %% Connections & Flows
    UA -->|"calls"| SyncAPI
    UA -->|"awaits"| AsyncAPI

    SyncAPI -->|"dispatches"| Dispatcher
    Dispatcher -->|"loads commands"| Commands

    SyncAPI -->|"uses"| BackoffSync
    BackoffSync -->|"wraps"| RetrySync
    RetrySync -->|"manages"| SyncConn
    SyncAPI -->|"auth|parse|lock"| Auth
    SyncAPI -->|"pubsub"| Event
    SyncAPI -->|"locking"| LockSync
    SyncAPI -->|"utils"| UtilsSync

    AsyncAPI -->|"wraps"| BackoffAsync
    BackoffAsync -->|"wraps"| RetryAsync
    RetryAsync -->|"manages"| AsyncConn
    AsyncAPI -->|"auth|parse|lock"| Auth
    AsyncAPI -->|"pubsub"| Event
    AsyncAPI -->|"locking"| LockAsync
    AsyncAPI -->|"utils"| UtilsAsync

    SyncConn -->|"connects"| Redis
    AsyncConn -->|"connects async"| Redis

    SyncConn -->|"serializes"| RESP2
    SyncConn -->|"serializes"| RESP3
    AsyncConn -->|"serializes"| RESP2
    AsyncConn -->|"serializes"| RESP3

    RESP2 -->|"uses"| ParserBase
    RESP3 -->|"uses"| ParserBase
    HiredisLib -->|"optional adapter"| HiredisAdapter
    HiredisAdapter -->|"replaces"| RESP2
    HiredisAdapter -->|"replaces"| RESP3

    Commands -->|"sent over"| SyncConn
    Commands -->|"sent over async"| AsyncConn

    Dispatcher -->|"routes cluster"| ClusterSync
    Dispatcher -->|"routes cluster async"| ClusterAsync
    ClusterSync -->|"connects to"| ClusterNodes
    ClusterAsync -->|"connects to"| ClusterNodes

    Dispatcher -->|"discovers|connects"| SentinelSync
    Dispatcher -->|"discovers async"| SentinelAsync
    SentinelSync -->|"queries"| SentinelService
    SentinelAsync -->|"queries"| SentinelService

    Event -->|"subscribes"| Redis
    Event -->|"publishes"| Redis

    %% Click Events
    click SyncAPI "https://github.com/redis/redis-py/blob/master/redis/client.py"
    click AsyncAPI "https://github.com/redis/redis-py/blob/master/redis/asyncio/client.py"
    click SyncConn "https://github.com/redis/redis-py/blob/master/redis/connection.py"
    click AsyncConn "https://github.com/redis/redis-py/blob/master/redis/asyncio/connection.py"
    click SentinelSync "https://github.com/redis/redis-py/blob/master/redis/sentinel.py"
    click SentinelAsync "https://github.com/redis/redis-py/blob/master/redis/asyncio/sentinel.py"
    click ClusterSync "https://github.com/redis/redis-py/blob/master/redis/cluster.py"
    click ClusterAsync "https://github.com/redis/redis-py/blob/master/redis/asyncio/cluster.py"
    click Dispatcher "https://github.com/redis/redis-py/blob/master/redis/client.py"
    click Commands "https://github.com/redis/redis-py/tree/master/redis/commands/"
    click RESP2 "https://github.com/redis/redis-py/blob/master/redis/_parsers/resp2.py"
    click RESP3 "https://github.com/redis/redis-py/blob/master/redis/_parsers/resp3.py"
    click ParserBase "https://github.com/redis/redis-py/blob/master/redis/_parsers/base.py"
    click HiredisAdapter "https://github.com/redis/redis-py/blob/master/redis/_parsers/hiredis.py"
    click BackoffSync "https://github.com/redis/redis-py/blob/master/redis/backoff.py"
    click RetrySync "https://github.com/redis/redis-py/blob/master/redis/retry.py"
    click LockSync "https://github.com/redis/redis-py/blob/master/redis/lock.py"
    click Auth "https://github.com/redis/redis-py/tree/master/redis/auth/"
    click Event "https://github.com/redis/redis-py/blob/master/redis/event.py"
    click UtilsSync "https://github.com/redis/redis-py/blob/master/redis/utils.py"
    click CRC "https://github.com/redis/redis-py/blob/master/redis/crc.py"
    click Cred "https://github.com/redis/redis-py/blob/master/redis/credentials.py"
    click OCSP "https://github.com/redis/redis-py/blob/master/redis/ocsp.py"
    click RetryAsync "https://github.com/redis/redis-py/blob/master/redis/asyncio/retry.py"
    click LockAsync "https://github.com/redis/redis-py/blob/master/redis/asyncio/lock.py"
    click UtilsAsync "https://github.com/redis/redis-py/blob/master/redis/asyncio/utils.py"

    %% Styles
    classDef api fill:#D0E4F5,stroke:#000,stroke-width:1px
    classDef conn fill:#D5E8D4,stroke:#000,stroke-width:1px
    classDef proto fill:#F8CECC,stroke:#000,stroke-width:1px
    classDef util fill:#E1D5E7,stroke:#000,stroke-width:1px
    classDef external fill:#F5F5F5,stroke:#000,stroke-width:1px
```
