# dropbox/godropbox architecture

> Architecture diagram of the GitHub repository dropbox/godropbox, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dropbox/godropbox
- Repository: https://github.com/dropbox/godropbox

## Overview

##### Project type and purpose
- This is a “common libraries” Go monorepo (godropbox) providing utility packages for Go services on Linux/x64.
- There is no single “frontend” or “backend” service; rather a collection of modules grouped by functionality (I/O, container, concurrency, database, caching, networking, etc.).

##### File structure
- Top‐level packages:
– Primitives & errors: errors, time2, sync2, context2, math2, murmur3, oihash
– I/O & buffering: io2, bufio2, bytes2, encoding2
– Containers & collections: container/, lockstore, resource_pool, singleton, sort2, stats
– Concurrency & orchestration: executor, rate_limiter, sync2, gocheck2
– Data stores & caching: caching, memcache
– Database support: database/binlog, database/sqlbuilder, database/sqltypes, proto/mysql
– Networking: net2, cinterop (C interop layered on top of Go), sys/filelock
– Test & utilities: gocheck2, examples under various dirs
- Cross‐package dependencies (e.g., io2 uses bufio2/bytes2; caching uses lockstore & resource_pool; database/sqlbuilder uses database/sqltypes; memcache stands on net2).

##### README for insights
- Confirms purpose: “common libraries for writing Go services”

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Core Primitives & Utilities
    subgraph "Core Primitives & Utilities"
        direction TB
        ERR["errors"]:::core
        TIME["time2"]:::core
        SYNC["sync2"]:::core
        CTX["context2"]:::core
        RAND["math2/rand2"]:::core
        MURMUR["murmur3"]:::core
        OIHASH["oihash"]:::core
        STRIN["strings/intern"]:::core
        CHASH["hash2/consistent_hash"]:::core
    end

    %% I/O & Encoding
    subgraph "I/O & Encoding"
        direction TB
        BUF["bufio2"]:::io
        BYT["bytes2"]:::io
        IO2["io2"]:::io
        ENC["encoding2"]:::io
    end

    %% Container & Data Structures
    subgraph "Container & Data Structures"
        direction TB
        CONT_CORE["container (bitarray, bitvector, set)"]:::container
        CONT_CONC["container/concurrent"]:::container
        LSTORE["lockstore"]:::container
        RPOOL["resource_pool"]:::container
        SING["singleton"]:::container
        SORT["sort2"]:::container
        STATS["stats"]:::container
        ARRAY["array2"]:::container
    end

    %% Concurrency & Coordination
    subgraph "Concurrency & Coordination"
        direction TB
        EXE["executor"]:::concurrency
        RATE["rate_limiter"]:::concurrency
        GC2["gocheck2"]:::concurrency
    end

    %% Resource Management & Caching
    subgraph "Resource Management & Caching"
        direction TB
        CACHE_IF["caching/interface"]:::cache
        CACHE_GEN["caching/generic_storage"]:::cache
        CACHE_LOC["caching/local_map_storage"]:::cache
        CACHE_RATE["caching/rate_limited_storage"]:::cache
        MEM_IF["memcache/interface"]:::cache
        MEM_RAW_ASC["memcache/raw_ascii_client"]:::cache
        MEM_RAW_BIN["memcache/raw_binary_client"]:::cache
        MEM_SHARD["memcache/sharded_client"]:::cache
    end

    %% Database & Protocols
    subgraph "Database & Protocols"
        direction TB
        BINLOG["database/binlog"]:::db
        SQLT["database/sqltypes"]:::db
        SQLB["database/sqlbuilder"]:::db
        PROTO["proto/mysql"]:::proto
    end

    %% Networking & C Interop
    subgraph "Networking & C Interop"
        direction TB
        NET2["net2"]:::network
        FILEL["sys/filelock"]:::network
        CINT["cinterop"]:::cinterop
    end

    %% External Systems
    MySQL(["MySQL Server"]):::external
    MEMC(["Memcached"]):::external
    CRUN(["C Runtime"]):::external
    LNX(["Linux OS"]):::linux

    %% Dependencies
    ERR -->|used by| IO2
    TIME -->|time functions| EXE
    SYNC -->|sync primitives| CONT_CONC
    CTX -->|cancellation| EXE
    RAND -->|random| CACHE_RATE
    MURMUR -->|hash| CACHE_IF
    OIHASH -->|hash| CHASH
    STRIN -->|interning| CACHE_LOC
    CHASH -->|consistent hashing| MEM_SHARD

    BUF -->|buffer| IO2
    BYT -->|byte ops| IO2
    IO2 -->|I/O| NET2
    ENC -->|encoding| MEM_RAW_ASC

    CONT_CORE -->|foundation| CONT_CONC
    CONT_CONC -->|uses| LSTORE
    LSTORE -->|locking| CACHE_IF
    RPOOL -->|pooling| CACHE_GEN

    EXE -->|exec tasks| CACHE_RATE
    RATE -->|limit| CACHE_RATE
    GC2 -->|tests| EXE

    CACHE_IF -->|impl| CACHE_GEN
    CACHE_GEN -->|stores| CACHE_LOC
    CACHE_LOC -->|local| CACHE_RATE
    CACHE_RATE -->|rate| CACHE_IF

    MEM_IF -->|protocol| MEM_RAW_ASC
    MEM_RAW_ASC -->|ASCII| MEMC
    MEM_RAW_BIN -->|Binary| MEMC
    MEM_SHARD -->|shard| MEMC

    BINLOG -->|reads| PROTO
    SQLT -->|types| SQLB
    PROTO -->|pb.go| SQLB
    SQLB -->|builds| BINLOG

    NET2 -->|network| CINT
    FILEL -->|lock| CINT
    CINT -->|interop| CRUN

    %% External integration arrows
    PROTO -->|MySQL protocol| MySQL
    MEM_SHARD -->|Memcached| MEMC
    CINT -->|C lib| CRUN
    LNX -.-> ERR
    LNX -.-> IO2
    LNX -.-> NET2

    %% Click Events
    click ERR "https://github.com/dropbox/godropbox/blob/master/errors/errors.go"
    click TIME "https://github.com/dropbox/godropbox/blob/master/time2/time2.go"
    click SYNC "https://github.com/dropbox/godropbox/blob/master/sync2/atomic.go"
    click CTX "https://github.com/dropbox/godropbox/blob/master/context2/context.go"
    click RAND "https://github.com/dropbox/godropbox/blob/master/math2/rand2/rand.go"
    click MURMUR "https://github.com/dropbox/godropbox/blob/master/murmur3/murmur3.go"
    click OIHASH "https://github.com/dropbox/godropbox/blob/master/oihash/oihash.go"
    click STRIN "https://github.com/dropbox/godropbox/blob/master/strings/intern.go"
    click CHASH "https://github.com/dropbox/godropbox/blob/master/hash2/consistent_hash.go"
    click BUF "https://github.com/dropbox/godropbox/blob/master/bufio2/look_ahead_buffer.go"
    click BYT "https://github.com/dropbox/godropbox/blob/master/bytes2/compare.go"
    click IO2 "https://github.com/dropbox/godropbox/blob/master/io2/pipelined_copy.go"
    click ENC "https://github.com/dropbox/godropbox/blob/master/encoding2/hex.go"
    click CONT_CORE "https://github.com/dropbox/godropbox/blob/master/container/bitarray/bitarray.go"
    click CONT_CONC "https://github.com/dropbox/godropbox/blob/master/container/concurrent/lrucache.go"
    click LSTORE "https://github.com/dropbox/godropbox/tree/master/lockstore/"
    click RPOOL "https://github.com/dropbox/godropbox/tree/master/resource_pool/"
    click SING "https://github.com/dropbox/godropbox/tree/master/singleton/"
    click SORT "https://github.com/dropbox/godropbox/tree/master/sort2/"
    click STATS "https://github.com/dropbox/godropbox/tree/master/stats/"
    click ARRAY "https://github.com/dropbox/godropbox/blob/master/array2/array2.go"
    click EXE "https://github.com/dropbox/godropbox/blob/master/executor/executor.go"
    click RATE "https://github.com/dropbox/godropbox/blob/master/rate_limiter/rate_limiter.go"
    click GC2 "https://github.com/dropbox/godropbox/blob/master/gocheck2/checkers.go"
    click CACHE_IF "https://github.com/dropbox/godropbox/blob/master/caching/interface.go"
    click CACHE_GEN "https://github.com/dropbox/godropbox/blob/master/caching/generic_storage.go"
    click CACHE_LOC "https://github.com/dropbox/godropbox/blob/master/caching/local_map_storage.go"
    click CACHE_RATE "https://github.com/dropbox/godropbox/blob/master/caching/rate_limited_storage.go"
    click MEM_IF "https://github.com/dropbox/godropbox/blob/master/memcache/interface.go"
    click MEM_RAW_ASC "https://github.com/dropbox/godropbox/blob/master/memcache/raw_ascii_client.go"
    click MEM_RAW_BIN "https://github.com/dropbox/godropbox/blob/master/memcache/raw_binary_client.go"
    click MEM_SHARD "https://github.com/dropbox/godropbox/blob/master/memcache/sharded_client.go"
    click BINLOG "https://github.com/dropbox/godropbox/tree/master/database/binlog/"
    click SQLT "https://github.com/dropbox/godropbox/blob/master/database/sqltypes/sqltypes.go"
    click SQLB "https://github.com/dropbox/godropbox/blob/master/database/sqlbuilder/statement.go"
    click PROTO "https://github.com/dropbox/godropbox/blob/master/proto/mysql/binlog.proto"
    click NET2 "https://github.com/dropbox/godropbox/blob/master/net2/connection_pool.go"
    click FILEL "https://github.com/dropbox/godropbox/blob/master/sys/filelock/filelock.go"
    click CINT "https://github.com/dropbox/godropbox/blob/master/cinterop/server.go"

    %% Styles
    classDef core fill:#A9A9A9,stroke:#333,stroke-width:1px
    classDef io fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef container fill:#90EE90,stroke:#333,stroke-width:1px
    classDef concurrency fill:#FFA500,stroke:#333,stroke-width:1px
    classDef cache fill:#DDA0DD,stroke:#333,stroke-width:1px
    classDef db fill:#FF6347,stroke:#333,stroke-width:1px
    classDef proto fill:#FFE4B5,stroke:#333,stroke-width:1px,stroke-dasharray: 5 5
    classDef network fill:#87CEEB,stroke:#333,stroke-width:1px
    classDef cinterop fill:#FFD700,stroke:#333,stroke-width:1px
    classDef external fill:#E0FFFF,stroke:#333,stroke-width:1px
    classDef linux fill:#F0F8FF,stroke:#333,stroke-width:1px,stroke-dasharray: 2 2
```
