# douyu/jupiter architecture

> Architecture diagram of the GitHub repository douyu/jupiter, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/douyu/jupiter
- Repository: https://github.com/douyu/jupiter

## Overview

##### Project type and purpose
- This is JUPITER: a governance-oriented microservice framework written in Go. It provides:
- A CLI toolkit (cmd/jupiter) and code generators for gRPC, HTTP (Gin, Echo), error handling, field-masking.
- Core libraries for configuration, service registration/discovery, error codes, metrics, tracing, circuit-breaking (Sentinel), logging, lifecycle hooks, and more.
- Pluggable implementations for cache (Freecache, LRU), clients (Etcd, gRPC, Redis, RocketMQ, REST), data stores (GORM/MySQL, MongoX, TableStore), executors (XXL-Job), and server transports (xecho, xgin, xfasthttp, xgrpc, xgoframe).
- A Proto schema directory and Buf config for code generation.

##### File structure
- Top-level “cmd/…” contains CLI and codegen tools.
- “core/…” contains foundational abstractions (Application bootstrap, component wiring, ecode, metric, tracing, hooks, signals).
- “pkg/…” contains pluggable modules in categories:
- cache, client, conf, executor, registry, server, store, util, worker, xlog
- “proto/…” holds protobuf definitions and generation config.
- “test/…” holds integration/test fixtures including Docker Compose.
- “tools/…” and “website/…” support documentation and auxiliary tooling.

##### README for additional insights
- Macro-purpose: developer-facing microservice framework with governance features.
- External systems expected: Docker, gRPC clients, Redis, Etcd, SQL/Mongo, RocketMQ, Sentinel, Jaeger, Prometheus.
- Companion projects: Juno (console), jupiter-layout (template).

##### Main components & interactions (to show
a. CLI & Codegen Layer
– cmd/jupiter: “jupiter new”, “jupiter run” for scaffolding & runtime injection
– cmd/protoc-gen-go-* tools: generate stubs for HTTP frameworks & error/fieldmask code
b. Application Bootstrap
– core/application: loads configuration (pkg/conf), sets up logging (pkg/xlog), metrics, tracing (core/xtrace), error codes, hooks, and run loop.
c. Configuration
– pkg/conf: unified config API with multiple datasource plugins (file, Apollo, Etcd, HTTP)
d. Service Registration & Discovery
– pkg/registry: register service instances to Etcd (or nop) and resolve endpoints
e. Server Layer
– pkg/server/xecho, xgin, xfasthttp, xgrpc, xgoframe: transport adapters that plug into Application
f. Client Layer
– pkg/client/{grpc, redis, etcdv3, resty, rocketmq}: abstracted clients with interceptors, load-balancing, retries
g. Caching & Data Stores
– pkg/cache/…: in-process cache (freecache, LRU)
– pkg/store/{gorm, mongox, tablestore}: DB ORM/driver wrappers
h. Messaging & Task Execution
– pkg/executor/xxl: wrapper for XXL-Job tasks
– pkg/client/rocketmq: producer/consumer abstractions
i. Governance & Resilience
– core/sentinel: circuit-breaker, rate-limiter, system protection
– core/metric & external scraping (Prometheus)
– core/xtrace: distributed tracing via Jaeger or OTLP
j. Utilities
– pkg/util, pkg/worker, pkg/xlog: support routines (retry, reflection, cron jobs, logging rotation)

##### Relationships & data flow
- The Application bootstrap invokes:
- Configuration → Service Registration → Logging/Tracing → Metric initialization → Server startup
- Servers expose endpoints → incoming requests pass through middleware (Sentinel, Logging, Tracing) → business handlers
- Handlers call downstream Clients (gRPC, REST, Redis, RocketMQ), Data Stores, or Executors
- Outbound calls go through interceptors (retry, load-balancing, tracing).
- Results flow back through the same middleware in reverse.

##### Architectural patterns & design principles
- Layered architecture: CLI / Bootstrap → Core framework → Pluggable modules → External systems
- Dependency inversion: high-level modules depend on abstractions in “core” and “pkg” interfaces
- Plugin pattern: pkg/conf, pkg/registry, pkg/cache, pkg/store all expose interface + registerable implementations
- Code generation: proto → transport adapters / error wrappers to reduce boilerplate
- Governance cross-cutting concerns: tracing, metrics, error codes, circuit-breaking reside in core and injected via middleware

##### Elements
- Clear component labels (e.g., “pkg/server/xgrpc”, “core/sentinel”)
- Initialization sequence
- Request/response flow
- Outbound call paths
- CLI tools vs generated code
- Core abstractions vs pluggable modules
- External systems

##### Final checklist for detail
- Major plugin categories (conf, registry, server, client, cache, store, executor)
- Governance aspects (Sentinel, metric, trace) as cross-cutting layers

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% CLI & Codegen Layer
    subgraph "CLI & Codegen" 
        C1["cmd/jupiter"]:::blue
        C2["protoc-gen-go-echo"]:::blue
        C3["protoc-gen-go-gin"]:::blue
        C4["protoc-gen-go-xerror"]:::blue
        C5["protoc-gen-go-xfieldmask"]:::blue
        Proto["proto/ & buf.yaml"]:::blue
    end

    %% Application Bootstrap
    subgraph "Application Bootstrap"
        A1["core/application"]:::green
    end

    %% Core Services & Governance
    subgraph "Core Framework"
        Conf["pkg/conf"]:::green
        Reg["pkg/registry"]:::green
        Xlog["pkg/xlog"]:::green
        Sent["core/sentinel"]:::green
        Met["core/metric"]:::green
        Trace["core/xtrace"]:::green
    end

    %% Server Transports
    subgraph "Server Transports"
        S1["pkg/server/xecho"]:::orange
        S2["pkg/server/xgin"]:::orange
        S3["pkg/server/xfasthttp"]:::orange
        S4["pkg/server/xgoframe"]:::orange
        S5["pkg/server/xgrpc"]:::orange
    end

    %% Client Plugins
    subgraph "Client Plugins"
        Cl1["pkg/client/etcdv3"]:::orange
        Cl2["pkg/client/grpc"]:::orange
        Cl3["pkg/client/redis"]:::orange
        Cl4["pkg/client/resty"]:::orange
        Cl5["pkg/client/rocketmq"]:::orange
    end

    %% Cache, Store & Executor
    subgraph "Cache, Store & Executor"
        Cache1["pkg/cache/xfreecache"]:::orange
        Cache2["pkg/cache/xgolanglru"]:::orange
        Store1["pkg/store/gorm"]:::orange
        Store2["pkg/store/mongox"]:::orange
        Store3["pkg/store/tablestore"]:::orange
        Exec["pkg/executor/xxl"]:::orange
    end

    %% External Systems
    subgraph "External Systems"
        E1["Etcd"]:::gray
        E2["Redis"]:::gray
        E3["MySQL"]:::gray
        E4["MongoDB"]:::gray
        E5["RocketMQ"]:::gray
        E6["Jaeger"]:::gray
        E7["Prometheus"]:::gray
    end

    %% Initialization Flow
    C1 --> A1
    Proto --> C2 & C3 & C4 & C5
    A1 --> Conf
    A1 --> Reg
    A1 --> Xlog
    A1 --> Sent
    A1 --> Met
    A1 --> Trace
    A1 --> S1 & S2 & S3 & S4 & S5

    %% Service Discovery & Config
    Conf --> E1
    Reg --> E1

    %% Request Flow through Middleware
    S1 & S2 & S3 & S4 & S5 -->|apply Sentinel|Sent
    S1 & S2 & S3 & S4 & S5 -->|apply Tracing|Trace
    S1 & S2 & S3 & S4 & S5 -->|apply Metrics|Met
    S1 & S2 & S3 & S4 & S5 -->|log via|Xlog

    %% Handler Outbound Calls
    Sent & Trace & Met --> Cl1 & Cl2 & Cl3 & Cl4 & Cl5
    Sent & Trace & Met --> Cache1 & Cache2 & Store1 & Store2 & Store3 & Exec

    %% External Dependencies
    Cl1 --> E1
    Cl2 --> E?["gRPC Services"]:::gray
    Cl3 --> E2
    Cl4 --> E? 
    Cl5 --> E5
    Store1 --> E3
    Store2 --> E4
    Store3 -->|"TableStore"|E? 
    Exec --> E?["XXL-Job"]:::gray

    %% Governance External
    Met --> E7
    Trace --> E6

    %% Click Events
    click C1 "https://github.com/douyu/jupiter/blob/master/cmd/jupiter/main.go"
    click C2 "https://github.com/douyu/jupiter/blob/master/cmd/protoc-gen-go-echo/main.go"
    click C3 "https://github.com/douyu/jupiter/blob/master/cmd/protoc-gen-go-gin/main.go"
    click C4 "https://github.com/douyu/jupiter/blob/master/cmd/protoc-gen-go-xerror/main.go"
    click C5 "https://github.com/douyu/jupiter/blob/master/cmd/protoc-gen-go-xfieldmask/main.go"
    click A1 "https://github.com/douyu/jupiter/blob/master/pkg/core/application/application.go"
    click A1 "https://github.com/douyu/jupiter/blob/master/pkg/core/application/init.go"
    click A1 "https://github.com/douyu/jupiter/blob/master/pkg/core/application/option.go"
    click A1 "https://github.com/douyu/jupiter/blob/master/pkg/core/application/runtime.go"
    click Conf "https://github.com/douyu/jupiter/blob/master/pkg/conf/conf.go"
    click Conf "https://github.com/douyu/jupiter/blob/master/pkg/conf/api.go"
    click Conf "https://github.com/douyu/jupiter/tree/master/pkg/conf/datasource/"
    click Reg "https://github.com/douyu/jupiter/blob/master/pkg/registry/registry.go"
    click Reg "https://github.com/douyu/jupiter/blob/master/pkg/registry/etcdv3/registry.go"
    click Reg "https://github.com/douyu/jupiter/blob/master/pkg/registry/nop.go"
    click Reg "https://github.com/douyu/jupiter/blob/master/pkg/registry/service.go"
    click S1 "https://github.com/douyu/jupiter/blob/master/pkg/server/xecho/server.go"
    click S2 "https://github.com/douyu/jupiter/blob/master/pkg/server/xgin/server.go"
    click S3 "https://github.com/douyu/jupiter/blob/master/pkg/server/xfasthttp/server.go"
    click S4 "https://github.com/douyu/jupiter/blob/master/pkg/server/xgoframe/server.go"
    click S5 "https://github.com/douyu/jupiter/blob/master/pkg/server/xgrpc/server.go"
    click Cl1 "https://github.com/douyu/jupiter/blob/master/pkg/client/etcdv3/client.go"
    click Cl2 "https://github.com/douyu/jupiter/blob/master/pkg/client/grpc/client.go"
    click Cl3 "https://github.com/douyu/jupiter/blob/master/pkg/client/redis/client.go"
    click Cl4 "https://github.com/douyu/jupiter/blob/master/pkg/client/resty/resty.go"
    click Cl5 "https://github.com/douyu/jupiter/blob/master/pkg/client/rocketmq/producer.go"
    click Cl5 "https://github.com/douyu/jupiter/blob/master/pkg/client/rocketmq/pull_consumer.go"
    click Cache1 "https://github.com/douyu/jupiter/blob/master/pkg/cache/xfreecache/cache.go"
    click Cache2 "https://github.com/douyu/jupiter/blob/master/pkg/cache/xgolanglru/cache.go"
    click Store1 "https://github.com/douyu/jupiter/blob/master/pkg/store/gorm/instance.go"
    click Store2 "https://github.com/douyu/jupiter/blob/master/pkg/store/mongox/instance.go"
    click Store3 "https://github.com/douyu/jupiter/blob/master/pkg/store/tablestore/instance.go"
    click Exec "https://github.com/douyu/jupiter/blob/master/pkg/executor/xxl/executor.go"
    click Sent "https://github.com/douyu/jupiter/blob/master/pkg/core/sentinel/sentinel.go"
    click Met "https://github.com/douyu/jupiter/blob/master/pkg/core/metric/metric.go"
    click Trace "https://github.com/douyu/jupiter/blob/master/pkg/core/xtrace/trace.go"
    click Xlog "https://github.com/douyu/jupiter/blob/master/pkg/xlog/log.go"
    click Xlog "https://github.com/douyu/jupiter/blob/master/pkg/xlog/config.go"
    click Proto "https://github.com/douyu/jupiter/tree/master/proto/"
    click Proto "https://github.com/douyu/jupiter/blob/master/buf.gen.yaml"
    click Proto "https://github.com/douyu/jupiter/blob/master/buf.work.yaml"

    %% Styles
    classDef blue fill:#D0E8FF,stroke:#0366D6,color:#0366D6;
    classDef green fill:#E6FFED,stroke:#22863A,color:#22863A;
    classDef orange fill:#FFF5E5,stroke:#D2691E,color:#D2691E;
    classDef gray fill:#F6F8FA,stroke:#6A737D,color:#6A737D;
```
