How douyu/jupiter works
Project type and purpose
• This is JUPITER: a governance-oriented microservice framework written in Go. It provides:
• A CLI toolkit (cmd/jupiter) and code generators for gRPC, HTTP (Gin, Echo), error handling, field-masking.
• Core libraries for configuration, service registration/discovery, error codes, metrics, tracing, circuit-breaking (Sentinel), logging, lifecycle hooks, and more.
• Pluggable implementations for cache (Freecache, LRU), clients (Etcd, gRPC, Redis, RocketMQ, REST), data stores (GORM/MySQL, MongoX, TableStore), executors (XXL-Job), and server transports (xecho, xgin, xfasthttp, xgrpc, xgoframe).
• A Proto schema directory and Buf config for code generation.
File structure
• Top-level “cmd/…” contains CLI and codegen tools.
• “core/…” contains foundational abstractions (Application bootstrap, component wiring, ecode, metric, tracing, hooks, signals).
• “pkg/…” contains pluggable modules in categories:
• cache, client, conf, executor, registry, server, store, util, worker, xlog
• “proto/…” holds protobuf definitions and generation config.
• “test/…” holds integration/test fixtures including Docker Compose.
• “tools/…” and “website/…” support documentation and auxiliary tooling.
README for additional insights
• Macro-purpose: developer-facing microservice framework with governance features.
• External systems expected: Docker, gRPC clients, Redis, Etcd, SQL/Mongo, RocketMQ, Sentinel, Jaeger, Prometheus.
• Companion projects: Juno (console), jupiter-layout (template).
Main components & interactions (to show
a. CLI & Codegen Layer
– cmd/jupiter: “jupiter new”, “jupiter run” for scaffolding & runtime injection
– cmd/protoc-gen-go-* tools: generate stubs for HTTP frameworks & error/fieldmask code
b. Application Bootstrap
– core/application: loads configuration (pkg/conf), sets up logging (pkg/xlog), metrics, tracing (core/xtrace), error codes, hooks, and run loop.
c. Configuration
– pkg/conf: unified config API with multiple datasource plugins (file, Apollo, Etcd, HTTP)
d. Service Registration & Discovery
– pkg/registry: register service instances to Etcd (or nop) and resolve endpoints
e. Server Layer
– pkg/server/xecho, xgin, xfasthttp, xgrpc, xgoframe: transport adapters that plug into Application
f. Client Layer
– pkg/client/{grpc, redis, etcdv3, resty, rocketmq}: abstracted clients with interceptors, load-balancing, retries
g. Caching & Data Stores
– pkg/cache/…: in-process cache (freecache, LRU)
– pkg/store/{gorm, mongox, tablestore}: DB ORM/driver wrappers
h. Messaging & Task Execution
– pkg/executor/xxl: wrapper for XXL-Job tasks
– pkg/client/rocketmq: producer/consumer abstractions
i. Governance & Resilience
– core/sentinel: circuit-breaker, rate-limiter, system protection
– core/metric & external scraping (Prometheus)
– core/xtrace: distributed tracing via Jaeger or OTLP
j. Utilities
– pkg/util, pkg/worker, pkg/xlog: support routines (retry, reflection, cron jobs, logging rotation)
Relationships & data flow
• The Application bootstrap invokes:
• Configuration → Service Registration → Logging/Tracing → Metric initialization → Server startup
• Servers expose endpoints → incoming requests pass through middleware (Sentinel, Logging, Tracing) → business handlers
• Handlers call downstream Clients (gRPC, REST, Redis, RocketMQ), Data Stores, or Executors
• Outbound calls go through interceptors (retry, load-balancing, tracing).
• Results flow back through the same middleware in reverse.
Architectural patterns & design principles
• Layered architecture: CLI / Bootstrap → Core framework → Pluggable modules → External systems
• Dependency inversion: high-level modules depend on abstractions in “core” and “pkg” interfaces
• Plugin pattern: pkg/conf, pkg/registry, pkg/cache, pkg/store all expose interface + registerable implementations
• Code generation: proto → transport adapters / error wrappers to reduce boilerplate
• Governance cross-cutting concerns: tracing, metrics, error codes, circuit-breaking reside in core and injected via middleware
Elements
• Clear component labels (e.g., “pkg/server/xgrpc”, “core/sentinel”)
• Initialization sequence
• Request/response flow
• Outbound call paths
• CLI tools vs generated code
• Core abstractions vs pluggable modules
• External systems
Final checklist for detail
• Major plugin categories (conf, registry, server, client, cache, store, executor)
• Governance aspects (Sentinel, metric, trace) as cross-cutting layers