How aws/smithy-go works
Project type and purpose
• This is an open‑source tool that combines:
• A code generator (Java‑based Smithy build plugins under codegen/smithy‑go‑codegen)
• A Go runtime library (all other Go packages: auth, transport, middleware, document, encoding, endpoints, metrics, tracing, etc.)
• Purpose: Given a Smithy model and protocol definition, generate Go client (and server) code, and support it with a rich runtime for HTTP transport, serialization, authentication, metrics, tracing, error handling, and more.
File structure
• Top‑level “codegen” directory: a Gradle/Java plugin project implementing two Smithy build plugins (go-codegen and go-server-codegen), including template resources and Java source/test.
• Root Go modules (go.mod, go.sum) and directories for the runtime:
• auth, aws-http-auth – credential providers and request signing (SigV4, bearer tokens)
• transport/http – HTTP client, middleware for headers, checksums, content-length, logging
• middleware – HTTP request/response pipeline abstraction and built‑in steps
• document, encoding – JSON, XML, CBOR, HTTP‑binding serializers/deserializers
• endpoints – endpoint resolution rules and data
• metrics, tracing – OpenTelemetry integrations
• other utilities: context, container/cache, io, logging, rand, ptr, sync, time, validation, waiter, testing
• Build/config files: Makefile, .github workflows, Travis, CHANGELOG, CONTRIBUTING, etc.
README for additional insights
• The README describes:
• Two Smithy build plugins: go-codegen and go-server-codegen
• Plugin configuration keys (service, module, generateGoMod, goDirective)
• Supported protocols (currently rpcv2Cbor; AWS protocols forthcoming)
• Minimum Go version and warning that interfaces are unstable
a. Main components
1. Smithy CLI & Build System
– Inputs: .smithy model files + smithy-build.json + plugin JARs
– Java plugin: codegen/smithy-go-codegen
– Output: generated Go source code (client, server stubs)
2. Generated Go code
– Model-specific client API, service operations, types
3. smithy-go runtime library
– Core layers:
• transport layer (HTTP client, request/response)
• middleware pipeline (initialize, serialize, build, finalize, deserialize steps)
• auth (SigV4, bearer token, identity resolution)
• document & encoding (serde for JSON, XML, CBOR, HTTP bindings)
• endpoints (rules engine lookup)
• metrics & tracing (OpenTelemetry adapters)
• utilities (context propagation, caching, pointers, sync, validation, waiters)
4. External dependencies/services
– HTTP endpoints (AWS, other services)
– OpenTelemetry collector (for metrics/traces)
b. Relationships and interactions
• Smithy CLI invokes Java plugin → reads model → produces Go code + templates
• Generated Go code imports and uses the smithy‑go runtime packages
• At runtime: client call → construct middleware stack → serialize input (document/encoding) → apply auth → sign HTTP request → send via transport/http → receive response → deserialize → run response middleware → return output or error
• Metrics/tracing layers wrap around transport/middleware pipeline and emit telemetry
c. Architectural patterns and design principles
• Plugin architecture (Smithy build plugins for extensibility)
• Layered architecture (separate concerns: generation, transport, middleware, serde, auth)
• Middleware pipeline (chain‑of‑responsibility for request/response processing)
• Code‑generation + runtime separation (strongly decoupled)
• Rule‑driven endpoint resolution (data + function code in endpoints/private)
d. Key technologies, frameworks, libraries
• Smithy CLI and Smithy build plugins (Java/Gradle)
• Go 1.20+
• Templates in codegen Java plugin (MUSTACHE or text templates under resources)
• Go modules (go.mod)
• OpenTelemetry (smithyotelmetrics, smithyoteltracing)
• Go standard library (net/http, context, encoding/json, xml, sync)
Detail
• Break the runtime into as many logical sub‑components as possible (auth, transport, middleware, serde, endpoints, metrics, tracing, utilities).
• Plugin internals at a high level: source → templates → output.