How googleapis/genai-toolbox works
Project Type and Purpose
• This is an open-source MCP (Multi-Channel Protocol) server for databases (MCP Toolbox).
• It provides a centralized control plane to define, manage, and invoke “tools” (SQL operations, commands) against multiple database sources, handling connection pooling, authentication, telemetry, and dynamic reloading.
• Consumers include: CLI (cmd/), container image, and SDK clients (Python, JS, etc.), which load toolsets via HTTP/JSON-RPC and invoke database operations.
File Structure Analysis
• Top-level:
• .ci/, .github/: CI/CD and repo automation
• cmd/: CLI entrypoints (options.go, root.go)
• internal/: core packages
– server/: config loader, MCP API handlers, JSON-RPC implementations, lifecycle
– sources/: adapters for each DB type (Postgres, MySQL, BigQuery, Redis, etc.)
– tools/: implementations of tool kinds (execute SQL, list tables, etc.)
– auth/, log/, telemetry/: cross-cutting concerns
– prebuiltconfigs/: YAML templates for common sources/tools
– util/, testutils/: helpers
• main.go: bootstrap
• docs/, .hugo/: documentation site
• Configuration via a tools.yaml (not in repo) read by server to define sources, tools, toolsets.
• Testing: unit tests under each package, integration tests under tests/.
README Insights
Client ⟷ MCP Toolbox server ⟷ Databases
• Key features: dynamic reloading, built-in auth, OpenTelemetry tracing/metrics, plugin-style adapters.
a. Main Components
1. Clients
• CLI (cmd/root)
• Container image
• SDKs (Python, JS, Genkit, LangChain)
2. MCP Toolbox Server
• Config Loader (parses tools.yaml, prebuiltconfigs)
• Tool Registry (in-memory store of tool definitions & toolsets)
• MCP API Layer
• HTTP server / JSON-RPC handler (internal/server/api.go, jsonrpc/)
• Auth Module (internal/auth)
• Telemetry & Logging (internal/telemetry, internal/log)
• Source Adapter Manager (loads appropriate adapter from internal/sources/*)
• Execution Engine / Connection Pool (internal/sources/*, database/sql under the hood)
3. External Data Sources
• Relational DBs: PostgreSQL, MySQL, MSSQL, SQLite
• Cloud DBs: BigQuery, Spanner, Bigtable
• NoSQL: Redis, Couchbase, Neo4j, Dgraph
• HTTP source, Valkey, etc.
b. Relationships & Data Flow
1. Client sends “load toolset” or “invoke tool” request over HTTP/JSON-RPC to MCP server.
2. MCP API Layer authenticates via Auth Module, logs request.
3. API Layer looks up the Tool definition in Tool Registry.
4. Execution Engine fetches a connection from Connection Pool for the configured Source.
5. Executes statement via Source Adapter, collects results.
6. Telemetry Module emits metrics/traces; Logging Module records events.
7. API Layer returns response to Client.
c. Architectural Patterns & Principles
• Plugin / Adapter Pattern for database sources and tool implementations
• Control-Plane / Data-Plane Separation:
• Control plane: config loader, registry, dynamic reloading
• Data plane: query execution, connection pooling
• MCP (JSON-RPC) Protocol layer abstraction
• Dependency Injection via interfaces (for sources, tools, telemetry)
d. Key Technologies & Frameworks
• Go modules (go.mod)
• Standard library: net/http, database/sql
• OpenTelemetry SDK
• JSON-RPC implementation under internal/server/mcp/jsonrpc
• Hugo for docs site
• GitHub Actions & Cloud Build for CI/CD