How brbarmex/ffi-foreign-function-interface works
Project Type and Purpose
• This is a language-interoperability proof-of-concept (POC) rather than a classic full-stack app.
• Its goal is to provide a single Go-based SDK (“cloudsdk”) that can be reused from C#, Java, Python (and any other FFI-capable language) to consume cloud message queues (AWS SQS or Azure Service Bus) with low latency, agnostic to provider, and using Protobuf for serialization.
High-Level Components
a. Go Core Module (“cloudsdk/”)
• main.go – entry point when building a standalone Go binary.
• queue/queue.go – abstraction over AWS/Azure queue consumption, uses the “cloudsdk_provider” env var to pick a strategy.
• protos/ – Protobuf definitions plus generated code for Go, C#, Java, Python.
b. FFI Layer
• cgo glue (Go //export functions in main.go or a dedicated .c file) that exposes Go functions as C symbols.
• A small C stub (in Go’s build process) to expose C ABI functions.
c. Language Bindings
• csharp/ – C# wrapper project that P/Invokes into the native library.
• java/ – Java code that uses JNI or JNA to load the shared library and call into it.
• (Python bindings can be added similarly using ctypes or cffi.)
d. External Services
• AWS SQS
• Azure Service Bus
e. Build & Deployment
• Makefile – orchestrates go build -buildmode=c-shared, Protobuf code generation, and packaging.
• go.mod – module dependencies.
Relationships & Data Flow
1. Language Client (C#/Java/Python)
→ loads the native “cloudsdk” shared library via FFI
→ calls GetEvents() or similar exported function
→ receives a serialized Protobuf byte array
→ uses the generated Protobuf classes in that language to deserialize into a strongly-typed message list
2. FFI Layer
→ marshals calling conventions between the host runtime (e.g., .NET, JVM) and Go’s C ABI
→ passes control & arguments into the Go Core
3. Go Core Module
→ reads “cloudsdk_provider” env var
→ selects AWS or Azure strategy implementation from queue/queue.go
→ fetches messages from the cloud queue
→ serializes them into Protobuf bytes
→ returns the byte buffer to the caller via the C ABI
4. External Cloud Service
← SDK issues HTTP/gRPC/SDK calls to AWS SQS or Azure Service Bus
← receives events
Key Architectural Patterns & Principles
• Strategy Pattern: for selecting the cloud provider at runtime based on an environment variable.
• Layered Architecture:
• Presentation Layer (language-specific client code)
• Integration Layer (FFI boundary + Protobuf contracts)
• Core SDK Layer (Go module implementing business logic)
• Infrastructure Layer (cloud message queues)
• Single Source of Truth: one Go codebase driving all language bindings.
• Tech Choices: Go + c-shared (cgo), Protobuf, P/Invoke (C#), JNI/JNA (Java), ctypes/cffi (Python).