How samuelint/langchain-openai-api-bridge works
Project Overview & Purpose
• This project is a FastAPI extension designed to expose Langchain (and Langgraph) agents as an OpenAI-compatible API. It supports two major API types: the Assistant API and the Chat Completions API.
• The project is tailored to integrate multiple LLM vendors (such as OpenAI, Anthropic, Groq, LLamaCPP) and serves requests from external OpenAI-compatible clients.
Main Components/Modules
a. API Gateway (FastAPI Integration):
• The FastAPI application is the entry point where API endpoints are registered. The “fastapi” directory and bindings in modules like langchain_openai_api_bridge/fastapi/langchain_openai_api_bridge_fastapi.py are the core parts.
• Two API routes exist – one tied to the Assistant API (with functionalities such as threads, messages, run stream, tool event streams) and one for Chat Completion.
b. Assistant and Chat Completion Modules:
• The “assistant” directory encapsulates functionalities that support the OpenAI Assistant API. It includes multiple adapters, message services, thread services, DTOs for agent creation, and repository integrations.
• The “chat_completion” directory handles the components necessary for exposing the Chat Completions API. This includes adapter modules (factories, object factories, stream adapters) that process chat response content.
c. Chat Model Adapters:
• The “chat_model_adapter” directory contains vendor-specific adapters (e.g. for Anthropic and LLamaCPP). These adapters provide compatibility layers so that the agents can mimic OpenAI chat models regardless of the underlying vendor.
d. Core Modules:
• The “core” directory holds components that manage agent creation, agent factories (base, function-based, and role adaptors), and support utilities (like dependency injection container and async iterator utilities). This area provides the backbone for orchestrating the bridge between Langchain agents and OpenAI API calls.
e. Repositories and Data Persistence:
• Inside both the assistant sub-modules and in separate repository directories, there are in-memory implementations for managing messages, threads, and run histories. These repositories offer a stateless or ephemeral data handling for API interactions.
f. Testing and Examples:
• The tests directory provides functional and unit tests showing practical integration scenarios, with sample agent factories and server implementations (e.g., for Anthropic, OpenAI, Groq, and LLamaCPP). This demonstrates real-world usage and integration points for both API types.
Relationships and Interactions
• The FastAPI entry point binds the bridge (e.g., LangchainOpenaiApiBridgeFastAPI) to the API endpoints. This component internally calls agent factories (user provided or default) to create Langchain agents.
• Depending on the endpoint (Assistant or Chat Completion), the request flows through adapter layers (e.g., from langchain inputs to OpenAI message formats) and then is handled by the respective services (like assistant_run_service or chat_completion services).
• Repositories (in-memory for this project) maintain conversation histories, thread details, and run metadata that can be used for further processing or later retrieval.
• Vendor-specific adapters (found in chat_model_adapter/ and within assistant/adapter) are responsible for translating requests to the target vendor’s format, ensuring compatibility and proper handling of streaming or multimodal responses.
Architectural Patterns & Design Principles
• Separation of Concerns: Organized directories (assistant, chat_completion, core, fastapi) denote clear functional boundaries.
• Adapter/Facade Pattern: Many “adapter” modules convert inputs, messages, and events between Langchain/Langgraph internal representations and OpenAI API-compatible structures.
• Extensibility: By supporting multiple agent factories and vendor-specific adapters, users can plug in custom logic for different LLM vendors.
• Dependency Injection: The usage of providers (e.g., agent_factory_provider in the FastAPI bridge and in-memory repository providers) supports loose coupling and testability.
Technologies & Frameworks
• Python language with FastAPI framework as the web server.
• Langchain and Langgraph providing agent/LLM orchestration.
• In-memory repository implementations used for prototyping; extensibility is available for production-grade stores.
• Testing is managed using pytest.
• The project also involves Docker/Poetry for dependency management and CI/CD pipelines as seen in GitHub workflows.