How jackmpcollins/magentic works
Project Type and Purpose
• magentic is an open-source Python library/tool (not a web app) that provides decorators (@prompt, @chatprompt, @prompt_chain, etc.) to let Python code seamlessly orchestrate LLM queries, structured outputs (via Pydantic), streaming, and function calls.
• It targets developers who want to embed LLM-driven logic into their Python applications, supporting multiple LLM backends (OpenAI, Anthropic, LiteLLM, Mistral), observability (OpenTelemetry), retries, and more.
High-Level File Structure Analysis
• src/magentic: core library code, subpackages for decorators, parsing, streaming, formatting, function-calling, logging, settings, and the chat_model interface plus plug‐in backends.
• docs + mkdocs.yml: static documentation site assets and content.
• tests: test harness using VCR-style cassettes to mock LLM responses, unit tests for core modules.
• CI/CD configs (.github/workflows): build, test, publish to PyPI, and deploy documentation.
Main Components
a. User Code Layer
• Python scripts/modules that import and use @prompt, @chatprompt, @prompt_chain decorators.
b. Magentic Core Library
• Decorators & runtime:
• _chat.py, _parsing.py, _pydantic.py, _streamed_response.py
• prompt_chain.py, chatprompt.py, prompt_function.py, function_call.py
• Formatting & utilities: formatting.py, logger.py, streaming.py, typing.py
• Configuration & settings: settings.py, environment var loader
c. ChatModel Interface & Backends
• Base class (chat_model/base.py)
• Backend implementations: openai_chat_model.py, anthropic_chat_model.py, litellm_chat_model.py, mistral_chat_model.py, plus retry_chat_model.py wrapper.
d. External LLM Providers
• HTTP APIs: OpenAI, Anthropic, LiteLLM, Mistral endpoints (could be local or cloud)
e. Function Execution Engine
• Drives FunctionCall objects: discovers candidate Python functions, invokes them, marshals I/O back into the LLM loop (prompt chains).
f. Observability & Retry Subsystem
• OpenTelemetry instrumentation, Pydantic Logfire integration, retry logic for schema adherence.
g. Documentation Pipeline
• MkDocs site generator (docs/ + mkdocs.yml)
• GitHub Actions job “deploy_docs.yml” to push to gh-pages.
h. CI/CD Pipeline
• GitHub Actions:
• test.yml → runs pytest with recorded cassettes
• publish.yml → builds & publishes PyPI package
• deploy_docs.yml → builds and deploys docs.
Relationships and Interactions
• User Code → Magentic Decorators: calls to decorated functions hand off to Magentic runtime.
• Runtime → ChatModel (interface): runtime formulates a Chat API request (messages, function schemas, streaming flags).
• ChatModel Backend → External API: sends HTTP request, receives streamed or full response.
• Response → Magentic Parsers: streams text or objects, including detecting FunctionCall JSON schemas.
• FunctionCall → Function Execution Engine: if LLM requests a tool call, Magentic invokes the user’s Python function, then feeds the result back into the prompt chain.
• Observability: each step emits OpenTelemetry spans/logs.
• Configuration: environment variables or decorator arguments override default ChatModel selection (settings module orchestrates).
• CI/CD: GitHub Actions orchestrates testing → publishing → docs deployment.
Architectural Patterns & Design Principles
• Plugin Architecture: ChatModel backends adhere to a common interface; new backends can be dropped in.
• Decorator Pattern: user-facing functions are no-ops rerouted via decorators to Magentic logic.
• Chain of Responsibility: prompt chains and function calls form a pipeline until final answer.
• Dependency Injection: ChatModel instance can be passed explicitly, via context manager, or via environment configuration.
• Streaming & Async Patterns: supports both synchronous and asynchronous generations, streaming outputs, and concurrent async tasks.
Technologies, Frameworks, Libraries
• Python 3.x, Pydantic for schema enforcement, OpenTelemetry for tracing, Pytest + VCR cassettes for testing, MkDocs for docs, GitHub Actions for CI/CD, pip/uv for distribution.