# teenyfactories/core architecture

> Architecture diagram of the GitHub repository teenyfactories/core, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-05-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/teenyfactories/core
- Repository: https://github.com/teenyfactories/core

## Overview

**1) Repository boundary**
This repo is the open-source core of TeenyFactories, a Python-first framework for building distributed AI agent “factories.” The core deliverable is the `teenyfactories` package under `python/`, plus a container image built from it (`ghcr.io/teenyfactories/agent:dev`). The orchestrator and end-user factory projects live outside this repo.

**2) Main runtime model**
The primary execution pattern is a long-running agent loop. User code registers handlers with decorators like `@tf.on_state(...).do`, then repeatedly calls `tf.run_pending()` to process work and `tf.sleep()` to idle. That means the architecture is event/state-driven rather than request/response web-based.

**3) State and collection subsystem**
`collection.py` is the core state abstraction. Collections represent named sets of items with state transitions such as `loaded` → `analysed`. Handlers react to those transitions and can update item state, which is the main internal data flow in the framework.

**4) Event dispatch and orchestration hooks**
`mcp.py` suggests integration with the Model Context Protocol or a similar agent-tooling interface. This is likely the boundary where external orchestrators or tools can interact with the agent runtime, expose capabilities, or receive state/event updates.

**5) LLM provider abstraction**
`teenyfactories/llm/` defines the model layer, with a base interface plus provider adapters for Anthropic, Azure Bedrock, Google, Ollama, and OpenAI. Architecturally, this isolates prompt/call logic from the rest of the system so agents can swap model backends without changing business logic.

**6) Embeddings and AI utility layer**
`embedding.py` provides embedding-related functionality separate from chat/completion models. This implies a second AI capability path: vectorization for retrieval, similarity, or clustering workflows, again behind a library boundary rather than embedded directly in agent code.

**7) Message queue subsystem**
`teenyfactories/message_queue/` contains a base queue abstraction with Redis and Postgres implementations. This is the main distributed-systems boundary: it likely carries jobs, state changes, or coordination signals between agent processes and/or between the orchestrator and factories.

**8) Cross-cutting support services**
`logging/logger.py`, `secrets.py`, `usage_recorder.py`, and `utils/` are supporting layers around the core execution engine. They handle observability, secret resolution, usage tracking, ID generation, and time handling, which are important for multi-agent and multi-process deployments but not the main business logic.

**9) Packaging and delivery**
The repo is built and released as a Python package via `pyproject.toml`, `setup.py`, and build scripts, and also as a container image through `python/Dockerfile.build`. GitHub Actions (`on_push.yml`, `publish-npm.yml`) indicate automated CI/CD; the presence of `nodejs/` suggests a companion npm distribution or JS entrypoint, but the implementation center of gravity is still Python.

**10) How the system fits with external components**
The README makes the architecture explicit: factories are user-owned code (`factory.yml` plus `agents/*.py`), the orchestrator discovers factories and spawns agent containers, and this repo supplies the shared runtime inside those containers. So the key architectural boundary is: orchestrator outside, TeenyFactories runtime inside each agent container, and user factory logic on top of the runtime.

**11) End-to-end data flow**
A typical flow is: external orchestrator or local loop loads a factory, agents read/update collection state, state transitions enqueue or trigger pending work, agent handlers run, and handlers may call LLM providers, embeddings, queues, or secret/config helpers before writing new state back to collections. That closed loop is the core diagram to capture.

Main entry points: [`python/teenyfactories/__init__.py`](https://github.com/teenyfactories/core/blob/main/python/teenyfactories/__init__.py) (Agent loop), [`python/teenyfactories/message_queue/base.py`](https://github.com/teenyfactories/core/blob/main/python/teenyfactories/message_queue/base.py) (Queue API), [`python/teenyfactories/collection.py`](https://github.com/teenyfactories/core/blob/main/python/teenyfactories/collection.py) (Collections), [`python/teenyfactories/llm/base.py`](https://github.com/teenyfactories/core/blob/main/python/teenyfactories/llm/base.py) (LLM API)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_external["External"]
  node_orchestrator["Orchestrator<br/>external control"]
  node_factory_code["Factory code<br/>user logic<br/>[README.md]"]
end

subgraph group_runtime["Runtime"]
  node_runtime_core["Agent loop<br/>runtime core<br/>[__init__.py]"]
  node_collection[("Collections<br/>state model<br/>[collection.py]")]
  node_mcp["MCP bridge<br/>tool boundary<br/>[mcp.py]"]
end

subgraph group_ai["AI"]
  node_llm_base["LLM API<br/>model interface<br/>[base.py]"]
  node_llm_providers["LLM providers<br/>adapters"]
  node_embedding["Embeddings<br/>vector API<br/>[embedding.py]"]
end

subgraph group_messaging["Messaging"]
  node_queue_base["Queue API<br/>queue interface<br/>[base.py]"]
  node_redis_queue["Redis queue<br/>queue backend<br/>[redis.py]"]
  node_postgres_queue["Postgres queue<br/>queue backend<br/>[postgres.py]"]
end

subgraph group_support["Support"]
  node_logging["Logging<br/>observability<br/>[logger.py]"]
  node_secrets["Secrets<br/>config resolution<br/>[secrets.py]"]
  node_usage["Usage<br/>telemetry<br/>[usage_recorder.py]"]
  node_utils["Utils<br/>shared helpers"]
end

subgraph group_delivery["Delivery"]
  node_python_pkg["Python package<br/>distribution<br/>[pyproject.toml]"]
  node_container["Agent image<br/>container build<br/>[Dockerfile.build]"]
  node_node_bridge["Node bridge<br/>js wrapper<br/>[index.js]"]
end

node_orchestrator -->|"starts"| node_runtime_core
node_factory_code -->|"uses"| node_runtime_core
node_runtime_core -->|"reads/writes"| node_collection
node_collection -->|"triggers"| node_runtime_core
node_runtime_core -->|"exposes"| node_mcp
node_runtime_core -->|"calls"| node_llm_base
node_llm_base -->|"adapts to"| node_llm_providers
node_runtime_core -->|"uses"| node_embedding
node_runtime_core -->|"enqueues"| node_queue_base
node_queue_base -->|"implements"| node_redis_queue
node_queue_base -->|"implements"| node_postgres_queue
node_runtime_core -->|"logs"| node_logging
node_runtime_core -->|"resolves"| node_secrets
node_runtime_core -->|"records"| node_usage
node_runtime_core -->|"shares"| node_utils
node_python_pkg -->|"packages"| node_runtime_core
node_container -->|"bakes in"| node_python_pkg
node_node_bridge -->|"wraps"| node_runtime_core

click node_factory_code "https://github.com/teenyfactories/core/blob/main/README.md"
click node_runtime_core "https://github.com/teenyfactories/core/blob/main/python/teenyfactories/__init__.py"
click node_collection "https://github.com/teenyfactories/core/blob/main/python/teenyfactories/collection.py"
click node_mcp "https://github.com/teenyfactories/core/blob/main/python/teenyfactories/mcp.py"
click node_llm_base "https://github.com/teenyfactories/core/blob/main/python/teenyfactories/llm/base.py"
click node_llm_providers "https://github.com/teenyfactories/core/tree/main/python/teenyfactories/llm/providers"
click node_embedding "https://github.com/teenyfactories/core/blob/main/python/teenyfactories/embedding.py"
click node_queue_base "https://github.com/teenyfactories/core/blob/main/python/teenyfactories/message_queue/base.py"
click node_redis_queue "https://github.com/teenyfactories/core/blob/main/python/teenyfactories/message_queue/providers/redis.py"
click node_postgres_queue "https://github.com/teenyfactories/core/blob/main/python/teenyfactories/message_queue/providers/postgres.py"
click node_logging "https://github.com/teenyfactories/core/blob/main/python/teenyfactories/logging/logger.py"
click node_secrets "https://github.com/teenyfactories/core/blob/main/python/teenyfactories/secrets.py"
click node_usage "https://github.com/teenyfactories/core/blob/main/python/teenyfactories/usage_recorder.py"
click node_utils "https://github.com/teenyfactories/core/tree/main/python/teenyfactories/utils"
click node_python_pkg "https://github.com/teenyfactories/core/blob/main/python/pyproject.toml"
click node_container "https://github.com/teenyfactories/core/blob/main/python/Dockerfile.build"
click node_node_bridge "https://github.com/teenyfactories/core/blob/main/nodejs/index.js"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_orchestrator,node_factory_code toneBlue
class node_runtime_core,node_collection,node_mcp toneAmber
class node_llm_base,node_llm_providers,node_embedding toneMint
class node_queue_base,node_redis_queue,node_postgres_queue toneRose
class node_logging,node_secrets,node_usage,node_utils toneIndigo
class node_python_pkg,node_container,node_node_bridge toneTeal
```

## Components

### External

Outside the core runtime boundary

- **Orchestrator**: external control · Spawns and coordinates factory containers; lives outside this repo
- [Factory code](https://github.com/teenyfactories/core/blob/main/README.md): user logic · User-owned agents and handlers layered on top of the runtime · `README.md`

### Runtime

Python agent execution and state loop

- [Agent loop](https://github.com/teenyfactories/core/blob/main/python/teenyfactories/__init__.py): runtime core · Long-running state/event loop that registers handlers and processes pending work · `python/teenyfactories/__init__.py`
- [Collections](https://github.com/teenyfactories/core/blob/main/python/teenyfactories/collection.py): state model · Named item sets and state transitions such as loaded to analysed · `python/teenyfactories/collection.py`
- [MCP bridge](https://github.com/teenyfactories/core/blob/main/python/teenyfactories/mcp.py): tool boundary · External protocol boundary for tool and orchestration interaction · `python/teenyfactories/mcp.py`

### AI

Model and embedding abstractions

- [LLM API](https://github.com/teenyfactories/core/blob/main/python/teenyfactories/llm/base.py): model interface · Provider-neutral chat and completion abstraction · `python/teenyfactories/llm/base.py`
- [LLM providers](https://github.com/teenyfactories/core/tree/main/python/teenyfactories/llm/providers): adapters · Anthropic, Bedrock, Google, Ollama, and OpenAI backends · `python/teenyfactories/llm/providers`
- [Embeddings](https://github.com/teenyfactories/core/blob/main/python/teenyfactories/embedding.py): vector API · Embedding-specific AI capability separate from chat models · `python/teenyfactories/embedding.py`

### Messaging

Distributed coordination and queue backends

- [Queue API](https://github.com/teenyfactories/core/blob/main/python/teenyfactories/message_queue/base.py): queue interface · Abstract message queue contract used by distributed runtime components · `python/teenyfactories/message_queue/base.py`
- [Redis queue](https://github.com/teenyfactories/core/blob/main/python/teenyfactories/message_queue/providers/redis.py): queue backend · Redis-backed message queue implementation · `python/teenyfactories/message_queue/providers/redis.py`
- [Postgres queue](https://github.com/teenyfactories/core/blob/main/python/teenyfactories/message_queue/providers/postgres.py): queue backend · PostgreSQL-backed message queue implementation · `python/teenyfactories/message_queue/providers/postgres.py`

### Support

Cross-cutting services

- [Logging](https://github.com/teenyfactories/core/blob/main/python/teenyfactories/logging/logger.py): observability · Structured logging and runtime diagnostics · `python/teenyfactories/logging/logger.py`
- [Secrets](https://github.com/teenyfactories/core/blob/main/python/teenyfactories/secrets.py): config resolution · Secret lookup and credential handling · `python/teenyfactories/secrets.py`
- [Usage](https://github.com/teenyfactories/core/blob/main/python/teenyfactories/usage_recorder.py): telemetry · Usage recording and accounting hooks · `python/teenyfactories/usage_recorder.py`
- [Utils](https://github.com/teenyfactories/core/tree/main/python/teenyfactories/utils): shared helpers · Common ID and time helpers used across the runtime · `python/teenyfactories/utils`

### Delivery

Packaging and build outputs

- [Python package](https://github.com/teenyfactories/core/blob/main/python/pyproject.toml): distribution · Packaged teenyfactories runtime built from the python tree · `python/pyproject.toml`
- [Agent image](https://github.com/teenyfactories/core/blob/main/python/Dockerfile.build): container build · Docker image used to ship the runtime to agent containers · `python/Dockerfile.build`
- [Node bridge](https://github.com/teenyfactories/core/blob/main/nodejs/index.js): js wrapper · Companion Node.js entrypoint and package distribution · `nodejs/index.js`

## Connections

- Orchestrator → Agent loop: starts · External orchestrator launches agent containers running the runtime
- Factory code → Agent loop: uses · User handlers and agents run on top of the runtime loop
- Agent loop → Collections: reads/writes · Main work loop advances collection state and dispatches handlers
- Collections → Agent loop: triggers · State transitions drive pending work and handler execution
- Agent loop → MCP bridge: exposes · Runtime can surface capabilities through the protocol boundary
- Agent loop → LLM API: calls · Handlers invoke provider-neutral model operations
- LLM API → LLM providers: adapts to · Base model interface is implemented by provider-specific backends
- Agent loop → Embeddings: uses · Embedding workflows are available alongside chat/completion calls
- Agent loop → Queue API: enqueues · Distributed coordination and work delivery go through the queue abstraction
- Queue API → Redis queue: implements · Redis backend fulfills the queue contract
- Queue API → Postgres queue: implements · Postgres backend fulfills the queue contract
- Agent loop → Logging: logs · Execution and state changes are observable through the logging layer
- Agent loop → Secrets: resolves · Handlers and providers can fetch credentials and secret values
- Agent loop → Usage: records · Runtime activity can be tracked for accounting or telemetry
- Agent loop → Utils: shares · Common ID and time helpers support runtime operations
- Python package → Agent loop: packages · The Python distribution packages the core runtime modules
- Agent image → Python package: bakes in · Container image builds the packaged Python runtime into a deployable agent image
- Node bridge → Agent loop: wraps · Node package provides a companion entrypoint around the core runtime
