How teenyfactories/core works
1) Repository boundary
This repo is the open-source core of TeenyFactories, a Python-first framework for building distributed AI agent “factories.” The core deliverable is the teenyfactories package under python/, plus a container image built from it (ghcr.io/teenyfactories/agent:dev). The orchestrator and end-user factory projects live outside this repo.
2) Main runtime model
The primary execution pattern is a long-running agent loop. User code registers handlers with decorators like @tf.on_state(...).do, then repeatedly calls tf.run_pending() to process work and tf.sleep() to idle. That means the architecture is event/state-driven rather than request/response web-based.
3) State and collection subsystem
collection.py is the core state abstraction. Collections represent named sets of items with state transitions such as loaded → analysed. Handlers react to those transitions and can update item state, which is the main internal data flow in the framework.
4) Event dispatch and orchestration hooks
mcp.py suggests integration with the Model Context Protocol or a similar agent-tooling interface. This is likely the boundary where external orchestrators or tools can interact with the agent runtime, expose capabilities, or receive state/event updates.
5) LLM provider abstraction
teenyfactories/llm/ defines the model layer, with a base interface plus provider adapters for Anthropic, Azure Bedrock, Google, Ollama, and OpenAI. Architecturally, this isolates prompt/call logic from the rest of the system so agents can swap model backends without changing business logic.
6) Embeddings and AI utility layer
embedding.py provides embedding-related functionality separate from chat/completion models. This implies a second AI capability path: vectorization for retrieval, similarity, or clustering workflows, again behind a library boundary rather than embedded directly in agent code.
7) Message queue subsystem
teenyfactories/message_queue/ contains a base queue abstraction with Redis and Postgres implementations. This is the main distributed-systems boundary: it likely carries jobs, state changes, or coordination signals between agent processes and/or between the orchestrator and factories.
8) Cross-cutting support services
logging/logger.py, secrets.py, usage_recorder.py, and utils/ are supporting layers around the core execution engine. They handle observability, secret resolution, usage tracking, ID generation, and time handling, which are important for multi-agent and multi-process deployments but not the main business logic.
9) Packaging and delivery
The repo is built and released as a Python package via pyproject.toml, setup.py, and build scripts, and also as a container image through python/Dockerfile.build. GitHub Actions (on_push.yml, publish-npm.yml) indicate automated CI/CD; the presence of nodejs/ suggests a companion npm distribution or JS entrypoint, but the implementation center of gravity is still Python.
10) How the system fits with external components
The README makes the architecture explicit: factories are user-owned code (factory.yml plus agents/*.py), the orchestrator discovers factories and spawns agent containers, and this repo supplies the shared runtime inside those containers. So the key architectural boundary is: orchestrator outside, TeenyFactories runtime inside each agent container, and user factory logic on top of the runtime.
11) End-to-end data flow
A typical flow is: external orchestrator or local loop loads a factory, agents read/update collection state, state transitions enqueue or trigger pending work, agent handlers run, and handlers may call LLM providers, embeddings, queues, or secret/config helpers before writing new state back to collections. That closed loop is the core diagram to capture.
Main entry points: __init__.py Agent loop · message_queue/base.py Queue API · collection.py Collections · llm/base.py LLM API
Components 18
External
Outside the core runtime boundary
- Orchestrator · Spawns and coordinates factory containers; lives outside this repo
- Factory code · User-owned agents and handlers layered on top of the runtime
README.md
Runtime
Python agent execution and state loop
- Agent loop · Long-running state/event loop that registers handlers and processes pending work
python/teenyfactories/__init__.py - Collections · Named item sets and state transitions such as loaded to analysed
python/teenyfactories/collection.py - MCP bridge · External protocol boundary for tool and orchestration interaction
python/teenyfactories/mcp.py
AI
Model and embedding abstractions
- LLM API · Provider-neutral chat and completion abstraction
python/teenyfactories/llm/base.py - LLM providers · Anthropic, Bedrock, Google, Ollama, and OpenAI backends
python/teenyfactories/llm/providers - Embeddings · Embedding-specific AI capability separate from chat models
python/teenyfactories/embedding.py
Messaging
Distributed coordination and queue backends
- Queue API · Abstract message queue contract used by distributed runtime components
python/teenyfactories/message_queue/base.py - Redis queue · Redis-backed message queue implementation
python/teenyfactories/message_queue/providers/redis.py - Postgres queue · PostgreSQL-backed message queue implementation
python/teenyfactories/message_queue/providers/postgres.py
Support
Cross-cutting services
- Logging · Structured logging and runtime diagnostics
python/teenyfactories/logging/logger.py - Secrets · Secret lookup and credential handling
python/teenyfactories/secrets.py - Usage · Usage recording and accounting hooks
python/teenyfactories/usage_recorder.py - Utils · Common ID and time helpers used across the runtime
python/teenyfactories/utils
Delivery
Packaging and build outputs
- Python package · Packaged teenyfactories runtime built from the python tree
python/pyproject.toml - Agent image · Docker image used to ship the runtime to agent containers
python/Dockerfile.build - Node bridge · Companion Node.js entrypoint and package distribution
nodejs/index.js
Connections 18
- Orchestrator → Agent loop: starts · External orchestrator launches agent containers running the runtime
- Factory code → Agent loop: uses · User handlers and agents run on top of the runtime loop
- Agent loop → Collections: reads/writes · Main work loop advances collection state and dispatches handlers
- Collections → Agent loop: triggers · State transitions drive pending work and handler execution
- Agent loop → MCP bridge: exposes · Runtime can surface capabilities through the protocol boundary
- Agent loop → LLM API: calls · Handlers invoke provider-neutral model operations
- LLM API → LLM providers: adapts to · Base model interface is implemented by provider-specific backends
- Agent loop → Embeddings: uses · Embedding workflows are available alongside chat/completion calls
- Agent loop → Queue API: enqueues · Distributed coordination and work delivery go through the queue abstraction
- Queue API → Redis queue: implements · Redis backend fulfills the queue contract
- Queue API → Postgres queue: implements · Postgres backend fulfills the queue contract
- Agent loop → Logging: logs · Execution and state changes are observable through the logging layer
- Agent loop → Secrets: resolves · Handlers and providers can fetch credentials and secret values
- Agent loop → Usage: records · Runtime activity can be tracked for accounting or telemetry
- Agent loop → Utils: shares · Common ID and time helpers support runtime operations
- Python package → Agent loop: packages · The Python distribution packages the core runtime modules
- Agent image → Python package: bakes in · Container image builds the packaged Python runtime into a deployable agent image
- Node bridge → Agent loop: wraps · Node package provides a companion entrypoint around the core runtime