How parthbs/crewai works
Project Type & Purpose
• CrewAI is a Python-based framework/library (not a standalone web app) for building multi-agent AI automations via two complementary paradigms—Crews (autonomous, collaborative agents) and Flows (event-driven, stateful workflows).
• It offers both a CLI (crewai commands) and a programmable SDK (src/crewai/...), adapters to LLM providers, memory systems, knowledge sources, telemetry, persistence, security, and tooling.
High-Level Architectural Components
a. User Interface / Entry Points
• CLI Layer: crewai create, crewai run, crewai install, crewai deploy
• SDK / Python Code: User-written scripts that import Crew, Flow, Agent, Task classes
b. Configuration
• YAML/TOML files for agents (agents.yaml), tasks (tasks.yaml), project (pyproject.toml), CLI config
c. Core Engine
• Crew Engine: crewai.project, crewai.crew, crewai.flow modules
• Flow Engine: decorators (@start, @listen, @router), Flow class, state management
• Event Bus: crewai.utilities.events orchestrating events between modules
• Agent Builder & Executor: builds Agent and Task instances, applies Process logic
d. Adapter Layer
• Agent Adapters: OpenAI, LangGraph, etc. (in src/crewai/agents/agent_adapters)
• Tool Adapters: wrap external tools as agent-invokable functions
• Converter Adapters: structured output converters
e. Service Layer
• LLM Module: crewai.llm, crewai.llms (base LLM + third-party connectors)
• Memory Module: contextual, short-term, long-term, external and user memories, RAG storage
• Knowledge Module: embedder, source loaders (PDF, CSV, JSON, etc.)
• Tools Module: base tools, agent tools, structured tools, caching tools
• Telemetry Module: emits usage events, metrics, integrates with observability backends
• Security Module: fingerprinting, security configs
• Persistence Layer: SQLite-based storage for Flows/tasks (crewai.flow.persistence)
f. External Dependencies & Services
• LLM providers (OpenAI, custom llms via Ollama, third-party APIs)
• Enterprise Control Plane & Observability endpoints
• External data sources (S3, databases, knowledge stores)
Component Relationships & Data Flows
• Developer → CLI / SDK → Core Engine
• CLI reads configuration, bootstraps a project (templates) or runs a crew/flow
• SDK code instantiates Crew/Flow objects directly
• Core Engine uses Event Bus to coordinate:
• Task kickoff → EventBus emits “start_kickoff” event → AgentExecutor
• AgentExecutor calls AgentBuilder → AgentAdapter → LLM Module or ToolAdapter
• Responses feed back through EventBus → Task outputs → next tasks or Flow transitions
• Memory & Knowledge Services:
• Agents pull context from Memory Module (short/long/contextual)
• Knowledge Module provides documents, embeddings which agents can query
• RAG storages persist and retrieve retrieved contexts
• Persistence & State:
• Flow state and kickoff metadata stored in SQLite via the Persistence Layer
• Telemetry & Security:
• Modules emit telemetry events to Telemetry Module
• Security Module enforces fingerprinting & config compliance
Architectural Patterns & Principles
• Layered Architecture: separate CLI/SDK, orchestration, adapters, services, external connectors
• Event-Driven: central Event Bus decouples modules, enables extensibility via listeners
• Adapter/Plugin Pattern: LLM and tools integrations via adapters keep core agnostic
• Separation of Concerns: distinct modules for memory, knowledge, tools, telemetry, security
• Open/Closed Principle: new agents, tools, LLMs added without altering core logic
Essential Details to Capture
• CLI commands and SDK entry points
• Crew and Flow orchestration layers
• Central Event Bus mediating between agents/tasks and services
• Adapter boundary for LLM and tooling integrations
• Memory, Knowledge, Persistence, Telemetry, Security service modules
• External LLM providers and observability endpoints
• Config files driving agent/task definitions