# parthbs/crewai architecture

> Architecture diagram of the GitHub repository parthbs/crewai, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/parthbs/crewai
- Repository: https://github.com/parthbs/crewai

## Overview

##### Project Type & Purpose
- CrewAI is a Python-based framework/library (not a standalone web app) for building multi-agent AI automations via two complementary paradigms—Crews (autonomous, collaborative agents) and Flows (event-driven, stateful workflows).
- It offers both a CLI (`crewai` commands) and a programmable SDK (`src/crewai/...`), adapters to LLM providers, memory systems, knowledge sources, telemetry, persistence, security, and tooling.

##### High-Level Architectural Components
a. User Interface / Entry Points
- CLI Layer: `crewai create`, `crewai run`, `crewai install`, `crewai deploy`
- SDK / Python Code: User-written scripts that import Crew, Flow, Agent, Task classes
b. Configuration
- YAML/TOML files for agents (`agents.yaml`), tasks (`tasks.yaml`), project (`pyproject.toml`), CLI config
c. Core Engine
- Crew Engine: `crewai.project`, `crewai.crew`, `crewai.flow` modules
- Flow Engine: decorators (`@start`, `@listen`, `@router`), `Flow` class, state management
- Event Bus: `crewai.utilities.events` orchestrating events between modules
- Agent Builder & Executor: builds `Agent` and `Task` instances, applies `Process` logic
d. Adapter Layer
- Agent Adapters: OpenAI, LangGraph, etc. (in `src/crewai/agents/agent_adapters`)
- Tool Adapters: wrap external tools as agent-invokable functions
- Converter Adapters: structured output converters
e. Service Layer
- LLM Module: `crewai.llm`, `crewai.llms` (base LLM + third-party connectors)
- Memory Module: contextual, short-term, long-term, external and user memories, RAG storage
- Knowledge Module: embedder, source loaders (PDF, CSV, JSON, etc.)
- Tools Module: base tools, agent tools, structured tools, caching tools
- Telemetry Module: emits usage events, metrics, integrates with observability backends
- Security Module: fingerprinting, security configs
- Persistence Layer: SQLite-based storage for Flows/tasks (`crewai.flow.persistence`)
f. External Dependencies & Services
- LLM providers (OpenAI, custom llms via Ollama, third-party APIs)
- Enterprise Control Plane & Observability endpoints
- External data sources (S3, databases, knowledge stores)

##### Component Relationships & Data Flows
- Developer → CLI / SDK → Core Engine
- CLI reads configuration, bootstraps a project (templates) or runs a crew/flow
- SDK code instantiates Crew/Flow objects directly
- Core Engine uses Event Bus to coordinate:
- Task kickoff → EventBus emits “start_kickoff” event → AgentExecutor
- AgentExecutor calls AgentBuilder → AgentAdapter → LLM Module or ToolAdapter
- Responses feed back through EventBus → Task outputs → next tasks or Flow transitions
- Memory & Knowledge Services:
- Agents pull context from Memory Module (short/long/contextual)
- Knowledge Module provides documents, embeddings which agents can query
- RAG storages persist and retrieve retrieved contexts
- Persistence & State:
- Flow state and kickoff metadata stored in SQLite via the Persistence Layer
- Telemetry & Security:
- Modules emit telemetry events to Telemetry Module
- Security Module enforces fingerprinting & config compliance

##### Architectural Patterns & Principles
- Layered Architecture: separate CLI/SDK, orchestration, adapters, services, external connectors
- Event-Driven: central Event Bus decouples modules, enables extensibility via listeners
- Adapter/Plugin Pattern: LLM and tools integrations via adapters keep core agnostic
- Separation of Concerns: distinct modules for memory, knowledge, tools, telemetry, security
- Open/Closed Principle: new agents, tools, LLMs added without altering core logic

##### Essential Details to Capture
- CLI commands and SDK entry points
- Crew and Flow orchestration layers
- Central Event Bus mediating between agents/tasks and services
- Adapter boundary for LLM and tooling integrations
- Memory, Knowledge, Persistence, Telemetry, Security service modules
- External LLM providers and observability endpoints
- Config files driving agent/task definitions

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Swimlane: User Interface
    subgraph "User Interface" 
        direction TB
        DEV["Developer"]:::external
        CLI["CLI Layer"]:::ui
        SDK["SDK Entry Points"]:::ui
        DEV -->|uses| CLI
        DEV -->|imports| SDK
    end

    %% Swimlane: Configuration
    subgraph "Configuration Files" 
        direction TB
        CrewConfig["Crew Config YAML/TOML"]:::config
        FlowConfig["Flow Config YAML/TOML"]:::config
        ProjectConfig["Project Config (pyproject.toml)"]:::config
    end

    %% Swimlane: Core Engine
    subgraph "Core Engine" 
        direction TB
        EventBus[(Event Bus)]:::core
        CrewEngine["Crew Engine"]:::core
        FlowEngine["Flow Engine"]:::core
        AgentExec["Agent Builder & Executor"]:::core
        CLI -->|read config / bootstrap| CrewEngine
        CLI -->|read config / bootstrap| FlowEngine
        SDK -->|instantiates| CrewEngine
        SDK -->|instantiates| FlowEngine
        CrewEngine -->|emit_event| EventBus
        FlowEngine -->|emit_event| EventBus
        EventBus -->|start_kickoff| AgentExec
    end

    %% Swimlane: Adapter Layer
    subgraph "Adapter Layer" 
        direction TB
        AgentAdapters["Agent Adapters"]:::adapter
        ToolAdapters["Tool & Converter Adapters"]:::adapter
        AgentExec -->|uses adapter| AgentAdapters
        AgentExec -->|uses adapter| ToolAdapters
    end

    %% Swimlane: Service Layer
    subgraph "Service Layer" 
        direction TB
        LLMModule["LLM Module"]:::service
        MemoryModule["Memory Module"]:::service
        KnowledgeModule["Knowledge Module"]:::service
        ToolsModule["Tools Module"]:::service
        Persistence["Persistence Layer (SQLite)"]:::service
        Telemetry["Telemetry Module"]:::service
        Security["Security Module"]:::service

        AgentAdapters -->|call_llm| LLMModule
        ToolAdapters -->|invoke_tool| ToolsModule
        AgentExec -->|query_memory| MemoryModule
        AgentExec -->|query_knowledge| KnowledgeModule
        FlowEngine -->|persist_state| Persistence
        EventBus -->|emit telemetry| Telemetry
        EventBus -->|check security| Security
    end

    %% Swimlane: External Systems
    subgraph "External Systems" 
        direction TB
        LLMProviders[("LLM Providers\n(OpenAI, Ollama)")]:::external
        Observability[("Observability & Control Plane")]:::external
        DataSources[("External Data Sources\n(S3, DBs)")]:::external

        LLMModule -->|API calls| LLMProviders
        Telemetry -->|send metrics| Observability
        KnowledgeModule -->|fetch docs| DataSources
        MemoryModule -->|store/retrieve| DataSources
    end

    %% Configuration connections
    CrewConfig -->|defines agents/tasks| CLI
    FlowConfig -->|defines agents/tasks| CLI
    ProjectConfig -->|defines project| CLI

    %% Click Events
    click CLI "https://github.com/parthbs/crewai/tree/main/src/crewai/cli/"
    click SDK "https://github.com/parthbs/crewai/blob/main/src/crewai/__init__.py"
    click SDK "https://github.com/parthbs/crewai/blob/main/src/crewai/agent.py"
    click SDK "https://github.com/parthbs/crewai/blob/main/src/crewai/crew.py"
    click SDK "https://github.com/parthbs/crewai/blob/main/src/crewai/flow/__init__.py"
    click SDK "https://github.com/parthbs/crewai/blob/main/src/crewai/project/__init__.py"
    click CrewConfig "https://github.com/parthbs/crewai/tree/main/src/crewai/cli/templates/crew/config/"
    click FlowConfig "https://github.com/parthbs/crewai/tree/main/src/crewai/cli/templates/flow/crews/poem_crew/config/"
    click ProjectConfig "https://github.com/parthbs/crewai/blob/main/pyproject.toml"
    click ProjectConfig "https://github.com/parthbs/crewai/blob/main/src/crewai/cli/templates/crew/pyproject.toml"
    click ProjectConfig "https://github.com/parthbs/crewai/blob/main/src/crewai/cli/templates/flow/pyproject.toml"
    click CrewEngine "https://github.com/parthbs/crewai/blob/main/src/crewai/crews/crew_output.py"
    click CrewEngine "https://github.com/parthbs/crewai/blob/main/src/crewai/project/crew_base.py"
    click FlowEngine "https://github.com/parthbs/crewai/blob/main/src/crewai/flow/flow.py"
    click FlowEngine "https://github.com/parthbs/crewai/blob/main/src/crewai/flow/flow_trackable.py"
    click FlowEngine "https://github.com/parthbs/crewai/blob/main/src/crewai/flow/config.py"
    click FlowEngine "https://github.com/parthbs/crewai/blob/main/src/crewai/flow/flow_visualizer.py"
    click FlowEngine "https://github.com/parthbs/crewai/blob/main/src/crewai/flow/assets/crewai_flow_visual_template.html"
    click EventBus "https://github.com/parthbs/crewai/blob/main/src/crewai/utilities/events/crewai_event_bus.py"
    click EventBus "https://github.com/parthbs/crewai/blob/main/src/crewai/utilities/events/base_events.py"
    click AgentExec "https://github.com/parthbs/crewai/tree/main/src/crewai/agents/agent_builder/"
    click AgentExec "https://github.com/parthbs/crewai/blob/main/src/crewai/agents/crew_agent_executor.py"
    click AgentAdapters "https://github.com/parthbs/crewai/tree/main/src/crewai/agents/agent_adapters/"
    click AgentAdapters "https://github.com/parthbs/crewai/blob/main/src/crewai/agents/agent_adapters/base_agent_adapter.py"
    click ToolAdapters "https://github.com/parthbs/crewai/blob/main/src/crewai/agents/agent_adapters/base_tool_adapter.py"
    click ToolAdapters "https://github.com/parthbs/crewai/blob/main/src/crewai/agents/agent_adapters/base_converter_adapter.py"
    click LLMModule "https://github.com/parthbs/crewai/blob/main/src/crewai/llm.py"
    click LLMModule "https://github.com/parthbs/crewai/tree/main/src/crewai/llms/"
    click MemoryModule "https://github.com/parthbs/crewai/tree/main/src/crewai/memory/"
    click KnowledgeModule "https://github.com/parthbs/crewai/tree/main/src/crewai/knowledge/embedder/"
    click KnowledgeModule "https://github.com/parthbs/crewai/tree/main/src/crewai/knowledge/source/"
    click KnowledgeModule "https://github.com/parthbs/crewai/tree/main/src/crewai/knowledge/storage/"
    click ToolsModule "https://github.com/parthbs/crewai/blob/main/src/crewai/tools/base_tool.py"
    click ToolsModule "https://github.com/parthbs/crewai/tree/main/src/crewai/tools/agent_tools/"
    click ToolsModule "https://github.com/parthbs/crewai/tree/main/src/crewai/tools/cache_tools/"
    click Persistence "https://github.com/parthbs/crewai/blob/main/src/crewai/flow/persistence/sqlite.py"
    click Persistence "https://github.com/parthbs/crewai/blob/main/src/crewai/flow/persistence/base.py"
    click Telemetry "https://github.com/parthbs/crewai/blob/main/src/crewai/telemetry/telemetry.py"
    click Telemetry "https://github.com/parthbs/crewai/blob/main/src/crewai/telemetry/constants.py"
    click Security "https://github.com/parthbs/crewai/blob/main/src/crewai/security/fingerprint.py"
    click Security "https://github.com/parthbs/crewai/blob/main/src/crewai/security/security_config.py"

    %% Styles
    classDef ui fill:#e3f2fd,stroke:#64b5f6
    classDef config fill:#f3e5f5,stroke:#ba68c8
    classDef core fill:#e8f5e9,stroke:#66bb6a
    classDef adapter fill:#fff3e0,stroke:#ffb74d
    classDef service fill:#f9fbe7,stroke:#dce775
    classDef external fill:#eeeeee,stroke:#bdbdbd,stroke-dasharray: 5 5
```
