# eracle/openoutreach architecture

> Architecture diagram of the GitHub repository eracle/openoutreach, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/eracle/openoutreach
- Repository: https://github.com/eracle/openoutreach

## Overview

1. **Overall shape**
   - OpenOutreach is a **single Django codebase** organized as several domain apps under `openoutreach/`, but operationally it behaves like a **background worker + CRM web admin + browser automation stack**.
   - The core runtime is the **daemon**; the web UI is mainly Django Admin/CRM for inspecting and configuring data.

2. **Top-level orchestration boundary: `core`**
   - `openoutreach/core` is the system’s control plane: configuration, onboarding, scheduling, daemon loop, task execution, LLM integration, logging, and follow-up prompting.
   - It owns the persistent **`Task` queue** and the main scheduling loop that decides what work to run next.
   - This is the place where campaign state, rate limits, and “what happens next” are coordinated.

3. **LinkedIn acquisition boundary: `linkedin`**
   - `openoutreach/linkedin` is the discovery and LinkedIn channel subsystem.
   - It contains **browser automation** (`browser/`), **LinkedIn API/Voyager scraping** (`api/`), **profile/lead DB access helpers** (`db/`), **ML ranking/qualification** (`ml/`), and the **pipeline** that turns search results into actionable candidates.
   - This app is responsible for discovering profiles, reading LinkedIn data, and driving connect/follow-up workflows.

4. **Automation runtime**
   - LinkedIn interactions run through **Playwright-based browser automation** with stealth-oriented session management.
   - The browser layer is isolated behind session/registry/launch modules so the rest of the system can treat browser control as an execution service rather than directly handling page state.
   - In Docker, the browser is exposed through VNC/noVNC for live observation.

5. **Qualification and lead selection flow**
   - Discovery data enters the pipeline as profiles, then gets embedded and scored by the ML layer.
   - The system uses a **Gaussian Process-based active learning loop** to choose which profile to qualify next: exploit likely positives when negatives dominate, otherwise explore via uncertainty/information gain.
   - An **LLM** performs the final qualification decision, and those labels feed back into the model for future ranking.

6. **Email outreach boundary: `emails`**
   - `openoutreach/emails` is a separate channel subsystem for the “email-first” route.
   - It includes email resolution/integration helpers (`bettercontact.py`), provider abstractions (`icemail.py`, `smtp.py`, `sender.py`), nudge logic, and a task handler for sending.
   - This channel is used only after a lead is qualified and a work email is found; otherwise the lead stays on the LinkedIn path.

7. **CRM/data model boundary: `crm`, `chat`, and core DB helpers**
   - `openoutreach/crm` stores the business entities: **Leads** and **Deals**, with migrations showing state evolution over time.
   - `openoutreach/chat` stores message history and deal linkage for conversation tracking.
   - `openoutreach/core/db` and `openoutreach/linkedin/db` contain domain-specific query/update helpers, so orchestration code doesn’t manipulate models directly everywhere.

8. **Stateful pipeline and channel routing**
   - The main system behavior is a **state machine persisted in the database**: profiles move through qualification, then fork into either email or LinkedIn workflow.
   - A qualified profile is routed by email availability:
     - email found → `READY_TO_EMAIL` → send opener → `EMAILED`
     - no email / no finder key → `READY_TO_CONNECT` → pending connection → accepted → follow-up
   - This design makes the system resumable across restarts because the database is the source of truth.

9. **Tasks, scheduler, and self-scheduling work**
   - The daemon runs a persistent task loop with task types like **connect**, **check pending**, **follow up**, and **email**.
   - Tasks schedule follow-on tasks rather than relying on one large batch job, which lets the system pace itself, honor limits, and recover cleanly.
   - `core/scheduler.py` and `core/daemon.py` are the main runtime control points; `manage.py` defaults to starting this daemon behavior.

10. **Onboarding and configuration**
   - First-run setup is interactive and handled by `core/onboarding*`, `core/management/commands/onboard.py`, and `setup_crm.py`.
   - Configuration includes LinkedIn credentials, LLM API settings, email-finder/sending settings, campaign description, and rate limits.
   - The system stores its persistent local data under `data/`, with SQLite as the default backing store.

11. **External services that materially affect architecture**
   - **LinkedIn** is the discovery and interaction platform.
   - **LLM provider** is required for qualification and message generation.
   - **BetterContact** is used to resolve work emails.
   - **IceMail/SMTP** provides the actual sending mailbox.
   - These integrations define the main runtime branches and are not optional if you want the full email-first flow.

12. **Operational packaging**
   - The repo supports local development and Docker deployment.
   - `compose/` and `local.yml` indicate containerized runs, while GitHub Actions handle tests and deployment.
   - The deployment model is “self-hosted appliance”: one image, local persistent data, browser automation, and admin UI all running together.

Main entry points: [`openoutreach/linkedin/pipeline/qualify.py`](https://github.com/eracle/openoutreach/blob/main/openoutreach/linkedin/pipeline/qualify.py) (Lead pipeline), [`openoutreach/core/scheduler.py`](https://github.com/eracle/openoutreach/blob/main/openoutreach/core/scheduler.py) (Scheduler), [`openoutreach/linkedin/tasks/connect.py`](https://github.com/eracle/openoutreach/blob/main/openoutreach/linkedin/tasks/connect.py) (LinkedIn tasks), [`openoutreach/crm/models/lead.py`](https://github.com/eracle/openoutreach/blob/main/openoutreach/crm/models/lead.py) (Leads)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Runtime & Ops"]
  node_manage_py["manage.py<br/>entrypoint<br/>[manage.py]"]
  node_settings["Settings<br/>config<br/>[settings.py]"]
  node_compose["Docker setup<br/>deployment"]
end

subgraph group_core["Core Control Plane"]
  node_daemon(("Daemon<br/>runtime loop<br/>[daemon.py]"))
  node_scheduler["Scheduler<br/>task planner<br/>[scheduler.py]"]
  node_conf["Config<br/>[conf.py]"]
  node_onboarding["Onboarding<br/>setup flow<br/>[onboarding.py]"]
  node_llm["LLM<br/>model adapter<br/>[llm.py]"]
  node_task_model[("Task queue<br/>state model<br/>[models.py]")]
  node_task_agents["Prompt agents<br/>[prompt.py]"]
end

subgraph group_linkedin["LinkedIn Acquisition"]
  node_browser["Browser runtime<br/>automation<br/>[session.py]"]
  node_api["LinkedIn API<br/>scraping<br/>[newsletter.py]"]
  node_lpipeline["Lead pipeline<br/>[qualify.py]"]
  node_ml["Lead ranking<br/>ml<br/>[qualifier.py]"]
  node_link_tasks["LinkedIn tasks<br/>channel tasks<br/>[connect.py]"]
end

subgraph group_email["Email Outreach"]
  node_emailing["Email sender<br/>outreach<br/>[sender.py]"]
  node_email_tasks["Email task<br/>channel task<br/>[send.py]"]
end

subgraph group_data["CRM & Conversation Data"]
  node_lead_model[("Leads<br/>crm model<br/>[lead.py]")]
  node_deal_model[("Deals<br/>crm model<br/>[deal.py]")]
  node_chat_model[("Chat history<br/>conversation model<br/>[models.py]")]
end

node_manage_py -->|"starts"| node_daemon
node_settings -->|"feeds"| node_conf
node_compose -->|"packages"| node_browser
node_onboarding -->|"writes"| node_conf
node_onboarding -->|"seeds"| node_lead_model
node_daemon -->|"drives"| node_scheduler
node_scheduler -->|"pulls"| node_task_model
node_scheduler -->|"dispatches"| node_link_tasks
node_scheduler -->|"dispatches"| node_email_tasks
node_scheduler -->|"uses"| node_task_agents
node_task_agents -->|"queries"| node_llm
node_lpipeline -->|"automates"| node_browser
node_lpipeline -->|"reads"| node_api
node_lpipeline -->|"scores"| node_ml
node_ml -->|"augments"| node_llm
node_lpipeline -->|"updates"| node_lead_model
node_lpipeline -->|"routes"| node_deal_model
node_link_tasks -->|"executes"| node_browser
node_link_tasks -->|"mutates"| node_lead_model
node_link_tasks -->|"syncs"| node_chat_model
node_email_tasks -->|"sends"| node_emailing
node_emailing -->|"reads"| node_lead_model
node_emailing -->|"updates"| node_deal_model
node_chat_model -->|"links"| node_deal_model

click node_manage_py "https://github.com/eracle/openoutreach/blob/main/manage.py"
click node_settings "https://github.com/eracle/openoutreach/blob/main/openoutreach/settings.py"
click node_compose "https://github.com/eracle/openoutreach/tree/main/compose/linkedin/Dockerfile"
click node_daemon "https://github.com/eracle/openoutreach/blob/main/openoutreach/core/daemon.py"
click node_scheduler "https://github.com/eracle/openoutreach/blob/main/openoutreach/core/scheduler.py"
click node_conf "https://github.com/eracle/openoutreach/blob/main/openoutreach/core/conf.py"
click node_onboarding "https://github.com/eracle/openoutreach/blob/main/openoutreach/core/onboarding.py"
click node_llm "https://github.com/eracle/openoutreach/blob/main/openoutreach/core/llm.py"
click node_task_model "https://github.com/eracle/openoutreach/blob/main/openoutreach/core/models.py"
click node_task_agents "https://github.com/eracle/openoutreach/blob/main/openoutreach/core/agents/prompt.py"
click node_browser "https://github.com/eracle/openoutreach/blob/main/openoutreach/linkedin/browser/session.py"
click node_api "https://github.com/eracle/openoutreach/blob/main/openoutreach/linkedin/api/newsletter.py"
click node_lpipeline "https://github.com/eracle/openoutreach/blob/main/openoutreach/linkedin/pipeline/qualify.py"
click node_ml "https://github.com/eracle/openoutreach/blob/main/openoutreach/linkedin/ml/qualifier.py"
click node_link_tasks "https://github.com/eracle/openoutreach/blob/main/openoutreach/linkedin/tasks/connect.py"
click node_lead_model "https://github.com/eracle/openoutreach/blob/main/openoutreach/crm/models/lead.py"
click node_deal_model "https://github.com/eracle/openoutreach/blob/main/openoutreach/crm/models/deal.py"
click node_chat_model "https://github.com/eracle/openoutreach/blob/main/openoutreach/chat/models.py"
click node_emailing "https://github.com/eracle/openoutreach/blob/main/openoutreach/emails/sender.py"
click node_email_tasks "https://github.com/eracle/openoutreach/blob/main/openoutreach/emails/tasks/send.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_manage_py,node_settings,node_compose toneBlue
class node_daemon,node_scheduler,node_conf,node_onboarding,node_llm,node_task_model,node_task_agents toneAmber
class node_browser,node_api,node_lpipeline,node_ml,node_link_tasks toneMint
class node_emailing,node_email_tasks toneRose
class node_lead_model,node_deal_model,node_chat_model toneIndigo
```

## Components

### Runtime & Ops

Entrypoints, deployment, and local persistence

- [manage.py](https://github.com/eracle/openoutreach/blob/main/manage.py): entrypoint · Django entrypoint that defaults into daemon-style operation · `manage.py`
- [Settings](https://github.com/eracle/openoutreach/blob/main/openoutreach/settings.py): config · Project configuration and environment wiring · `openoutreach/settings.py`
- [Docker setup](https://github.com/eracle/openoutreach/tree/main/compose/linkedin/Dockerfile): deployment · Containerized runtime and browser packaging · `compose/linkedin/Dockerfile`

### Core Control Plane

Daemon, scheduling, onboarding, and shared orchestration

- [Daemon](https://github.com/eracle/openoutreach/blob/main/openoutreach/core/daemon.py): runtime loop · Persistent worker loop that drives work execution · `openoutreach/core/daemon.py`
- [Scheduler](https://github.com/eracle/openoutreach/blob/main/openoutreach/core/scheduler.py): task planner · Chooses next tasks and coordinates self-scheduling work · `openoutreach/core/scheduler.py`
- [Config](https://github.com/eracle/openoutreach/blob/main/openoutreach/core/conf.py): site config · Campaign settings, limits, and provider configuration · `openoutreach/core/conf.py`
- [Onboarding](https://github.com/eracle/openoutreach/blob/main/openoutreach/core/onboarding.py): setup flow · First-run setup and CRM bootstrap workflow · `openoutreach/core/onboarding.py`
- [LLM](https://github.com/eracle/openoutreach/blob/main/openoutreach/core/llm.py): model adapter · Shared language-model integration for qualification and messaging · `openoutreach/core/llm.py`
- [Task queue](https://github.com/eracle/openoutreach/blob/main/openoutreach/core/models.py): state model · Persistent task records that drive follow-on work · `openoutreach/core/models.py`
- [Prompt agents](https://github.com/eracle/openoutreach/blob/main/openoutreach/core/agents/prompt.py): agents · Qualification, opener, and follow-up prompt logic · `openoutreach/core/agents/prompt.py`

### LinkedIn Acquisition

Browser automation, scraping, ranking, and task pipeline

- [Browser runtime](https://github.com/eracle/openoutreach/blob/main/openoutreach/linkedin/browser/session.py): automation · Playwright session and browser launch management · `openoutreach/linkedin/browser/session.py`
- [LinkedIn API](https://github.com/eracle/openoutreach/blob/main/openoutreach/linkedin/api/newsletter.py): scraping · Voyager and other LinkedIn data access helpers · `openoutreach/linkedin/api/newsletter.py`
- [Lead pipeline](https://github.com/eracle/openoutreach/blob/main/openoutreach/linkedin/pipeline/qualify.py): pipeline · Search, pooling, qualification routing, and readiness staging · `openoutreach/linkedin/pipeline/qualify.py`
- [Lead ranking](https://github.com/eracle/openoutreach/blob/main/openoutreach/linkedin/ml/qualifier.py): ml · Embeddings, text features, and active-learning qualification · `openoutreach/linkedin/ml/qualifier.py`
- [LinkedIn tasks](https://github.com/eracle/openoutreach/blob/main/openoutreach/linkedin/tasks/connect.py): channel tasks · Connect, pending-check, and follow-up task handlers · `openoutreach/linkedin/tasks/connect.py`

### Email Outreach

Email resolution, sending, and nudges

- [Email sender](https://github.com/eracle/openoutreach/blob/main/openoutreach/emails/sender.py): outreach · Email provider abstraction, resolution, nudges, and sending · `openoutreach/emails/sender.py`
- [Email task](https://github.com/eracle/openoutreach/blob/main/openoutreach/emails/tasks/send.py): channel task · Task handler that sends outreach by email · `openoutreach/emails/tasks/send.py`

### CRM & Conversation Data

Persistent business entities and message history

- [Leads](https://github.com/eracle/openoutreach/blob/main/openoutreach/crm/models/lead.py): crm model · Primary lead entity with routing and lifecycle state · `openoutreach/crm/models/lead.py`
- [Deals](https://github.com/eracle/openoutreach/blob/main/openoutreach/crm/models/deal.py): crm model · Deal state, summaries, and next-check timing · `openoutreach/crm/models/deal.py`
- [Chat history](https://github.com/eracle/openoutreach/blob/main/openoutreach/chat/models.py): conversation model · Message history linked to leads and deals · `openoutreach/chat/models.py`

## Connections

- manage.py → Daemon: starts · Default execution path starts the worker loop
- Settings → Config: feeds · Environment and Django settings populate runtime config
- Docker setup → Browser runtime: packages · Container runtime includes browser automation support
- Onboarding → Config: writes · Setup captures campaign and provider settings
- Onboarding → Leads: seeds · Bootstraps initial CRM data and campaign state
- Daemon → Scheduler: drives · Daemon repeatedly asks scheduler what to do next
- Scheduler → Task queue: pulls · Scheduler reads persisted tasks as the work queue
- Scheduler → LinkedIn tasks: dispatches · Routes work into LinkedIn channel tasks
- Scheduler → Email task: dispatches · Routes work into email channel tasks
- Scheduler → Prompt agents: uses · Scheduling and prompt decisions share agent logic
- Prompt agents → LLM: queries · Prompts rely on LLM calls for decisions and generation
- Lead pipeline → Browser runtime: automates · Search and interaction flows run through browser sessions
- Lead pipeline → LinkedIn API: reads · Pipeline consumes LinkedIn API/scraped data
- Lead pipeline → Lead ranking: scores · Profiles are embedded, ranked, and qualified
- Lead ranking → LLM: augments · Modeling and qualification are informed by LLM labels
- Lead pipeline → Leads: updates · Pipeline writes lifecycle state back to leads
- Lead pipeline → Deals: routes · Qualified leads update deal state and next actions
- LinkedIn tasks → Browser runtime: executes · Connect and follow-up work runs in the browser
- LinkedIn tasks → Leads: mutates · Task handlers advance lead state through the funnel
- LinkedIn tasks → Chat history: syncs · Conversation state is recorded alongside outreach
- Email task → Email sender: sends · Email tasks invoke provider and sender abstractions
- Email sender → Leads: reads · Email routing depends on lead contact data and state
- Email sender → Deals: updates · Email sends advance deal/outreach state
- Chat history → Deals: links · Chat entries are associated with deals for tracking