# roshan10k/lead_tool architecture

> Architecture diagram of the GitHub repository roshan10k/lead_tool, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/roshan10k/lead_tool
- Repository: https://github.com/roshan10k/lead_tool

## Overview

Lead Discovery MVP turns exact searches, natural-language goals, or uploaded company lists into enriched, verified leads, then supports fit qualification and outreach tracking. The web client starts and polls jobs through the API; background pipeline stages discover or accept candidates, scrape sites, extract contact details, verify email domains, and persist results. The README documents the major workflows and integrations; source excerpts and the SOURCE INDEX support the shown internal calls, while unsampled details are not inferred.

Main entry points: [`apps/api/src/services/pipeline.ts`](https://github.com/roshan10k/lead_tool/blob/main/apps/api/src/services/pipeline.ts) (Background pipeline), [`apps/api/src/app.ts`](https://github.com/roshan10k/lead_tool/blob/main/apps/api/src/app.ts) (HTTP API), [`apps/api/src/services/scrape.ts`](https://github.com/roshan10k/lead_tool/blob/main/apps/api/src/services/scrape.ts) (Website scraping), [`apps/api/src/services/extract.ts`](https://github.com/roshan10k/lead_tool/blob/main/apps/api/src/services/extract.ts) (Contact extraction)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_web["Lead workspace"]
  node_webui["Search and lead UI<br/>[page.tsx]"]
  node_searchform["Search form<br/>[SearchForm.tsx]"]
  node_leadviews["Lead views"]
  node_rtk["API client<br/>[apiSlice.ts]"]
end

subgraph group_api["API and jobs"]
  node_routes["HTTP API<br/>[app.ts]"]
  node_csv["CSV import parser<br/>[csvImport.ts]"]
end

subgraph group_pipeline["Lead processing"]
  node_pipeline["Background pipeline<br/>[pipeline.ts]"]
  node_agent["Goal search agent<br/>[searchAgent.ts]"]
  node_discovery["Business discovery<br/>[discovery.ts]"]
  node_scrape["Website scraping<br/>[scrape.ts]"]
  node_extract["Contact extraction<br/>[extract.ts]"]
  node_contact["Structured contact finder<br/>[contactFinder.ts]"]
  node_verify["Email verification<br/>[verifyEmail.ts]"]
  node_emailrules["Email plausibility rules<br/>[email.ts]"]
end

subgraph group_data["Persistence and qualification"]
  node_qualification["Lead qualification<br/>[qualifyLeads.ts]"]
  node_dbclient["Database access<br/>[client.ts]"]
  node_schema[("Lead and job records<br/>[schema.ts]")]
end

node_user(("User"))
node_discovery_sources["Serper / DuckDuckGo"]
node_groq["Groq LLM"]
node_business_sites["Business websites"]
node_dns["DNS"]

node_user -->|"uses"| node_webui
node_webui -.->|"presents"| node_searchform
node_webui -.->|"presents"| node_leadviews
node_searchform -.->|"starts searches"| node_rtk
node_leadviews -.->|"loads lead data"| node_rtk
node_rtk -->|"HTTP requests"| node_routes
node_routes -->|"starts search jobs"| node_pipeline
node_routes -->|"parses uploaded CSV"| node_csv
node_routes -->|"starts qualification"| node_qualification
node_pipeline -->|"plans goal search"| node_agent
node_agent -->|"tests queries"| node_discovery
node_pipeline -->|"finds candidates"| node_discovery
node_discovery -->|"queries sources"| node_discovery_sources
node_pipeline -->|"scrapes candidate"| node_scrape
node_scrape -->|"fetches pages"| node_business_sites
node_pipeline -->|"extracts lead"| node_extract
node_extract -->|"requests extraction"| node_groq
node_pipeline -->|"finds contact details"| node_contact
node_contact -->|"uses scraping"| node_scrape
node_contact -->|"uses extraction"| node_extract
node_scrape -->|"checks page emails"| node_emailrules
node_extract -->|"filters extracted email"| node_emailrules
node_pipeline -->|"checks email domain"| node_verify
node_verify -->|"looks up mail records"| node_dns
node_pipeline -->|"stores lead results"| node_dbclient
node_dbclient -->|"uses tables"| node_schema
node_qualification -->|"reads and writes jobs"| node_dbclient
node_qualification -->|"judges lead fit"| node_groq
node_routes -->|"serves API records"| node_dbclient

click node_webui "https://github.com/roshan10k/lead_tool/blob/main/apps/web/app/page.tsx"
click node_searchform "https://github.com/roshan10k/lead_tool/blob/main/apps/web/components/SearchForm.tsx"
click node_leadviews "https://github.com/roshan10k/lead_tool/blob/main/apps/web/components/LeadGroupDetail.tsx"
click node_rtk "https://github.com/roshan10k/lead_tool/blob/main/apps/web/lib/apiSlice.ts"
click node_routes "https://github.com/roshan10k/lead_tool/blob/main/apps/api/src/app.ts"
click node_csv "https://github.com/roshan10k/lead_tool/blob/main/apps/api/src/lib/csvImport.ts"
click node_pipeline "https://github.com/roshan10k/lead_tool/blob/main/apps/api/src/services/pipeline.ts"
click node_agent "https://github.com/roshan10k/lead_tool/blob/main/apps/api/src/services/searchAgent.ts"
click node_discovery "https://github.com/roshan10k/lead_tool/blob/main/apps/api/src/services/discovery.ts"
click node_scrape "https://github.com/roshan10k/lead_tool/blob/main/apps/api/src/services/scrape.ts"
click node_extract "https://github.com/roshan10k/lead_tool/blob/main/apps/api/src/services/extract.ts"
click node_contact "https://github.com/roshan10k/lead_tool/blob/main/apps/api/src/services/contactFinder.ts"
click node_verify "https://github.com/roshan10k/lead_tool/blob/main/apps/api/src/services/verifyEmail.ts"
click node_emailrules "https://github.com/roshan10k/lead_tool/blob/main/apps/api/src/lib/email.ts"
click node_qualification "https://github.com/roshan10k/lead_tool/blob/main/apps/api/src/services/qualifyLeads.ts"
click node_dbclient "https://github.com/roshan10k/lead_tool/blob/main/apps/api/src/db/client.ts"
click node_schema "https://github.com/roshan10k/lead_tool/blob/main/apps/api/src/db/schema.ts"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_webui,node_searchform,node_leadviews,node_rtk,node_user toneBlue
class node_routes,node_csv toneAmber
class node_pipeline,node_agent,node_discovery,node_scrape,node_extract,node_contact,node_verify,node_emailrules toneMint
class node_qualification,node_dbclient,node_schema toneRose
class node_discovery_sources,node_groq,node_business_sites,node_dns toneIndigo
```

## Components

### Lead workspace

- [Search and lead UI](https://github.com/roshan10k/lead_tool/blob/main/apps/web/app/page.tsx): `apps/web/app/page.tsx`
- [Search form](https://github.com/roshan10k/lead_tool/blob/main/apps/web/components/SearchForm.tsx): `apps/web/components/SearchForm.tsx`
- [Lead views](https://github.com/roshan10k/lead_tool/blob/main/apps/web/components/LeadGroupDetail.tsx): `apps/web/components/LeadGroupDetail.tsx`
- [API client](https://github.com/roshan10k/lead_tool/blob/main/apps/web/lib/apiSlice.ts): `apps/web/lib/apiSlice.ts`

### API and jobs

- [HTTP API](https://github.com/roshan10k/lead_tool/blob/main/apps/api/src/app.ts): `apps/api/src/app.ts`
- [CSV import parser](https://github.com/roshan10k/lead_tool/blob/main/apps/api/src/lib/csvImport.ts): `apps/api/src/lib/csvImport.ts`

### Lead processing

- [Background pipeline](https://github.com/roshan10k/lead_tool/blob/main/apps/api/src/services/pipeline.ts): `apps/api/src/services/pipeline.ts`
- [Goal search agent](https://github.com/roshan10k/lead_tool/blob/main/apps/api/src/services/searchAgent.ts): `apps/api/src/services/searchAgent.ts`
- [Business discovery](https://github.com/roshan10k/lead_tool/blob/main/apps/api/src/services/discovery.ts): `apps/api/src/services/discovery.ts`
- [Website scraping](https://github.com/roshan10k/lead_tool/blob/main/apps/api/src/services/scrape.ts): `apps/api/src/services/scrape.ts`
- [Contact extraction](https://github.com/roshan10k/lead_tool/blob/main/apps/api/src/services/extract.ts): `apps/api/src/services/extract.ts`
- [Structured contact finder](https://github.com/roshan10k/lead_tool/blob/main/apps/api/src/services/contactFinder.ts): `apps/api/src/services/contactFinder.ts`
- [Email verification](https://github.com/roshan10k/lead_tool/blob/main/apps/api/src/services/verifyEmail.ts): `apps/api/src/services/verifyEmail.ts`
- [Email plausibility rules](https://github.com/roshan10k/lead_tool/blob/main/apps/api/src/lib/email.ts): `apps/api/src/lib/email.ts`

### Persistence and qualification

- [Lead qualification](https://github.com/roshan10k/lead_tool/blob/main/apps/api/src/services/qualifyLeads.ts): `apps/api/src/services/qualifyLeads.ts`
- [Database access](https://github.com/roshan10k/lead_tool/blob/main/apps/api/src/db/client.ts): `apps/api/src/db/client.ts`
- [Lead and job records](https://github.com/roshan10k/lead_tool/blob/main/apps/api/src/db/schema.ts): `apps/api/src/db/schema.ts`

### Other components

- **User**
- **Serper / DuckDuckGo**
- **Groq LLM**
- **Business websites**
- **DNS**

## Connections

- User → Search and lead UI: uses
- Search and lead UI → Search form: presents
- Search and lead UI → Lead views: presents
- Search form → API client: starts searches
- Lead views → API client: loads lead data
- API client → HTTP API: HTTP requests (evidence: `apps/web/lib/apiSlice.ts`)
- HTTP API → Background pipeline: starts search jobs (evidence: `apps/api/src/app.ts`)
- HTTP API → CSV import parser: parses uploaded CSV (evidence: `apps/api/src/app.ts`)
- HTTP API → Lead qualification: starts qualification (evidence: `apps/api/src/app.ts`)
- Background pipeline → Goal search agent: plans goal search (evidence: `apps/api/src/services/pipeline.ts`)
- Goal search agent → Business discovery: tests queries (evidence: `apps/api/src/services/searchAgent.ts`)
- Background pipeline → Business discovery: finds candidates (evidence: `apps/api/src/services/pipeline.ts`)
- Business discovery → Serper / DuckDuckGo: queries sources
- Background pipeline → Website scraping: scrapes candidate (evidence: `apps/api/src/services/pipeline.ts`)
- Website scraping → Business websites: fetches pages
- Background pipeline → Contact extraction: extracts lead (evidence: `apps/api/src/services/pipeline.ts`)
- Contact extraction → Groq LLM: requests extraction
- Background pipeline → Structured contact finder: finds contact details (evidence: `apps/api/src/services/pipeline.ts`)
- Structured contact finder → Website scraping: uses scraping (evidence: `apps/api/src/services/contactFinder.ts`)
- Structured contact finder → Contact extraction: uses extraction (evidence: `apps/api/src/services/contactFinder.ts`)
- Website scraping → Email plausibility rules: checks page emails (evidence: `apps/api/src/services/scrape.ts`)
- Contact extraction → Email plausibility rules: filters extracted email (evidence: `apps/api/src/services/extract.ts`)
- Background pipeline → Email verification: checks email domain (evidence: `apps/api/src/services/pipeline.ts`)
- Email verification → DNS: looks up mail records
- Background pipeline → Database access: stores lead results (evidence: `apps/api/src/services/pipeline.ts`)
- Database access → Lead and job records: uses tables (evidence: `apps/api/src/db/client.ts`)
- Lead qualification → Database access: reads and writes jobs (evidence: `apps/api/src/services/qualifyLeads.ts`)
- Lead qualification → Groq LLM: judges lead fit
- HTTP API → Database access: serves API records (evidence: `apps/api/src/app.ts`)
