# gurpreetkaurjethra/ai-lead-generation-agent architecture

> Architecture diagram of the GitHub repository gurpreetkaurjethra/ai-lead-generation-agent, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/gurpreetkaurjethra/ai-lead-generation-agent
- Repository: https://github.com/gurpreetkaurjethra/ai-lead-generation-agent

## Overview

##### Project type and purpose
- This is a single-script, full-stack AI agent application with a Streamlit front end that automates lead discovery, qualification, and storage.
- It orchestrates multiple external services (Firecrawl’s Search & Extract APIs, OpenAI GPT-4o, Composio Google Sheets) via phidata.

##### File structure
- Top‐level files only:
- ai_lead_generation_agent.py – the monolithic Streamlit app + orchestration logic
- requirements.txt – lists Firecrawl SDK, phidata, composio, streamlit, openai, etc.
- IMG_AILG – folder of architecture/UX screenshots
- No separate backend directory: all components are invoked from within the script.

##### Key insights from the README
- Streamlit is used for the web UI (“streamlit run …”).
- phidata is used for “agent orchestration” – i.e., managing the workflow of search → extract → qualification → sheet update.
- Firecrawl provides two endpoints: search (to find Quora URLs) and extract (to get profile data).
- Composio integrates with Google Sheets to persist leads.
- OpenAI GPT-4o is used to qualify extracted profiles.

##### Main components and their interactions
a. Components:
1. Streamlit UI
2. Agent Orchestrator (phidata)
3. Firecrawl Search API
4. Firecrawl Extract API
5. OpenAI GPT-4o Qualification
6. Composio Google Sheets Integration
7. Google Sheets (external data store)
b. Relationships & data flow:
- User → Streamlit UI: enters search criteria & starts run
- Streamlit UI → phidata agent: triggers workflow
- Agent → Firecrawl Search API: retrieves list of Quora URLs
- Agent → Firecrawl Extract API (in loop): fetches profile data for each URL
- Agent → OpenAI GPT-4o: passes raw profile data for qualification & enrichment
- Agent → Composio SDK: writes formatted, qualified leads to Google Sheets
- Google Sheets ← Composio: stores final lead table
- Streamlit UI polls or displays progress / final results
c. Architectural patterns:
- Orchestrator (phidata) implements a pipeline/workflow pattern
- Monolith + external service integration rather than microservices
- Reactive UI (Streamlit) calls into orchestration
d. Key technologies:
- Streamlit for UI
- phidata for workflow orchestration
- Firecrawl SDK & REST APIs for web scraping
- OpenAI Python SDK for GPT-4o calls
- Composio Python SDK for Sheets integration
- Google Sheets as persistent storage

- Blue rectangle for Streamlit UI
- Green circle/hexagon for phidata Orchestrator
- Orange ellipses for external APIs (Firecrawl Search, Firecrawl Extract, OpenAI)
1. User → Streamlit
2. Streamlit → phidata orchestrator
3. phidata → Firecrawl Search API → returns URLs
4. phidata → Firecrawl Extract API → returns profiles
5. phidata → OpenAI → returns qualifiers
6. phidata → Composio → writes to Google Sheets
7. Google Sheets → final persisted leads

##### Detailed capture of essential elements
- Expected data formats (JSON response from Firecrawl, tabular rows to Sheets).

- Clearly separates UI, orchestration, service integrations, and data store
- Shows the precise flow through Firecrawl, OpenAI, and Composio
- Highlights the core design principle: a workflow-driven, fully automated lead generation agent

## Diagram (Mermaid)

```mermaid
flowchart TD
    User["User"]:::user
    User --> Streamlit
    Streamlit["Streamlit UI"]:::frontend
    Streamlit --> Orchestrator

    subgraph "Orchestrator (phidata)"
        Orchestrator["Agent Orchestrator"]:::orchestrator
        searchStep["1. Search Profiles"]:::orchestrator
        extractStep["2. Extract Profiles"]:::orchestrator
        qualifyStep["3. Qualify Leads"]:::orchestrator
        persistStep["4. Persist to Sheets"]:::orchestrator
        Orchestrator --> searchStep
        searchStep --> extractStep
        extractStep --> qualifyStep
        qualifyStep --> persistStep
    end

    searchStep --> FirecrawlSearch
    FirecrawlSearch["Firecrawl Search API"]:::api
    extractStep --> FirecrawlExtract
    FirecrawlExtract["Firecrawl Extract API"]:::api
    qualifyStep --> OpenAI["OpenAI GPT-4o"]:::api
    persistStep --> Composio["Composio SDK"]:::sdk
    Composio --> GoogleSheets
    GoogleSheets["Google Sheets"]:::datastore

    subgraph "Documentation & Assets"
        Deps["Dependencies\n(requirements.txt)"]:::deps
        Screens["Architecture/UX\nScreenshots (IMG_AILG/)"]:::doc
        README["Project Overview\n(README.md)"]:::doc
    end

    classDef frontend fill:#bbdefb,stroke:#1e88e5,color:#000;
    classDef orchestrator fill:#c8e6c9,stroke:#43a047,color:#000;
    classDef api fill:#ffe0b2,stroke:#fb8c00,color:#000;
    classDef sdk fill:#fff59d,stroke:#fbc02d,color:#000;
    classDef datastore fill:#e1bee7,stroke:#8e24aa,color:#000;
    classDef user fill:#ffccbc,stroke:#d84315,color:#000;
    classDef deps fill:#f0f4c3,stroke:#afb42b,color:#000;
    classDef doc fill:#cfd8dc,stroke:#546e7a,color:#000;

    click Streamlit "https://github.com/gurpreetkaurjethra/ai-lead-generation-agent/blob/main/ai_lead_generation_agent.py"
    click Orchestrator "https://github.com/gurpreetkaurjethra/ai-lead-generation-agent/blob/main/ai_lead_generation_agent.py"
    click searchStep "https://github.com/gurpreetkaurjethra/ai-lead-generation-agent/blob/main/ai_lead_generation_agent.py"
    click extractStep "https://github.com/gurpreetkaurjethra/ai-lead-generation-agent/blob/main/ai_lead_generation_agent.py"
    click qualifyStep "https://github.com/gurpreetkaurjethra/ai-lead-generation-agent/blob/main/ai_lead_generation_agent.py"
    click persistStep "https://github.com/gurpreetkaurjethra/ai-lead-generation-agent/blob/main/ai_lead_generation_agent.py"
    click FirecrawlSearch "https://github.com/gurpreetkaurjethra/ai-lead-generation-agent/blob/main/ai_lead_generation_agent.py"
    click FirecrawlExtract "https://github.com/gurpreetkaurjethra/ai-lead-generation-agent/blob/main/ai_lead_generation_agent.py"
    click OpenAI "https://github.com/gurpreetkaurjethra/ai-lead-generation-agent/blob/main/ai_lead_generation_agent.py"
    click Composio "https://github.com/gurpreetkaurjethra/ai-lead-generation-agent/blob/main/ai_lead_generation_agent.py"
    click Deps "https://github.com/gurpreetkaurjethra/ai-lead-generation-agent/blob/main/requirements.txt"
    click Screens "https://github.com/gurpreetkaurjethra/ai-lead-generation-agent/tree/main/IMG_AILG/"
    click README "https://github.com/gurpreetkaurjethra/ai-lead-generation-agent/blob/main/README.md"
```
