# momenashra/agentic-patterns architecture

> Architecture diagram of the GitHub repository momenashra/agentic-patterns, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/momenashra/agentic-patterns
- Repository: https://github.com/momenashra/agentic-patterns

## Overview

##### Project type and purpose
- This is an open-source demonstration repository showcasing two related patterns for LLM-driven agents:
a. A **Reflection Pattern** (in `Reflection-pattern/`) that implements an agent capable of self-reflection and iterative improvement.
b. An **Autonomous Multi-Agent Web Scraping Framework** (in `multiagent-pattern/` plus the top-level scripts described in the README) that orchestrates Search, Scrape, Aggregate, and Reporting agents to collect and summarize product data.

##### File structure
- Top-level files (per README, even if not shown in your tree):
- `run_pipeline.py` – launches the multi-agent crew
- `tasks.py` – defines the Searcher, Scraper, Aggregator, and Reporter agents
- `tools.py` – custom utilities (e.g., JSON writer)
- `output/` – stores JSON results and generated reports
- `Reflection-pattern/` sub-directory:
- `reflection_pattern.ipynb` – demonstration notebook
- `src/reflection_agent.py` – core agent logic implementing reflection loops
- `utils/completions.py` & `utils/logging.py` – abstraction over LLM calls and logging
- `multiagent-pattern/` sub-directory:
- Notebooks (`Product_searcher.ipynb`, `step_5_enhanced_report.html`) showing stages of the scraping pipeline

##### README for additional insights
- Describes four pipeline phases: Search → Scrape → Aggregate → Report
- Mentions use of **CrewAI** for orchestration and external services (Cohere, AgentOps) for LLM and tooling
- Emphasizes modularity (agents & tools can be swapped), scalability (parallel execution), and reproducibility

a. Main components
- Reflection Agent Subsystem
- LLM Interface (via `completions.py`)
- Reflection Loop Controller (`reflection_agent.py`)
- Logging/Telemetry (`logging.py`)
- Multi-Agent Pipeline Subsystem
1. **Search Agent** – queries target sites
2. **Scraper Agent** – extracts page data
3. **Aggregator Agent** – cleans & merges JSON
4. **Writer Agent** – generates human-readable report
- Orchestration Layer
- CrewAI orchestrator (in `run_pipeline.py` and `tasks.py`)
- External Services
- Cohere API
- AgentOps API
- Data Stores / Outputs
- Intermediate JSON files (`output/`)
- Final HTML/Markdown/PDF reports

b. Relationships & interactions
- User invokes `run_pipeline.py` → CrewAI initializes agent instances
- **Search Agent** calls Cohere (optional NLP) + target websites → returns URL list
- URL list flows to **Scraper Agent**, which writes raw JSON
- Raw JSON flows to **Aggregator Agent** → outputs cleaned JSON
- Clean JSON flows to **Writer Agent** → emits report files
- Reflection Agent (in isolation) interacts with LLM via `completions.py`, loops on its own output for refinement

c. Architectural patterns & principles
- **Pipeline Pattern**: clear stage separation, data flows in a directed acyclic graph
- **Modular Agents**: each agent is a micro-component with a single responsibility
- **Orchestration via CrewAI**: pluggable execution order (parallel vs. sequential)
- **Extensibility**: tools and agents can be swapped without changing core orchestration

d. Key technologies / libraries
- Python 3.10+
- CrewAI (pipeline orchestration)
- Tavily-Python, ScrapeGraph-Py for scraping helpers
- Cohere API, AgentOps API for LLM calls
- Standard Python modules (`logging`, JSON, `argparse`)

## Diagram (Mermaid)

```mermaid
flowchart TB
  subgraph "Reflection-Pattern Module"
    REFNotebook["Reflection Notebook"]:::notebook
    Controller["Reflection Agent Controller"]:::agent
    LLM["LLM Interface (completions.py)"]:::interface
    Logging["Logging/Telemetry Module"]:::logging
  end

  Controller -->|uses| LLM
  Controller -->|logs to| Logging
  Controller -->|reads/writes| REFNotebook
  LLM -->|"calls Cohere/AgentOps"| ExternalLLM["Cohere API / AgentOps API"]:::external

  subgraph "Multi-Agent Pipeline"
    Orchestrator["CrewAI Orchestrator"]:::orchestrator
    Search["Search Agent"]:::agent
    Scraper["Scraper Agent"]:::agent
    Aggregator["Aggregator Agent"]:::agent
    Writer["Writer Agent"]:::agent
    NotebookMA["Search Agent Notebook"]:::notebook
    ReportHTML["Enhanced Report Output"]:::report
    JSONIntermediate["Intermediate JSON Files"]:::datastore
    FinalReports["Final Reports (HTML/Markdown/PDF)"]:::datastore
  end

  Orchestrator --> Search
  Search --> NotebookMA
  Search -->|URL list| Scraper
  Scraper --> JSONIntermediate
  JSONIntermediate --> Aggregator
  Aggregator -->|cleaned JSON| Writer
  Writer --> JSONIntermediate
  Writer --> FinalReports
  Writer --> ReportHTML
  Search -->|optional NLP| ExternalLLM

  click Controller "https://github.com/momenashra/agentic-patterns/blob/main/Reflection-pattern/src/reflection_agent.py"
  click LLM "https://github.com/momenashra/agentic-patterns/blob/main/Reflection-pattern/utils/completions.py"
  click Logging "https://github.com/momenashra/agentic-patterns/blob/main/Reflection-pattern/utils/logging.py"
  click REFNotebook "https://github.com/momenashra/agentic-patterns/blob/main/Reflection-pattern/reflection_pattern.ipynb"
  click NotebookMA "https://github.com/momenashra/agentic-patterns/blob/main/multiagent-pattern/Product_searcher.ipynb"
  click ReportHTML "https://github.com/momenashra/agentic-patterns/blob/main/multiagent-pattern/step_5_enhanced_report.html"

  classDef agent fill:#1f77b4,stroke:#333,color:#fff
  classDef interface fill:#1f77b4,stroke:#333,color:#fff
  classDef logging fill:#ff7f0e,stroke:#333,color:#fff
  classDef orchestrator fill:#9467bd,stroke:#333,color:#fff
  classDef external fill:#2ca02c,stroke:#333,color:#fff
  classDef datastore fill:#bcbd22,stroke:#333,color:#000
  classDef notebook fill:#8c564b,stroke:#333,color:#fff
  classDef report fill:#e377c2,stroke:#333,color:#fff
```
