# mshumer/ai-journalist architecture

> Architecture diagram of the GitHub repository mshumer/ai-journalist, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mshumer/ai-journalist
- Repository: https://github.com/mshumer/ai-journalist

## Overview

##### Project Type and Purpose
- This is a Python-based experimental AI tool implemented as a Jupyter notebook (Claude_Journalist.ipynb).
- Purpose: Given a topic, it orchestrates a research-and-writing pipeline using the Anthropic Claude 3 AI model, a SERP search API, and the newspaper3k scraper to produce and edit a publish-quality article.

##### File Structure Analysis
- Top-level:
- Claude_Journalist.ipynb – the single notebook containing all code, prompts, and orchestration.
- README.md – project overview, workflow steps, and prerequisites.
- LICENSE – MIT license.
- No separate “src” or “services” directories; the notebook is the entry point for orchestration of the pipeline.

##### Key Workflow Components (from README)
1. User prompt for topic
2. Prompt Claude to generate search terms (Claude 3 “Haiku”)
3. Perform web searches via SERP API
4. Prompt Claude to select top URLs
5. Scrape article text with newspaper3k
6. Prompt Claude to draft article
7. Prompt Claude to edit/rewrite the draft
8. Output final and edited articles

##### Main Components
a. User Interface
- Jupyter Notebook (CLI/interactive cell execution)
- Topic input cell, result display cell
b. Orchestrator / Controller
- Python code within notebook coordinating each step
c. External Services
- Anthropic Claude 3 API (two roles: researcher/generator & editor)
- SERP API (Google/Bing search wrapper)
d. Web Scraper
- newspaper3k library fetching HTML/article content
e. Data Stores (transient/in-memory)
- Search terms list, search results, raw HTML/text, draft articles, edited articles
f. Output
- Printed article(s) in notebook, optionally saved to file

##### Relationships & Data Flow
1. User → Notebook: “Topic”
2. Notebook → Claude 3 (API call #1): generates search terms
3. Notebook → SERP API: fetch search results for each term
4. Notebook → Claude 3 (API call #2): filters/chooses top URLs
5. Notebook → newspaper3k: scrapes selected URLs
6. Notebook → Claude 3 (API call #3): drafts article from scraped text
7. Notebook → Claude 3 (API call #4): edits/revises draft
8. Notebook → User: displays final output

##### Architectural Patterns & Principles
- Pipeline / Orchestration pattern: linear, stage-by-stage data flow
- Single-process, synchronous calls (blocking notebook cells)
- Externalized logic in prompts to Claude—keeps code generic, prompt-driven design
- Transient in-memory storage in Python variables—no persistent DB

##### Technologies & Frameworks
- Python 3.x within Jupyter
- Anthropic API client (Claude 3)
- SERP API client (requests wrapper)
- newspaper3k for scraping/parsing
- Standard libraries (requests, json, etc.)

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Swimlanes
    subgraph "User Interaction"
        U["User"]:::user
    end

    subgraph "Notebook Orchestrator"
        A["Input Topic"]:::notebook
        B["Generate Search Terms"]:::notebook
        C["Fetch Search Results"]:::notebook
        D["Select Top URLs"]:::notebook
        E["Scrape Articles"]:::notebook
        F["Draft Article"]:::notebook
        G["Edit Article"]:::notebook
        H["Display/Save Output"]:::notebook
    end

    subgraph "External Services & Libraries"
        CLAUDE["Claude 3 API"]:::external
        SERP["SERP API"]:::external
        SCRAPER["newspaper3k"]:::external
    end

    subgraph "In-Memory Data Stores"
        S1["Search Terms"]:::data
        S2["URLs List"]:::data
        S3["Raw HTML/Text"]:::data
        S4["Draft Article"]:::data
        S5["Edited Article"]:::data
    end

    subgraph "Documentation"
        R["README.md"]:::doc
        L["LICENSE"]:::doc
    end

    %% Data Flow
    U -->|"provides topic"| A
    A -->|"topic"| B
    B -->|"calls API"| CLAUDE
    CLAUDE -->|"search terms"| S1
    B -->|"search terms"| S1

    S1 --> C
    C -->|"calls API"| SERP
    SERP -->|"results"| S2
    C -->|"URLs list"| S2

    S2 --> D
    D -->|"calls API"| CLAUDE
    CLAUDE -->|"top URLs"| S2
    D -->|"selected URLs"| S2

    S2 --> E
    E -->|"calls library"| SCRAPER
    SCRAPER -->|"raw content"| S3
    E -->|"raw HTML/text"| S3

    S3 --> F
    F -->|"calls API"| CLAUDE
    CLAUDE -->|"draft text"| S4
    F -->|"draft article"| S4

    S4 --> G
    G -->|"calls API"| CLAUDE
    CLAUDE -->|"edited text"| S5
    G -->|"edited article"| S5

    S5 --> H
    H -->|"final article"| U

    %% Click Events
    click A "https://github.com/mshumer/ai-journalist/blob/main/Claude_Journalist.ipynb"
    click B "https://github.com/mshumer/ai-journalist/blob/main/Claude_Journalist.ipynb"
    click C "https://github.com/mshumer/ai-journalist/blob/main/Claude_Journalist.ipynb"
    click D "https://github.com/mshumer/ai-journalist/blob/main/Claude_Journalist.ipynb"
    click E "https://github.com/mshumer/ai-journalist/blob/main/Claude_Journalist.ipynb"
    click F "https://github.com/mshumer/ai-journalist/blob/main/Claude_Journalist.ipynb"
    click G "https://github.com/mshumer/ai-journalist/blob/main/Claude_Journalist.ipynb"
    click H "https://github.com/mshumer/ai-journalist/blob/main/Claude_Journalist.ipynb"
    click CLAUDE "https://github.com/mshumer/ai-journalist/blob/main/Claude_Journalist.ipynb"
    click SERP "https://github.com/mshumer/ai-journalist/blob/main/Claude_Journalist.ipynb"
    click SCRAPER "https://github.com/mshumer/ai-journalist/blob/main/Claude_Journalist.ipynb"
    click S1 "https://github.com/mshumer/ai-journalist/blob/main/Claude_Journalist.ipynb"
    click S2 "https://github.com/mshumer/ai-journalist/blob/main/Claude_Journalist.ipynb"
    click S3 "https://github.com/mshumer/ai-journalist/blob/main/Claude_Journalist.ipynb"
    click S4 "https://github.com/mshumer/ai-journalist/blob/main/Claude_Journalist.ipynb"
    click S5 "https://github.com/mshumer/ai-journalist/blob/main/Claude_Journalist.ipynb"
    click R "https://github.com/mshumer/ai-journalist/blob/main/README.md"
    click L "https://github.com/mshumer/ai-journalist/tree/main/LICENSE"

    %% Styles
    classDef user fill:#a2d2ff,stroke:#000,stroke-width:1px
    classDef notebook fill:#b7e4c7,stroke:#000,stroke-width:1px
    classDef external fill:#ffb703,stroke:#000,stroke-width:1px
    classDef data fill:#adb5bd,stroke:#000,stroke-width:1px
    classDef doc fill:#caf0f8,stroke:#000,stroke-width:1px
```
