How mshumer/ai-journalist works
Project Type and Purpose
• This is a Python-based experimental AI tool implemented as a Jupyter notebook (Claude_Journalist.ipynb).
• Purpose: Given a topic, it orchestrates a research-and-writing pipeline using the Anthropic Claude 3 AI model, a SERP search API, and the newspaper3k scraper to produce and edit a publish-quality article.
File Structure Analysis
• Top-level:
• Claude_Journalist.ipynb – the single notebook containing all code, prompts, and orchestration.
• README.md – project overview, workflow steps, and prerequisites.
• LICENSE – MIT license.
• No separate “src” or “services” directories; the notebook is the entry point for orchestration of the pipeline.
Key Workflow Components (from README)
1. User prompt for topic
2. Prompt Claude to generate search terms (Claude 3 “Haiku”)
3. Perform web searches via SERP API
4. Prompt Claude to select top URLs
5. Scrape article text with newspaper3k
6. Prompt Claude to draft article
7. Prompt Claude to edit/rewrite the draft
8. Output final and edited articles
Main Components
a. User Interface
• Jupyter Notebook (CLI/interactive cell execution)
• Topic input cell, result display cell
b. Orchestrator / Controller
• Python code within notebook coordinating each step
c. External Services
• Anthropic Claude 3 API (two roles: researcher/generator & editor)
• SERP API (Google/Bing search wrapper)
d. Web Scraper
• newspaper3k library fetching HTML/article content
e. Data Stores (transient/in-memory)
• Search terms list, search results, raw HTML/text, draft articles, edited articles
f. Output
• Printed article(s) in notebook, optionally saved to file
Relationships & Data Flow
1. User → Notebook: “Topic”
2. Notebook → Claude 3 (API call #1): generates search terms
3. Notebook → SERP API: fetch search results for each term
4. Notebook → Claude 3 (API call #2): filters/chooses top URLs
5. Notebook → newspaper3k: scrapes selected URLs
6. Notebook → Claude 3 (API call #3): drafts article from scraped text
7. Notebook → Claude 3 (API call #4): edits/revises draft
8. Notebook → User: displays final output
Architectural Patterns & Principles
• Pipeline / Orchestration pattern: linear, stage-by-stage data flow
• Single-process, synchronous calls (blocking notebook cells)
• Externalized logic in prompts to Claude—keeps code generic, prompt-driven design
• Transient in-memory storage in Python variables—no persistent DB
Technologies & Frameworks
• Python 3.x within Jupyter
• Anthropic API client (Claude 3)
• SERP API client (requests wrapper)
• newspaper3k for scraping/parsing
• Standard libraries (requests, json, etc.)