# amal-thomas-dev/news-scraper-website architecture

> Architecture diagram of the GitHub repository amal-thomas-dev/news-scraper-website, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-08-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/amal-thomas-dev/news-scraper-website
- Repository: https://github.com/amal-thomas-dev/news-scraper-website

## Overview

#### Purpose and entry points

Crewe News Scraper is a two-process application that collects current articles from Crewe Nub News and presents them in a Vue UI. The backend exposes `GET /api/articles` on port 5001; the frontend Vite app runs separately on port 5173.

- Backend startup and HTTP routing: `backend/src/index.ts`
- Frontend browser bootstrap: `frontend/src/main.js`
- Root UI composition: `frontend/src/App.vue`

#### Backend API boundary

The Express service is the sole boundary between the browser and the publisher’s site. It owns article retrieval, extraction, normalization, and API response behavior rather than exposing scraping implementation to the frontend.

- Server/API endpoint: `backend/src/index.ts`
- Shared article/request data shapes: `backend/src/types.ts`

#### Scraping pipeline

Article collection is centered in the scraper module. Its declared stack—Axios, Cheerio, and Puppeteer—indicates layered retrieval: lightweight HTTP/HTML parsing where possible, with browser automation available for pages requiring rendered content. The scraper transforms publisher markup into the backend’s article model before it reaches the API.

- Scraper orchestration and publisher integration: `backend/src/scraper.ts`
- Parsing/normalization support: `backend/src/utils.ts`
- Output contracts: `backend/src/types.ts`

#### External dependency and operational boundary

Crewe Nub News is an external, mutable upstream source. Scraping is therefore the main reliability boundary: upstream HTML changes, availability, anti-bot behavior, or JavaScript-rendered pages can affect `/api/articles`. Puppeteer also introduces a browser-runtime dependency beyond the normal Node/Express process.

- Backend package/runtime definition: `backend/package.json`
- Scraping integration: `backend/src/scraper.ts`

#### Frontend application flow

The Vue application mounts from `main.js`, composes the application shell in `App.vue`, and uses the Home view as the article-listing screen. The expected primary flow is browser load → Home fetches the backend endpoint → returned article records are rendered as reusable cards.

- Vue bootstrap: `frontend/src/main.js`
- Application shell: `frontend/src/App.vue`
- Primary data-fetching/listing view: `frontend/src/views/Home.vue`

#### Presentation components

The UI separates page-level article state from reusable presentation units. `Header` provides shared top-level chrome, while `ArticleCard` renders individual normalized article records. This keeps publisher-specific extraction details behind the API contract.

- Shared navigation/header: `frontend/src/components/Header.vue`
- Per-article renderer: `frontend/src/components/ArticleCard.vue`
- Global visual styles: `frontend/src/index.css`

#### Material runtime and build infrastructure

Backend execution is Node.js with Express and TypeScript compilation/runtime configuration. Frontend execution is Vue 3 through Vite, with Tailwind/PostCSS handling styling. There is no repository evidence of a database, queue, cache, authentication service, or deployment infrastructure; article data is retrieved from the upstream source at request-time or according to scraper behavior.

- TypeScript compiler settings: `backend/tsconfig.json`
- Vite runtime/build configuration: `frontend/vite.config.js`
- Tailwind styling configuration: `frontend/tailwind.config.js`

Main entry points: [`backend/src/scraper.ts`](https://github.com/amal-thomas-dev/news-scraper-website/blob/main/backend/src/scraper.ts) (Scraper orchestration), [`frontend/src/App.vue`](https://github.com/amal-thomas-dev/news-scraper-website/blob/main/frontend/src/App.vue) (App shell), [`backend/src/index.ts`](https://github.com/amal-thomas-dev/news-scraper-website/blob/main/backend/src/index.ts) (GET /api/articles), [`frontend/src/views/Home.vue`](https://github.com/amal-thomas-dev/news-scraper-website/blob/main/frontend/src/views/Home.vue) (Home article listing)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_frontend["Vue frontend"]
  node_vite{{"Vite + Vue runtime<br/>frontend runtime<br/>[vite.config.js]"}}
  node_bootstrap["Vue bootstrap<br/>entry point<br/>[main.js]"]
  node_app["App shell<br/>Vue component<br/>[App.vue]"]
  node_home["Home article listing<br/>Vue view<br/>[Home.vue]"]
  node_header["Header<br/>Vue component<br/>[Header.vue]"]
  node_article_card["Article card<br/>Vue component<br/>[ArticleCard.vue]"]
  node_styles["Global styles<br/>CSS<br/>[index.css]"]
end

subgraph group_backend["Node backend"]
  node_express_api{{"GET /api/articles<br/>Express server<br/>[index.ts]"}}
  node_contracts["Article contracts<br/>TypeScript types<br/>[types.ts]"]
  node_scraper["Scraper orchestration<br/>scraping service<br/>[scraper.ts]"]
  node_normalizer["Parsing and normalization<br/>scraping utilities<br/>[utils.ts]"]
  node_http_parser["Axios + Cheerio<br/>HTML retrieval/parser<br/>[scraper.ts]"]
  node_puppeteer{{"Puppeteer browser runtime<br/>rendered-page retrieval<br/>[scraper.ts]"}}
  node_node_runtime{{"Node + TypeScript runtime<br/>backend runtime<br/>[package.json]"}}
end

node_browser(("Browser<br/>client runtime"))
node_publisher(("Crewe Nub News<br/>external publisher"))

node_browser -->|"loads UI"| node_vite
node_vite -->|"serves"| node_bootstrap
node_bootstrap -->|"mounts"| node_app
node_app -->|"composes"| node_header
node_app -->|"renders"| node_home
node_styles -->|"styles"| node_app
node_home -->|"GET /api/articles"| node_express_api
node_home -->|"renders articles"| node_article_card
node_node_runtime -->|"runs"| node_express_api
node_express_api -->|"collects articles"| node_scraper
node_express_api -->|"responds as"| node_contracts
node_scraper -->|"uses"| node_http_parser
node_scraper -.->|"uses when rendering is needed"| node_puppeteer
node_http_parser -->|"fetches and parses"| node_publisher
node_puppeteer -.->|"loads rendered pages"| node_publisher
node_scraper -->|"normalizes markup"| node_normalizer
node_normalizer -->|"produces"| node_contracts

click node_vite "https://github.com/amal-thomas-dev/news-scraper-website/blob/main/frontend/vite.config.js"
click node_bootstrap "https://github.com/amal-thomas-dev/news-scraper-website/blob/main/frontend/src/main.js"
click node_app "https://github.com/amal-thomas-dev/news-scraper-website/blob/main/frontend/src/App.vue"
click node_home "https://github.com/amal-thomas-dev/news-scraper-website/blob/main/frontend/src/views/Home.vue"
click node_header "https://github.com/amal-thomas-dev/news-scraper-website/blob/main/frontend/src/components/Header.vue"
click node_article_card "https://github.com/amal-thomas-dev/news-scraper-website/blob/main/frontend/src/components/ArticleCard.vue"
click node_styles "https://github.com/amal-thomas-dev/news-scraper-website/blob/main/frontend/src/index.css"
click node_express_api "https://github.com/amal-thomas-dev/news-scraper-website/blob/main/backend/src/index.ts"
click node_contracts "https://github.com/amal-thomas-dev/news-scraper-website/blob/main/backend/src/types.ts"
click node_scraper "https://github.com/amal-thomas-dev/news-scraper-website/blob/main/backend/src/scraper.ts"
click node_normalizer "https://github.com/amal-thomas-dev/news-scraper-website/blob/main/backend/src/utils.ts"
click node_http_parser "https://github.com/amal-thomas-dev/news-scraper-website/blob/main/backend/src/scraper.ts"
click node_puppeteer "https://github.com/amal-thomas-dev/news-scraper-website/blob/main/backend/src/scraper.ts"
click node_node_runtime "https://github.com/amal-thomas-dev/news-scraper-website/blob/main/backend/package.json"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_vite,node_bootstrap,node_app,node_home,node_header,node_article_card,node_styles toneBlue
class node_express_api,node_contracts,node_scraper,node_normalizer,node_http_parser,node_puppeteer,node_node_runtime toneAmber
class node_browser,node_publisher toneNeutral
```

## Components

### Vue frontend

Browser application served by Vite.

- [Vite + Vue runtime](https://github.com/amal-thomas-dev/news-scraper-website/blob/main/frontend/vite.config.js): frontend runtime · `frontend/vite.config.js`
- [Vue bootstrap](https://github.com/amal-thomas-dev/news-scraper-website/blob/main/frontend/src/main.js): entry point · `frontend/src/main.js`
- [App shell](https://github.com/amal-thomas-dev/news-scraper-website/blob/main/frontend/src/App.vue): Vue component · `frontend/src/App.vue`
- [Home article listing](https://github.com/amal-thomas-dev/news-scraper-website/blob/main/frontend/src/views/Home.vue): Vue view · Fetches and owns article-list state. · `frontend/src/views/Home.vue`
- [Header](https://github.com/amal-thomas-dev/news-scraper-website/blob/main/frontend/src/components/Header.vue): Vue component · `frontend/src/components/Header.vue`
- [Article card](https://github.com/amal-thomas-dev/news-scraper-website/blob/main/frontend/src/components/ArticleCard.vue): Vue component · Renders one normalized article. · `frontend/src/components/ArticleCard.vue`
- [Global styles](https://github.com/amal-thomas-dev/news-scraper-website/blob/main/frontend/src/index.css): CSS · `frontend/src/index.css`

### Node backend

Express API and scraping pipeline.

- [GET /api/articles](https://github.com/amal-thomas-dev/news-scraper-website/blob/main/backend/src/index.ts): Express server · Browser-facing API on port 5001. · `backend/src/index.ts`
- [Article contracts](https://github.com/amal-thomas-dev/news-scraper-website/blob/main/backend/src/types.ts): TypeScript types · Shared request and article data shapes. · `backend/src/types.ts`
- [Scraper orchestration](https://github.com/amal-thomas-dev/news-scraper-website/blob/main/backend/src/scraper.ts): scraping service · Retrieves publisher pages and produces article records. · `backend/src/scraper.ts`
- [Parsing and normalization](https://github.com/amal-thomas-dev/news-scraper-website/blob/main/backend/src/utils.ts): scraping utilities · `backend/src/utils.ts`
- [Axios + Cheerio](https://github.com/amal-thomas-dev/news-scraper-website/blob/main/backend/src/scraper.ts): HTML retrieval/parser · Lightweight retrieval and publisher markup parsing. · `backend/src/scraper.ts`
- [Puppeteer browser runtime](https://github.com/amal-thomas-dev/news-scraper-website/blob/main/backend/src/scraper.ts): rendered-page retrieval · Fallback capability for JavaScript-rendered pages. · `backend/src/scraper.ts`
- [Node + TypeScript runtime](https://github.com/amal-thomas-dev/news-scraper-website/blob/main/backend/package.json): backend runtime · `backend/package.json`

### Other components

- **Browser**: client runtime
- **Crewe Nub News**: external publisher · Mutable upstream HTML and availability boundary.

## Connections

- Browser → Vite + Vue runtime: loads UI
- Vite + Vue runtime → Vue bootstrap: serves
- Vue bootstrap → App shell: mounts
- App shell → Header: composes
- App shell → Home article listing: renders
- Global styles → App shell: styles
- Home article listing → GET /api/articles: GET /api/articles
- Home article listing → Article card: renders articles
- Node + TypeScript runtime → GET /api/articles: runs
- GET /api/articles → Scraper orchestration: collects articles
- GET /api/articles → Article contracts: responds as
- Scraper orchestration → Axios + Cheerio: uses
- Scraper orchestration → Puppeteer browser runtime: uses when rendering is needed
- Axios + Cheerio → Crewe Nub News: fetches and parses
- Puppeteer browser runtime → Crewe Nub News: loads rendered pages
- Scraper orchestration → Parsing and normalization: normalizes markup
- Parsing and normalization → Article contracts: produces