# iptv-org/epg architecture

> Architecture diagram of the GitHub repository iptv-org/epg, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-07-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/iptv-org/epg
- Repository: https://github.com/iptv-org/epg

## Overview

1. This repository is a Node.js/TypeScript toolchain for generating TV EPG data, not a single app. The main runtime job is “grab guide data from many site-specific sources and write it into XMLTV/JSON/GZip outputs.”

2. The central execution path is the `epg grab` command under `scripts/commands/epg/grab.ts`. It orchestrates a set of site adapters, reads channel selections from `*.channels.xml` files, applies CLI options like site list, language filtering, concurrency, timeout, delay, proxy, and output format, then produces `guide.xml` and optional `guide.json` / `guide.xml.gz`.

3. Each source site lives in its own folder under `sites/<domain>/` and is effectively an isolated adapter boundary. A site folder usually contains a site config, a site-specific test, one or more `*.channels.xml` mapping files, fixture data under `__data__`, and a short readme. Some sites have multiple regions/locales, which is why many folders emit multiple channel XML files.

4. The `*.channels.xml` files are the main domain input for scraping. They map external source IDs to internal XMLTV channel IDs and language metadata. The channel commands (`parse`, `format`, `edit`, `lint`, `validate`) exist to maintain and verify this mapping layer before scraping runs.

5. The shared implementation lives in `scripts/core`, `scripts/models`, `scripts/constants.ts`, and the `scripts/api.ts` helper. This layer holds the reusable parsing, table/HTML utilities, and typed data models for channels, guides, programs, sites, workers, issues, and API guide records. It is the main boundary between generic orchestration and site-specific scraping logic.

6. Site scraping itself is data-source driven: the repo does not hardcode one transport or one HTML shape. Site configs describe how to load pages/APIs, and the fixtures in `sites/*/__data__` capture real responses for tests. Those fixtures show the supported source types: HTML pages, JSON APIs, XML feeds, text payloads, compressed files, and occasional session/auth artifacts.

7. There is a separate maintenance pipeline for repository metadata. `scripts/commands/guides/*` updates `GUIDES.md` and exports guide metadata, while `scripts/commands/sites/*` initializes new site folders and refreshes `SITES.md`. These commands keep the catalog of available sources and community guides in sync with the codebase.

8. The `workers` commands and `workers.txt` appear to manage a worker catalog used by the scraping system. The presence of `scripts/models/worker.ts`, `tests/.../workers_load`, and `workers.json` fixtures suggests a structured mapping from channels/guides to worker definitions rather than ad hoc scraping.

9. Tests are organized as command-level integration tests under `tests/commands/*` plus per-site tests in each site folder. The `tests/__data__` tree contains input and expected output snapshots for parsing, validation, guide grabbing, site scaffolding, guide export/update, and worker loading. This indicates the repo is heavily snapshot/fixture-driven.

10. The repository also has a small build/ops layer: ESLint/Prettier for formatting, Husky pre-commit checks, GitHub Actions for check/format/update/Docker publish, and a `Dockerfile` plus `pm2.config.js` for containerized scheduling. In Docker, the tool is run as a scheduled guide loader with a mounted `channels.xml` and a public output directory.

11. External dependencies matter architecturally in two places: source metadata comes from the `iptv-org/database` repository, and the produced guide is meant to be consumed as a file or served over HTTP. The README’s examples show local serving via `serve`, and scheduled execution via cron/chronos, so output delivery is file-first rather than API-first.

Main entry points: [`tests/commands/epg/grab.test.ts`](https://github.com/iptv-org/epg/blob/master/tests/commands/epg/grab.test.ts) (Command tests), [`scripts/commands/epg/grab.ts`](https://github.com/iptv-org/epg/blob/master/scripts/commands/epg/grab.ts) (EPG grab), [`sites`](https://github.com/iptv-org/epg/tree/master/sites) (Sites), [`scripts/core/index.ts`](https://github.com/iptv-org/epg/blob/master/scripts/core/index.ts) (Core utils)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_group_commands["Commands"]
  node_node_epg_grab["EPG grab<br/>command<br/>[grab.ts]"]
  node_node_channels_parse["Parse channels<br/>command<br/>[parse.ts]"]
  node_node_channels_format["Format channels<br/>command<br/>[format.ts]"]
  node_node_channels_edit["Edit channels<br/>command<br/>[edit.ts]"]
  node_node_channels_validate["Validate channels<br/>command<br/>[validate.ts]"]
  node_node_guides_update["Update guides<br/>maintenance command<br/>[update.ts]"]
  node_node_guides_export["Export guides<br/>maintenance command<br/>[export.ts]"]
  node_node_sites_update["Update sites<br/>maintenance command<br/>[update.ts]"]
  node_node_sites_init["Init site<br/>scaffold command<br/>[init.ts]"]
  node_node_workers_load["Load workers<br/>catalog command<br/>[load.ts]"]
  node_node_api_load["API load<br/>data command<br/>[load.ts]"]
end

subgraph group_group_shared["Shared layer"]
  node_node_core["Core utils<br/>shared library<br/>[index.ts]"]
  node_node_html_table["HTML table<br/>[htmlTable.ts]"]
  node_node_models["Data models<br/>types library<br/>[index.ts]"]
  node_node_api["API helper<br/>adapter helper<br/>[api.ts]"]
  node_node_default_config["Default config<br/>config base<br/>[default.config.js]"]
end

subgraph group_group_sites["Site adapters"]
  node_node_site_boundary["Sites<br/>adapter boundary"]
  node_node_site_config["Site config<br/>site adapter"]
  node_node_site_channels["Channel map<br/>site input"]
  node_node_site_fixture["Fixtures<br/>site data<br/>[content.json]"]
  node_node_site_test["Site test<br/>[abc.net.au.test.js]"]
end

subgraph group_group_support["Support assets"]
  node_node_tests_commands["Command tests<br/>integration tests<br/>[grab.test.ts]"]
  node_node_tests_snapshots["Snapshots<br/>[base.guide.xml]"]
  node_node_tests_inputs["Test inputs<br/>fixture inputs"]
  node_node_guides_md["GUIDES.md<br/>catalog output<br/>[GUIDES.md]"]
  node_node_sites_md["SITES.md<br/>catalog output<br/>[SITES.md]"]
  node_node_workers_txt["workers.txt<br/>worker catalog<br/>[workers.txt]"]
end

node_node_epg_grab -->|"uses defaults"| node_node_default_config
node_node_epg_grab -->|"orchestrates"| node_node_core
node_node_epg_grab -->|"structures"| node_node_models
node_node_epg_grab -->|"exports"| node_node_api
node_node_epg_grab -->|"runs adapters"| node_node_site_boundary
node_node_epg_grab -->|"reads mappings"| node_node_site_channels
node_node_channels_parse -->|"parses with"| node_node_core
node_node_channels_format -->|"formats with"| node_node_core
node_node_channels_edit -->|"edits with"| node_node_core
node_node_channels_validate -->|"validates against"| node_node_models
node_node_guides_update -->|"writes"| node_node_guides_md
node_node_guides_export -->|"exports via"| node_node_api
node_node_sites_update -->|"writes"| node_node_sites_md
node_node_sites_init -->|"scaffolds"| node_node_site_boundary
node_node_workers_load -->|"reads"| node_node_workers_txt
node_node_workers_load -->|"builds"| node_node_models
node_node_api_load -->|"hydrates"| node_node_api
node_node_api_load -->|"uses models"| node_node_models
node_node_site_boundary -->|"contains"| node_node_site_config
node_node_site_boundary -->|"contains"| node_node_site_channels
node_node_site_boundary -->|"contains"| node_node_site_fixture
node_node_site_boundary -->|"contains"| node_node_site_test
node_node_site_test -->|"verifies"| node_node_site_fixture
node_node_tests_commands -->|"consumes"| node_node_tests_inputs
node_node_tests_commands -->|"compares"| node_node_tests_snapshots
node_node_tests_commands -->|"covers"| node_node_epg_grab
node_node_tests_commands -->|"covers"| node_node_channels_parse
node_node_tests_commands -->|"covers"| node_node_guides_update
node_node_tests_commands -->|"covers"| node_node_sites_update
node_node_tests_commands -->|"covers"| node_node_workers_load

click node_node_epg_grab "https://github.com/iptv-org/epg/blob/master/scripts/commands/epg/grab.ts"
click node_node_channels_parse "https://github.com/iptv-org/epg/blob/master/scripts/commands/channels/parse.ts"
click node_node_channels_format "https://github.com/iptv-org/epg/blob/master/scripts/commands/channels/format.ts"
click node_node_channels_edit "https://github.com/iptv-org/epg/blob/master/scripts/commands/channels/edit.ts"
click node_node_channels_validate "https://github.com/iptv-org/epg/blob/master/scripts/commands/channels/validate.ts"
click node_node_guides_update "https://github.com/iptv-org/epg/blob/master/scripts/commands/guides/update.ts"
click node_node_guides_export "https://github.com/iptv-org/epg/blob/master/scripts/commands/guides/export.ts"
click node_node_sites_update "https://github.com/iptv-org/epg/blob/master/scripts/commands/sites/update.ts"
click node_node_sites_init "https://github.com/iptv-org/epg/blob/master/scripts/commands/sites/init.ts"
click node_node_workers_load "https://github.com/iptv-org/epg/blob/master/scripts/commands/workers/load.ts"
click node_node_api_load "https://github.com/iptv-org/epg/blob/master/scripts/commands/api/load.ts"
click node_node_core "https://github.com/iptv-org/epg/blob/master/scripts/core/index.ts"
click node_node_html_table "https://github.com/iptv-org/epg/blob/master/scripts/core/htmlTable.ts"
click node_node_models "https://github.com/iptv-org/epg/blob/master/scripts/models/index.ts"
click node_node_api "https://github.com/iptv-org/epg/blob/master/scripts/api.ts"
click node_node_default_config "https://github.com/iptv-org/epg/blob/master/scripts/default.config.js"
click node_node_site_boundary "https://github.com/iptv-org/epg/tree/master/sites"
click node_node_site_config "https://github.com/iptv-org/epg/blob/master/sites/abc.net.au/abc.net.au.config.js"
click node_node_site_channels "https://github.com/iptv-org/epg/blob/master/sites/abc.net.au/abc.net.au_adl.channels.xml"
click node_node_site_fixture "https://github.com/iptv-org/epg/blob/master/sites/abc.net.au/__data__/content.json"
click node_node_site_test "https://github.com/iptv-org/epg/blob/master/sites/abc.net.au/abc.net.au.test.js"
click node_node_tests_commands "https://github.com/iptv-org/epg/blob/master/tests/commands/epg/grab.test.ts"
click node_node_tests_snapshots "https://github.com/iptv-org/epg/blob/master/tests/__data__/expected/epg_grab/base.guide.xml"
click node_node_tests_inputs "https://github.com/iptv-org/epg/blob/master/tests/__data__/input/epg_grab/custom.channels.xml"
click node_node_guides_md "https://github.com/iptv-org/epg/blob/master/GUIDES.md"
click node_node_sites_md "https://github.com/iptv-org/epg/blob/master/SITES.md"
click node_node_workers_txt "https://github.com/iptv-org/epg/blob/master/workers.txt"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_node_epg_grab,node_node_channels_parse,node_node_channels_format,node_node_channels_edit,node_node_channels_validate,node_node_guides_update,node_node_guides_export,node_node_sites_update,node_node_sites_init,node_node_workers_load,node_node_api_load toneBlue
class node_node_core,node_node_html_table,node_node_models,node_node_api,node_node_default_config toneAmber
class node_node_site_boundary,node_node_site_config,node_node_site_channels,node_node_site_fixture,node_node_site_test toneMint
class node_node_tests_commands,node_node_tests_snapshots,node_node_tests_inputs,node_node_guides_md,node_node_sites_md,node_node_workers_txt toneRose
```

## Components

### Commands

CLI entrypoints and maintenance jobs

- [EPG grab](https://github.com/iptv-org/epg/blob/master/scripts/commands/epg/grab.ts): command · Main runtime orchestration for scraping and guide generation · `scripts/commands/epg/grab.ts`
- [Parse channels](https://github.com/iptv-org/epg/blob/master/scripts/commands/channels/parse.ts): command · Reads and normalizes channel mapping XML · `scripts/commands/channels/parse.ts`
- [Format channels](https://github.com/iptv-org/epg/blob/master/scripts/commands/channels/format.ts): command · Rewrites channel mapping XML in canonical form · `scripts/commands/channels/format.ts`
- [Edit channels](https://github.com/iptv-org/epg/blob/master/scripts/commands/channels/edit.ts): command · Assists manual channel mapping updates · `scripts/commands/channels/edit.ts`
- [Validate channels](https://github.com/iptv-org/epg/blob/master/scripts/commands/channels/validate.ts): command · Checks mapping integrity against known data · `scripts/commands/channels/validate.ts`
- [Update guides](https://github.com/iptv-org/epg/blob/master/scripts/commands/guides/update.ts): maintenance command · Refreshes GUIDES.md metadata · `scripts/commands/guides/update.ts`
- [Export guides](https://github.com/iptv-org/epg/blob/master/scripts/commands/guides/export.ts): maintenance command · Exports guide metadata to JSON · `scripts/commands/guides/export.ts`
- [Update sites](https://github.com/iptv-org/epg/blob/master/scripts/commands/sites/update.ts): maintenance command · Refreshes SITES.md and site catalog data · `scripts/commands/sites/update.ts`
- [Init site](https://github.com/iptv-org/epg/blob/master/scripts/commands/sites/init.ts): scaffold command · Creates a new site folder template · `scripts/commands/sites/init.ts`
- [Load workers](https://github.com/iptv-org/epg/blob/master/scripts/commands/workers/load.ts): catalog command · Loads worker definitions from text and JSON inputs · `scripts/commands/workers/load.ts`
- [API load](https://github.com/iptv-org/epg/blob/master/scripts/commands/api/load.ts): data command · Loads guide metadata into API-facing records · `scripts/commands/api/load.ts`

### Shared layer

Reusable parsing, models, and API helpers

- [Core utils](https://github.com/iptv-org/epg/blob/master/scripts/core/index.ts): shared library · Reusable parsing, table, and execution utilities · `scripts/core/index.ts`
- [HTML table](https://github.com/iptv-org/epg/blob/master/scripts/core/htmlTable.ts): utility · Table rendering/parsing helper used by commands · `scripts/core/htmlTable.ts`
- [Data models](https://github.com/iptv-org/epg/blob/master/scripts/models/index.ts): types library · Typed models for channels, guides, programs, sites, workers, and issues · `scripts/models/index.ts`
- [API helper](https://github.com/iptv-org/epg/blob/master/scripts/api.ts): adapter helper · Transforms repository data into API guide records · `scripts/api.ts`
- [Default config](https://github.com/iptv-org/epg/blob/master/scripts/default.config.js): config base · Shared runtime defaults for command behavior · `scripts/default.config.js`

### Site adapters

Per-source scraping boundaries and fixtures

- [Sites](https://github.com/iptv-org/epg/tree/master/sites): adapter boundary · Per-domain site folders containing isolated scraping logic · `sites`
- [Site config](https://github.com/iptv-org/epg/blob/master/sites/abc.net.au/abc.net.au.config.js): site adapter · Domain-specific scraping rules and transport setup · `sites/abc.net.au/abc.net.au.config.js`
- [Channel map](https://github.com/iptv-org/epg/blob/master/sites/abc.net.au/abc.net.au_adl.channels.xml): site input · Region and language-specific XMLTV channel mappings · `sites/abc.net.au/abc.net.au_adl.channels.xml`
- [Fixtures](https://github.com/iptv-org/epg/blob/master/sites/abc.net.au/__data__/content.json): site data · Captured HTML/JSON/XML responses for adapter tests · `sites/abc.net.au/__data__/content.json`
- [Site test](https://github.com/iptv-org/epg/blob/master/sites/abc.net.au/abc.net.au.test.js): site test · Per-site integration test for scraper behavior · `sites/abc.net.au/abc.net.au.test.js`

### Support assets

Snapshots, catalogs, and repository ops

- [Command tests](https://github.com/iptv-org/epg/blob/master/tests/commands/epg/grab.test.ts): integration tests · Snapshot-driven tests for CLI commands · `tests/commands/epg/grab.test.ts`
- [Snapshots](https://github.com/iptv-org/epg/blob/master/tests/__data__/expected/epg_grab/base.guide.xml): fixture snapshots · Expected inputs and outputs for command behavior · `tests/__data__/expected/epg_grab/base.guide.xml`
- [Test inputs](https://github.com/iptv-org/epg/blob/master/tests/__data__/input/epg_grab/custom.channels.xml): fixture inputs · Input XML, JSON, and catalog files used by command tests · `tests/__data__/input/epg_grab/custom.channels.xml`
- [GUIDES.md](https://github.com/iptv-org/epg/blob/master/GUIDES.md): catalog output · Generated guide catalog document · `GUIDES.md`
- [SITES.md](https://github.com/iptv-org/epg/blob/master/SITES.md): catalog output · Generated site catalog document · `SITES.md`
- [workers.txt](https://github.com/iptv-org/epg/blob/master/workers.txt): worker catalog · Text catalog consumed by worker loading · `workers.txt`

## Connections

- EPG grab → Default config: uses defaults · Applies shared runtime settings and CLI overrides
- EPG grab → Core utils: orchestrates · Uses shared parsing, concurrency, and IO helpers
- EPG grab → Data models: structures · Builds guides, programs, sites, and issue records
- EPG grab → API helper: exports · Can emit API-style guide records in addition to files
- EPG grab → Sites: runs adapters · Loads site-specific scraping implementations
- EPG grab → Channel map: reads mappings · Consumes selected channel XML mappings for scraping
- Parse channels → Core utils: parses with · Uses shared XML and text utilities
- Format channels → Core utils: formats with · Relies on shared normalization and table utilities
- Edit channels → Core utils: edits with · Uses shared helpers for interactive channel maintenance
- Validate channels → Data models: validates against · Checks channel and feed identities in typed data
- Update guides → GUIDES.md: writes · Regenerates the guide catalog document
- Export guides → API helper: exports via · Produces JSON guide metadata from repository data
- Update sites → SITES.md: writes · Regenerates the site catalog document
- Init site → Sites: scaffolds · Creates new site adapter folders from templates
- Load workers → workers.txt: reads · Consumes the worker catalog text file
- Load workers → Data models: builds · Materializes worker records from catalog data
- API load → API helper: hydrates · Loads external guide metadata into API records
- API load → Data models: uses models · Converts loaded data into typed records
- Sites → Site config: contains · Each site folder centers on a domain config
- Sites → Channel map: contains · Channel XML mappings live alongside the adapter
- Sites → Fixtures: contains · Fixtures capture real source responses
- Sites → Site test: contains · Each adapter has a site-specific test
- Site test → Fixtures: verifies · Tests assert behavior against captured responses
- Command tests → Test inputs: consumes · Uses command input snapshots and sample configs
- Command tests → Snapshots: compares · Checks generated outputs against expected snapshots
- Command tests → EPG grab: covers · Validates the main guide-grabbing workflow
- Command tests → Parse channels: covers · Validates channel parsing behavior
- Command tests → Update guides: covers · Validates guide metadata maintenance
- Command tests → Update sites: covers · Validates site catalog maintenance
- Command tests → Load workers: covers · Validates worker catalog loading
