# mdwiki-td/publish architecture

> Architecture diagram of the GitHub repository mdwiki-td/publish, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mdwiki-td/publish
- Repository: https://github.com/mdwiki-td/publish

## Overview

##### Project Type and Purpose
- This project is a specialized, full-process publishing tool for Wikipedia articles translated by an external tool (ContentTranslation), and it focuses on pre-processing translated wikitext before publishing.
- The README explains that its main job is to refine the wikitext (fix references, enhance infoboxes, add categories, etc.) and then perform the final API call to publish the article and link it with Wikidata.

##### Analyzing the File Structure
- Top-level PHP scripts (main.php, index.php, do_edit.php, etc.) indicate that PHP is the primary language used.
- The presence of multiple directories like:
- bots: Contains scripts like index.php and others (mdwiki_sql.php, wd.php) likely related to automated tasks,
- tests: Contains test scripts for verifying functionality,
- textfixes: Contains several files (Citation.php, md_cat.php, text_fix.php, etc.) that are clearly focused on making text changes to the wikitext.
- Files such as config.php, get_token.php, token.php, and include.php are likely used for configuration, authentication, or common helper functions.
- The file structure also shows integration points (e.g., “fix_refs” mentioned in the README), suggesting that this repository might rely on or interact with external repositories to achieve certain pre-processing tasks.

##### Main Components
a. Front-End / Entry Point:
- main.php: Acts as the starting script for processing the publishing workflow.
- index.php and do_edit.php might serve as additional entry points or web hooks, if the tool is web-accessible.
b. Text Pre-processing Modules:
- Integration with the external “fix_refs” functionality (used to adjust references, infoboxes, categories, etc.)
- The “textfixes” directory modules handle multiple aspects of text transformation.
c. Configuration and Helper Files:
- config.php, include.php, and token.php for managing settings, shared functions, and authentication tokens.
d. Auxiliary/Bot Operations:
- The “bots” directory contains scripts that may perform periodic or automated tasks related to updating or managing the wikitext.
e. Testing:
- The “tests” directory which includes test.php to validate core functionalities during development.
f. External Systems:
- Wikipedia API: The system prepares and makes requests to the Wikipedia API (using prepared parameters) to edit articles.
- Wikidata linking is part of a successful edit process.

##### Relationships and Interactions
- The pre-processed text, combined with configuration details and API parameter preparation (handled in functions like prepareApiParams), flows into the Wikipedia API call.
- Post successful API calls, further processing like linking to Wikidata is triggered.
- Log flows and error handling (through functions like handleNoAccess and processEdit) should be illustrated.
- Supporting modules like bots might interact with the main workflow to automate certain actions.

##### Architectural Patterns and Design Principles
- Although the overall structure is relatively flat, the separation of concerns is visible: distinct directories for text pre-processing (textfixes), automated tasks (bots), testing, and main workflow (main.php).
- The pipeline design pattern can be implied as the text goes through several transformation steps before being sent to the API.
- Dependency on external repositories (fix_refs) points to an integration pattern where this repository acts as an orchestration layer over several services/modules.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Input Sources
    A["Translated Wikitext Input"]:::input
    B["all_pages_revids.json"]:::input

    %% Front-End / Entry Points
    subgraph "Front-End / Entry Points"
        MP["main.php"]:::entrypoint
        IP["index.php"]:::entrypoint
        DEP["do_edit.php"]:::entrypoint
    end

    %% Configuration & Helpers
    subgraph "Configuration & Helpers"
        CP["config.php"]:::config
        INC["include.php"]:::config
        TP["token.php"]:::config
        GT["get_token.php"]:::config
        HP["helps.php"]:::config
        TFINC["textfixes/include.php"]:::config
    end

    %% Text Pre-processing Modules
    subgraph "Text Pre-processing Modules"
        FR["fix_refs (Integration)"]:::processing
        TF["Textfixes Modules (Citation, md_cat, text_fix, text_fix_refs, index.php)"]:::processing
    end

    %% API Preparation and External API
    AP["API Preparation"]:::processing
    subgraph "External API"
        API["Wikipedia API"]:::external
        WD["Wikidata Linking"]:::external
    end

    %% Auxiliary/Bot Services
    subgraph "Auxiliary/Bot Services"
        BOT["bots (bots/index.php, mdwiki_sql.php, wd.php)"]:::bot
    end

    %% Testing & Quality Assurance
    subgraph "Testing & Quality Assurance"
        TEST["tests (test.php)"]:::test
    end

    %% Data Flow Connections
    A --> MP
    B --> MP

    CP --> MP
    INC --> MP
    TP --> MP
    GT --> MP
    HP --> MP
    TFINC --> MP

    MP -->|"DataTransformation"| FR
    MP -->|"DataTransformation"| TF

    FR -->|"ProcessedText"| AP
    TF -->|"ProcessedText"| AP

    AP -->|"APIRequest"| API
    API -->|"SuccessfulEdit"| WD

    MP -.->|"ErrorHandling"| BOT
    MP -.->|"Validation"| TEST

    %% Styles
    classDef entrypoint fill:#f9f,stroke:#333,stroke-width:2px;
    classDef config fill:#bbf,stroke:#333,stroke-width:2px;
    classDef processing fill:#bfb,stroke:#333,stroke-width:2px;
    classDef external fill:#ffe,stroke:#333,stroke-width:2px;
    classDef bot fill:#fbb,stroke:#333,stroke-width:2px;
    classDef test fill:#efc,stroke:#333,stroke-width:2px;
    classDef input fill:#fcc,stroke:#333,stroke-width:2px;

    %% Click Events for Component Mapping
    click MP "https://github.com/mdwiki-td/publish/blob/main/main.php"
    click IP "https://github.com/mdwiki-td/publish/blob/main/index.php"
    click DEP "https://github.com/mdwiki-td/publish/blob/main/do_edit.php"
    click FR "https://github.com/mdwiki-td/publish/tree/main/fix_refs"
    click TF "https://github.com/mdwiki-td/publish/tree/main/textfixes"
    click CP "https://github.com/mdwiki-td/publish/blob/main/config.php"
    click INC "https://github.com/mdwiki-td/publish/blob/main/include.php"
    click TP "https://github.com/mdwiki-td/publish/blob/main/token.php"
    click GT "https://github.com/mdwiki-td/publish/blob/main/get_token.php"
    click HP "https://github.com/mdwiki-td/publish/blob/main/helps.php"
    click TFINC "https://github.com/mdwiki-td/publish/blob/main/textfixes/include.php"
    click BOT "https://github.com/mdwiki-td/publish/tree/main/bots"
    click TEST "https://github.com/mdwiki-td/publish/blob/main/tests/test.php"
```
