How mdwiki-td/fix_refs works
Based on our analysis of the file tree and README, this repository is a specialized tool for processing and “fixing” Wikipedia article references in wikitext format. It is a PHP-based script tool where the main entry point (index.php) acts as the orchestrator for calling several parsing and bot modules, applying a series of text fixes to page content.
Project’s Main Components
• Core Controller: The main script (index.php at the root and src/index.php) acts as the entry point. This component is responsible for reading the configuration (via the fixwikirefs.json file) and coordinating the execution of the text-fixing routines.
• WikiParse Module: In the src/WikiParse directory, you have several PHP files and subdirectories that handle the parsing of various wikipedia syntax elements.
– The “parsewiki” folder contains submodules for parsing categories, citations, external links, internal links, tables, and templates.
– The DataModel subfolder defines objects (like Citation, ExternalLink, InternalLink, Table, Template) that represent the parsed elements.
– Other files (Category.php, Template.php, etc.) are responsible for specific parsing or formatting functionalities.
• Bots and Automated Scripts: Under src/bots there are several utilities (e.g., es_months.php, es_refs.php, fix_pt_months.php, remove_duplicate_refs.php) that perform language-specific adjustments and other automated fixes.
• Additional Utility Scripts: Other files like infobox.php, md_cat.php, and sw.php provide support functionality (e.g., expanding infobox parameters, ensuring MDWiki category presence, or moving dots) which are part of the overall processing.
• Configuration Management: Though not visible in the file tree, the README mentions reading a JSON config file (fixwikirefs.json) which the system uses to control behavior.
Relationships and Interactions
• Start at the Core Controller:
• The main script loads configuration settings.
• It calls the fix_page_here function to begin the fixing process.
• The fix_page_here function then delegates tasks to the WikiParse module for parsing the wikitext.
• The WikiParse module, including the parsewiki subcomponents, processes the text to identify elements such as citations, links, and templates.
• In parallel or sequence, various Bots (from src/bots) are invoked to apply language-specific rules and remove duplicate references.
• The processed data flows back to the main script which finally returns the fixed text.
Architectural Patterns or Design Principles
• Modular Structure: The design is modular with clear separations between parsing (WikiParse), business logic (the core and fix functions), and utility bots.
• Data-Driven Processing: The use of a configuration file (fixwikirefs.json) allows the processing routines to be controlled externally, following a configuration-driven design.
• Layered Approach: There is a clear sequence from the input stage (reading and configuration), parsing (data models and parser classes), rule application (bots and language-specific fixes), and final output.
Technologies and Libraries
• PHP is the main language.
• The Data Models (in parsewiki/DataModel) are implemented as PHP classes representing different parts of the wikitext.
• GitHub workflows (found under .github/workflows) indicate that continuous integration is used for updates, which is part of the maintenance and deployment layer.
Instructing the Principal Software Engineer
• Branch out from this central component into three main sub-components:
– “Bot Module” including the various scripts under src/bots (es_months, remove_duplicate_refs, etc.) which are responsible for specific textual fixes.
– “Utility Functions” representing supporting scripts (such as infobox.php, md_cat.php, etc.).