How wikiar/newapi works
Main Components
• API Client/Wrapper Interface: This is the high-level module (exposed by the “newapi” calls shown in the README) that provides the public methods, e.g., Login, MainPage, CatDepth, and NEW_API.
• Login Module: Represented by components such as “super_login.py” and the sub-packages under “super/login_bots”. This part is responsible for authentication, managing cookies, sessions, and using the MediaWiki client (mwclient).
• Page Operations Module: Including components such as “page.py”, “wiki_page.py”, “mdwiki_page.py”, “ncc_page.py”, and sub-components in “super/page_bots”. This module handles page existence checks, text retrieval, editing, template processing, and various page queries.
• Bot Functionality Modules: These include “botEdit.py”, “db_bot.py”, “pymysql_bot.py”, the “botapi_bots” under “super”, and the “login_bots” and “page_bots” folders. They encapsulate automation routines and error-handling for specific automated tasks.
• Utility and Helper Functions: Modules like “except_err.py”, “pformat.py”, “printe.py”, “txtlib.py”, and “wd_sparql.py” that support logging, error handling, text formatting, or interfacing with databases.
• Automated Documentation & Continuous Integration: As indicated by the “.github” folder and workflows (e.g., “snorkell-auto-documentation.yml”), include a note about how deployment/CI aspects are handled, even if they are not core to the API’s functionality.
Establish Relationships and Data Flow
• The API Client acts as the gateway, with methods that internally route calls to either the login or page operations modules.
• The Login Module communicates with both Wikimedia’s API and some internal session/token management utilities.
• The Page Operations Module also may query additional services (or use common utilities) to process and handle page data, and might depend on bot modules when automating edits or retrieval.
• Utility components are shared between these major sections, ensuring that common logic (like error handling and formatting) is centralized.
Architectural Patterns & Design Principles
• Modular Design: The project is cleanly split into modules that correspond to specific functionality (login, page edits, bots, etc.). This separation of concerns should be evident.
• Layered Structure: Indicate a high-level presentation/API layer, a business logic layer (handling login and page operations), and utility/support layers.
• Error Handling & Abstraction: Several modules (e.g., “except_err.py”, “ar_err.py” in sub-folders) indicate robust error handling, which is a key design principle in this project.
Additional Advice for the Principal Software Engineer
• Annotations or legends that explain key relationships, such as “calls”, “depends on”, or “provides data to”.
• A separate section/note for CI/CD workflows (depicting the .github workflows) that, although not core to runtime architecture, are integral for project sustainability.