How test-fullautomation/python-jsonpreprocessor works
Project Type and Purpose
• This is a standalone, open-source Python3 library (tool) called JsonPreprocessor.
• Purpose: Extend JSON handling with features like comments, imports, parameter overrides, and return a Python dict.
File Structure Analysis
• Top-level code folder:
• JsonPreprocessor/ – core package module with CJsonPreprocessor.py (main logic), version.
• Build & packaging scripts:
• setup.py, requirements.txt, genpackagedoc.py, additions/CExtendedSetup.py
• Configuration:
• config/ – repository and RobotFramework AIO JSON configurations.
• packagedoc/ – GenPackageDoc configuration for PDF docs.
• Samples & tests:
• sample/ – sample Python scripts and JSONs demonstrating features.
• test/ and atest/ – unit tests (pytest) and acceptance tests; extensive JSONP testcases.
• Docs & CI:
• JsonPreprocessor/JsonPreprocessor.pdf – prebuilt PDF.
• .github/workflows/ – CI for publishing to PyPI and triggering runs.
Architectural Components
• a. Core Library Module
– CJsonPreprocessor (parses JSONP, strips/handles comments, resolves imports, applies overrides)
– Internal sub-modules (e.g., parser, importer, override resolver)
• b. Public API / CLI
– setup.py install exposes library; or import JsonPreprocessor.CJsonPreprocessor in Python
• c. Configuration Manager
– Reads config/repository_config.json and RobotFramework release configs
• d. Documentation Generator
– GenPackageDoc invoked by genpackagedoc.py using packagedoc/packagedoc_config.json
– Requires external LaTeX (GENDOC_LATEXPATH)
• e. Test Harness
– Pytest runner scripts (atest/* and test/pytest/*)
– Test data under atest/testdata and test/testfiles
• f. Sample & Examples
– Demonstrate usage via Python scripts and JSON files
Component Relationships & Data Flows
• User (or CI) installs package → exposes JsonPreprocessor API
↳ Application code or CLI loads CJsonPreprocessor
→ Parser reads JSONP file →
↳ Importer follows import statements & inlines nested files →
↳ Override Resolver applies parameter overrides →
↳ Final Python dict output
• Documentation pipeline: genpackagedoc.py ← reads source & packagedoc config → runs LaTeX → produces PDF
• Test pipeline: pytest workflows invoke test runners → feed JSONP testcases → validate output against expected dicts
• Packaging/CI: GitHub Actions → build & publish to PyPI using setup.py and python-publish.yml
Architectural Patterns & Design Principles
• Modularity: clear separation between parsing, importing, overriding
• Single Responsibility: each module handles one JSONP feature
• Extensibility: new preprocess directives can be added via sub-modules
• Configuration-driven: repository & doc generation behavior controlled by JSON configs
• Test-driven: comprehensive test suite covering every feature case