How daferrom/open-llm-prototype works
Main components
• GitHub Actions Workflow: Represented by the .github/workflows/update_docs.yml that automates the whole process.
• Diff Generator: The module/script (src/scripts/get_diff.py) responsible for generating a diff file based on recent Git commits.
• Documentation Generator: The script (src/scripts/gh_test_request.py) that leverages GPT-4 for summarizing and generating documentation from the diff file. Note that client_api_ai (in src/client_api_ai/) contains helper modules (client_api_ai.py and prompt_templates.py) for formulating and interacting with GPT-4.
• Confluence Publisher: The script (src/scripts/create_confluence_doc.py) along with the confluence_service module (which includes pages_service.py, service_file.py, and spaces_service.py) that handles communication with Confluence and publishing the generated docs.
• Utility and Helper Modules: Under src/utils (handling file access and environment configurations), as well as helper components like xhtml_generator (to generate XHTML content) and doctype_diff_validator (for validating diff file formats) and a diff_example directory illustrating a sample use-case.
• Support Components: External configuration and dependencies like requirements.txt for Python packages; environment variable setup for authentication tokens and keys.
Relationships and interactions
• Execution Pipeline:
a. The workflow first runs get_diff.py to create a diff file representing the latest Git commit changes.
b. Next, gh_test_request.py is executed, which routes the diff file to GPT-4 (using the client_api_ai modules) to generate documentation.
c. Finally, create_confluence_doc.py is invoked, which utilizes the functions in the confluence_service modules to publish the generated documentation to a Confluence space.
• Auxiliary Utilities: Utilities (e.g., env_utils.py and access_files_util.py) are used across these steps for configuration and file management. The xhtml_generator might be used for formatting output before it is delivered to Confluence.
Architectural patterns and design principles
• Modular Structure: The project is organized into well-separated modules (scripts, services, utilities) that each have a clear responsibility. This assists in maintainability.
• Pipeline-Oriented Design: The flow from diff-generation to documentation generation and then to content publication illustrates a clear sequential pipeline.
• Integration with External Services: The design integrates with external APIs (Confluence API and GPT-4 API), so these should be clearly depicted and their interfaces noted.
• Automation and CI/CD: The presence of GitHub Actions manifests an automated CI/CD pipeline to run these processes on code pushes.
• A clear start point (GitHub Actions trigger with .github/workflows/update_docs.yml).
• Distinct components:
• Diff Generation (src/scripts/get_diff.py)
• Documentation Generation (src/scripts/gh_test_request.py with dependencies on src/client_api_ai)
• Confluence Publishing (src/scripts/create_confluence_doc.py with src/confluence_service)
• Supporting Utilities (src/utils, src/xhtml_generator, src/doctype_diff_validator)