# daferrom/open-llm-prototype architecture

> Architecture diagram of the GitHub repository daferrom/open-llm-prototype, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/daferrom/open-llm-prototype
- Repository: https://github.com/daferrom/open-llm-prototype

## Overview

##### Main components
- GitHub Actions Workflow: Represented by the .github/workflows/update_docs.yml that automates the whole process.
- Diff Generator: The module/script (src/scripts/get_diff.py) responsible for generating a diff file based on recent Git commits.
- Documentation Generator: The script (src/scripts/gh_test_request.py) that leverages GPT-4 for summarizing and generating documentation from the diff file. Note that client_api_ai (in src/client_api_ai/) contains helper modules (client_api_ai.py and prompt_templates.py) for formulating and interacting with GPT-4.
- Confluence Publisher: The script (src/scripts/create_confluence_doc.py) along with the confluence_service module (which includes pages_service.py, service_file.py, and spaces_service.py) that handles communication with Confluence and publishing the generated docs.
- Utility and Helper Modules: Under src/utils (handling file access and environment configurations), as well as helper components like xhtml_generator (to generate XHTML content) and doctype_diff_validator (for validating diff file formats) and a diff_example directory illustrating a sample use-case.
- Support Components: External configuration and dependencies like requirements.txt for Python packages; environment variable setup for authentication tokens and keys.

##### Relationships and interactions
- Execution Pipeline:
a. The workflow first runs get_diff.py to create a diff file representing the latest Git commit changes.
b. Next, gh_test_request.py is executed, which routes the diff file to GPT-4 (using the client_api_ai modules) to generate documentation.
c. Finally, create_confluence_doc.py is invoked, which utilizes the functions in the confluence_service modules to publish the generated documentation to a Confluence space.
- Auxiliary Utilities: Utilities (e.g., env_utils.py and access_files_util.py) are used across these steps for configuration and file management. The xhtml_generator might be used for formatting output before it is delivered to Confluence.

##### Architectural patterns and design principles
- Modular Structure: The project is organized into well-separated modules (scripts, services, utilities) that each have a clear responsibility. This assists in maintainability.
- Pipeline-Oriented Design: The flow from diff-generation to documentation generation and then to content publication illustrates a clear sequential pipeline.
- Integration with External Services: The design integrates with external APIs (Confluence API and GPT-4 API), so these should be clearly depicted and their interfaces noted.
- Automation and CI/CD: The presence of GitHub Actions manifests an automated CI/CD pipeline to run these processes on code pushes.

- A clear start point (GitHub Actions trigger with .github/workflows/update_docs.yml).
- Distinct components:
- Diff Generation (src/scripts/get_diff.py)
- Documentation Generation (src/scripts/gh_test_request.py with dependencies on src/client_api_ai)
- Confluence Publishing (src/scripts/create_confluence_doc.py with src/confluence_service)
- Supporting Utilities (src/utils, src/xhtml_generator, src/doctype_diff_validator)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% CI/CD Pipeline
    subgraph "CI/CD Pipeline"
        GA["GitHub Actions Workflow"]:::github
    end

    %% Processing Pipeline
    subgraph "Processing Pipeline"
        DG["Diff Generator"]:::script
        DocGen["Documentation Generator"]:::script
        CP["Confluence Publisher"]:::script
        GA -->|"triggers"| DG
        DG -->|"creates_diff"| DocGen
        DocGen -->|"generates_docs"| CP
    end

    %% Documentation Support Modules
    subgraph "Documentation Support"
        CAI["client_api_ai: client_api_ai.py"]:::module
        PT["prompt_templates.py"]:::module
        DocGen -->|"uses"| CAI
        DocGen -->|"uses"| PT
    end

    %% Publishing Support Modules
    subgraph "Publishing Support"
        PS["pages_service.py"]:::module
        SF["service_file.py"]:::module
        SS["spaces_service.py"]:::module
        CP -->|"calls"| PS
        CP -->|"calls"| SF
        CP -->|"calls"| SS
    end

    %% Utility Modules
    subgraph "Utility Modules"
        AF["access_files_util.py"]:::utility
        EU["env_utils.py"]:::utility
        XH["XHTML Generator"]:::utility
        DV["Diff Validator"]:::utility
        DE["Diff Example"]:::utility
        DG -->|"utilizes"| AF
        DG -->|"utilizes"| EU
        DocGen -->|"formats_with"| XH
        CP -->|"validates_with"| DV
        CP -->|"references"| DE
    end

    %% External Services
    subgraph "External Services"
        GPT["GPT-4 API"]:::service
        Confluence["Confluence API"]:::service
        DocGen -->|"calls"| GPT
        CP -->|"calls"| Confluence
    end

    %% External Dependencies
    ED["External Dependencies"]:::dependency
    CP -->|"requires"| ED

    %% Click Events
    click GA "https://github.com/daferrom/open-llm-prototype/blob/main/.github/workflows/update_docs.yml"
    click DG "https://github.com/daferrom/open-llm-prototype/blob/main/src/scripts/get_diff.py"
    click DocGen "https://github.com/daferrom/open-llm-prototype/blob/main/src/scripts/gh_test_request.py"
    click CAI "https://github.com/daferrom/open-llm-prototype/blob/main/src/client_api_ai/client_api_ai.py"
    click PT "https://github.com/daferrom/open-llm-prototype/blob/main/src/client_api_ai/prompt_templates.py"
    click CP "https://github.com/daferrom/open-llm-prototype/blob/main/src/scripts/create_confluence_doc.py"
    click PS "https://github.com/daferrom/open-llm-prototype/blob/main/src/confluence_service/pages_service.py"
    click SF "https://github.com/daferrom/open-llm-prototype/blob/main/src/confluence_service/service_file.py"
    click SS "https://github.com/daferrom/open-llm-prototype/blob/main/src/confluence_service/spaces_service.py"
    click AF "https://github.com/daferrom/open-llm-prototype/blob/main/src/utils/access_files_util.py"
    click EU "https://github.com/daferrom/open-llm-prototype/blob/main/src/utils/env_utils.py"
    click XH "https://github.com/daferrom/open-llm-prototype/blob/main/src/xhtml_generator/xhtml_generator.py"
    click DV "https://github.com/daferrom/open-llm-prototype/blob/main/src/doctype_diff_validator/doctype_diff_validator.py"
    click DE "https://github.com/daferrom/open-llm-prototype/tree/main/src/diff_example"
    click ED "https://github.com/daferrom/open-llm-prototype/blob/main/requirements.txt"

    %% Styles
    classDef github fill:#ffcc00,stroke:#333,stroke-width:2px;
    classDef script fill:#99ccff,stroke:#333,stroke-width:2px;
    classDef module fill:#ccffcc,stroke:#333,stroke-width:2px;
    classDef utility fill:#eeeeee,stroke:#333,stroke-width:2px;
    classDef service fill:#ff9999,stroke:#333,stroke-width:2px;
    classDef dependency fill:#f9c,stroke:#333,stroke-width:2px;
```
