# dermatologist/pyomop architecture

> Architecture diagram of the GitHub repository dermatologist/pyomop, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dermatologist/pyomop
- Repository: https://github.com/dermatologist/pyomop

## Overview

##### Main Components
- Library Core: Represented by the modules under src/pyomop. This includes components like:
- Engine Factory (engine_factory.py): Responsible for database connections.
- CDM Models & Tables (cdm6_tables.py): These define the structure of the OMOP CDM.
- Query Components: Such as CdmVocabulary, CdmVector, llm_engine, and llm_query, which handle vocabulary processing, conversion to pandas dataframes, and LLM-driven query processing.
- Main Entrypoints (e.g., __main__.py, main.py): Offering both command-line execution and script-based usage.

- Database Interaction Layer: Highlight that the project utilizes SQLAlchemy for ORM, which manages sessions and connections across supported databases (SQLite, PostgreSQL, MySQL, etc.).

- Data Conversion & Query Handling: This includes the conversion of query results to pandas dataframes as well as support for custom SQL queries (via sqldict.py).

- LLM Integration: Include the newer module components (llm_engine.py and llm_query.py) for processing natural language queries through an LLM. Indicate that these are optional extensions enabled by installing additional extras.

##### Out the Relationships and Interactions
- That customers or end users interact primarily with a command-line interface (CLI usage via main.py and pyomop command-line help) or by importing the library in their own scripts.

- Flow:
- The user instantiates a CdmEngineFactory that sets up the SQLAlchemy engine.
- They then create a session (possibly asynchronously) to interact with the database via ORM models defined in the project.
- Queries are executed either synchronously (for older versions) or asynchronously (for version 4.0.0 and later), and results can be wrapped or converted using modules like CdmVector.
- For vocabulary and domain-specific querying logic, the library uses components from CdmVocabulary and sqldict for predefined query templates.
- Optionally, if LLM capabilities are enabled and installed, the llm_engine and llm_query modules interact with external LLM services to transform natural language queries into SQL queries.

- How error handling, connection disposal, and session management are integral parts of the Engine Factory’s workflow.

##### Architectural Patterns and Design Principles
- Layered Architecture: Clearly separate layers such as the API/client layer (CLI/examples), the domain logic (engine and model definitions), and the data access layer (SQLAlchemy ORM).
- Modularity and Extensibility: The separation of concerns in modules like vocabulary creation and vector conversion shows a modular design, which is further extended by LLM-based query capabilities.
- Support for Asynchronous and Synchronous Operations: Emphasize the dual-mode operation allowing the system to handle both blocking and non-blocking DB operations.

##### Relevant Technologies
- Python, SQLAlchemy, and Pandas – these are core to data processing and ORM functionality.
- AsyncIO – used for asynchronous operations.
- LLM integration components like llama-index (as mentioned in the README) for natural language processing.
- Testing frameworks (pytest, tox, Travis CI) and documentation tools (Sphinx via docs/conf.py) as part of the development and deployment pipeline.

##### Essential Elements
- Annotations where necessary to represent asynchronous flows versus synchronous ones.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User Layer
    subgraph "User Layer"
        USER["User/CLI"]:::userStyle
    end

    %% Entrypoint Layer
    subgraph "Entrypoints"
        MAIN_CLI["\"__main__.py Entrypoint\""]:::userStyle
        MAIN_SCRIPT["\"main.py Entrypoint\""]:::userStyle
    end

    %% Core Library Layer
    subgraph "Core Library"
        EF["CdmEngineFactory"]:::coreStyle
        CM["CDM Models & Tables"]:::coreStyle
    end

    %% Query Processing Layer
    subgraph "Query Processing"
        VOC["CdmVocabulary"]:::queryStyle
        VEC["CdmVector"]:::queryStyle
        SQLDICT["sqldict"]:::queryStyle
    end

    %% LLM Integration (Optional)
    subgraph "LLM Integration (Optional)"
        LLM_ENG["LLM Engine"]:::llmStyle
        LLM_QUERY["LLM Query"]:::llmStyle
    end

    %% Data Access Layer
    subgraph "Data Access Layer"
        ORM["SQLAlchemy ORM"]:::dataStyle
        DB["Database"]:::dataStyle
    end

    %% Async/Synchronous Control
    subgraph "Async/Sync Control"
        ASYNC["Async/Sync Flow"]:::asyncStyle
    end

    %% Connections
    USER -->|"uses"| MAIN_CLI
    USER -->|"uses"| MAIN_SCRIPT

    MAIN_CLI -->|"initiates"| EF
    MAIN_SCRIPT -->|"initiates"| EF

    EF -->|"defines&managesModels"| CM
    EF -->|"establishesSessions"| ORM
    EF -->|"handlesAsyncSync"| ASYNC

    ORM -->|"executesQueries"| DB

    EF -->|"supportsQueryConversion"| VOC
    VOC -->|"convertsResults"| VEC
    SQLDICT -->|"executesCustomQueries"| ORM

    MAIN_CLI -->|"optionalLLMQuery"| LLM_ENG
    MAIN_SCRIPT -->|"optionalLLMQuery"| LLM_ENG
    LLM_ENG -->|"processesNLP"| LLM_QUERY
    LLM_QUERY -->|"outputsSQL"| SQLDICT

    %% Click Events
    click EF "https://github.com/dermatologist/pyomop/blob/develop/src/pyomop/engine_factory.py"
    click CM "https://github.com/dermatologist/pyomop/blob/develop/src/pyomop/cdm6_tables.py"
    click VOC "https://github.com/dermatologist/pyomop/blob/develop/src/pyomop/vocabulary.py"
    click VEC "https://github.com/dermatologist/pyomop/blob/develop/src/pyomop/vector.py"
    click SQLDICT "https://github.com/dermatologist/pyomop/blob/develop/src/pyomop/sqldict.py"
    click LLM_ENG "https://github.com/dermatologist/pyomop/blob/develop/src/pyomop/llm_engine.py"
    click LLM_QUERY "https://github.com/dermatologist/pyomop/blob/develop/src/pyomop/llm_query.py"
    click MAIN_CLI "https://github.com/dermatologist/pyomop/blob/develop/src/pyomop/__main__.py"
    click MAIN_SCRIPT "https://github.com/dermatologist/pyomop/blob/develop/main.py"

    %% Styles
    classDef userStyle fill:#A7D7FF,stroke:#333,stroke-width:1px;
    classDef coreStyle fill:#FFD27F,stroke:#333,stroke-width:1px;
    classDef queryStyle fill:#C4F1BE,stroke:#333,stroke-width:1px;
    classDef llmStyle fill:#F1C0E8,stroke:#333,stroke-width:1px;
    classDef dataStyle fill:#FCECC9,stroke:#333,stroke-width:1px;
    classDef asyncStyle fill:#E2E2E2,stroke:#333,stroke-width:1px;
```
