# javierg123/poc---localgenesysdatalake architecture

> Architecture diagram of the GitHub repository javierg123/poc---localgenesysdatalake, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/javierg123/poc---localgenesysdatalake
- Repository: https://github.com/javierg123/poc---localgenesysdatalake

## Overview

##### Project type and purpose
- This is a small Python-based proof of concept (POC) data-pipeline tool whose goal is to create a data-extraction job against the Genesys Data Lake, poll for completion, fetch the resulting data, and load it into a local Microsoft SQL Server database.

##### File structure
- Top-level scripts:
- CreateJob.py – invokes the Genesys Data Lake API to submit a data-extraction job
- CheckJobID.py – polls the status of the submitted job until it completes
- GetJobResult.py – retrieves the completed job’s data payload and writes it to MSSQL
- Supporting files:
- requirements.txt – Python dependencies (e.g., requests, pyodbc)
- .gitignore, LICENSE, README.md

##### README for additional insights
- The README confirms the workflow: “create a Job, get data and store into a local MSSQL.”

##### Main components and their interactions
a. Components
1) Client Scripts (entry points):
– CreateJob.py
– CheckJobID.py
– GetJobResult.py
2) External Service:
– Genesys Data Lake REST API
3) Local Storage:
– Microsoft SQL Server
4) Supporting modules/config:
– Python libraries (requests for HTTP, pyodbc for DB)
– Configuration (environment variables or a .env file)
b. Relationships and data flow
2) Genesys API ⇒ returns jobId
5) GetJobResult.py ⇒ Local MSSQL: inserts or updates tables with retrieved data via pyodbc
c. Architectural patterns & design principles
- Simple pipeline pattern (submit → poll → fetch → load)
- Single responsibility per script (each .py file does one step)
- External dependency isolation (HTTP calls separated from DB logic)
d. Key technologies
– Python 3.x
– requests (HTTP client)
– pyodbc (ODBC driver for MSSQL)
– Genesys Data Lake REST API
– Microsoft SQL Server

##### Detailed capture of essential elements
- Environment/config dependencies (API keys, connection strings)
- That CheckJobID.py loops/polls until the job is done
- That GetJobResult.py may perform batching or paging if data is large
- Error-handling or retry logic if present

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Config Store
    Config["Config / Secrets Store"]:::config

    %% Client Scripts
    CJ["CreateJob.py"]:::script
    CP["CheckJobID.py"]:::script
    GR["GetJobResult.py"]:::script

    %% External Service
    API["Genesys Data Lake API"]:::external

    %% Database
    DB[(Local MSSQL Database)]:::db

    %% Documentation & Dependencies
    REQ["requirements.txt"]:::config
    MD["README.md"]:::config
    IGN[".gitignore"]:::config
    LIC["LICENSE"]:::config

    %% Data Pipeline Flow
    Config --> CJ
    CJ -->|"POST /jobs"| API
    API -->|"jobId"| CJ
    CJ --> CP
    Config --> CP
    CP -->|"GET /jobs/{jobId}/status (poll loop)"| API
    API -->|"status"| CP
    CP -->|"on COMPLETED"| GR
    Config --> GR
    GR -->|"GET /jobs/{jobId}/data"| API
    API -->|"data payload"| GR
    GR -->|"INSERT/UPDATE via pyodbc"| DB

    %% Supporting Files Connections
    REQ --> CJ
    REQ --> CP
    REQ --> GR
    MD --> CJ
    MD --> CP
    MD --> GR
    IGN --> REQ
    LIC --> MD

    %% Click Events
    click CJ "https://github.com/javierg123/poc---localgenesysdatalake/blob/main/CreateJob.py"
    click CP "https://github.com/javierg123/poc---localgenesysdatalake/blob/main/CheckJobID.py"
    click GR "https://github.com/javierg123/poc---localgenesysdatalake/blob/main/GetJobResult.py"
    click REQ "https://github.com/javierg123/poc---localgenesysdatalake/blob/main/requirements.txt"
    click MD "https://github.com/javierg123/poc---localgenesysdatalake/blob/main/README.md"
    click IGN "https://github.com/javierg123/poc---localgenesysdatalake/blob/main/.gitignore"
    click LIC "https://github.com/javierg123/poc---localgenesysdatalake/tree/main/LICENSE"

    %% Styles
    classDef script fill:#ADD8E6,stroke:#000,stroke-width:1px
    classDef external fill:#90EE90,stroke:#000,stroke-width:1px
    classDef db fill:#f9f,stroke:#000,stroke-width:1px
    classDef config fill:#D3D3D3,stroke:#000,stroke-width:1px
```
