How javierg123/poc---localgenesysdatalake works
Project type and purpose
• This is a small Python-based proof of concept (POC) data-pipeline tool whose goal is to create a data-extraction job against the Genesys Data Lake, poll for completion, fetch the resulting data, and load it into a local Microsoft SQL Server database.
File structure
• Top-level scripts:
• CreateJob.py – invokes the Genesys Data Lake API to submit a data-extraction job
• CheckJobID.py – polls the status of the submitted job until it completes
• GetJobResult.py – retrieves the completed job’s data payload and writes it to MSSQL
• Supporting files:
• requirements.txt – Python dependencies (e.g., requests, pyodbc)
• .gitignore, LICENSE, README.md
README for additional insights
• The README confirms the workflow: “create a Job, get data and store into a local MSSQL.”
Main components and their interactions
a. Components
1) Client Scripts (entry points):
– CreateJob.py
– CheckJobID.py
– GetJobResult.py
2) External Service:
– Genesys Data Lake REST API
3) Local Storage:
– Microsoft SQL Server
4) Supporting modules/config:
– Python libraries (requests for HTTP, pyodbc for DB)
– Configuration (environment variables or a .env file)
b. Relationships and data flow
2) Genesys API ⇒ returns jobId
5) GetJobResult.py ⇒ Local MSSQL: inserts or updates tables with retrieved data via pyodbc
c. Architectural patterns & design principles
• Simple pipeline pattern (submit → poll → fetch → load)
• Single responsibility per script (each .py file does one step)
• External dependency isolation (HTTP calls separated from DB logic)
d. Key technologies
– Python 3.x
– requests (HTTP client)
– pyodbc (ODBC driver for MSSQL)
– Genesys Data Lake REST API
– Microsoft SQL Server
Detailed capture of essential elements
• Environment/config dependencies (API keys, connection strings)
• That CheckJobID.py loops/polls until the job is done
• That GetJobResult.py may perform batching or paging if data is large
• Error-handling or retry logic if present