# cloudera/dataserviceslabs architecture

> Architecture diagram of the GitHub repository cloudera/dataserviceslabs, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/cloudera/dataserviceslabs
- Repository: https://github.com/cloudera/dataserviceslabs

## Overview

This repository is a collection of Cloudera Data Services workshops, not one deployed application. In the sampled AgenticAI workflow, a customer submits a request through Gradio; a CrewAI manager routes it to policy, promotion, or order-feedback agents, which use their tools to retrieve knowledge or query customer data. A separate chatbot workshop retrieves context from a vector index and uses an LLM to answer questions. The tree also includes data-engineering lab scripts. README capabilities for CDW and CDF lack corresponding runtime files in the supplied tree, so they are not drawn. Arrows reflect sampled code or explicit documentation; other workshop wiring is not inferred.

Main entry points: [`ClouderaAI/AgenticAI/chat_app/crew_agents/agents_and_tasks.py`](https://github.com/cloudera/dataserviceslabs/blob/main/ClouderaAI/AgenticAI/chat_app/crew_agents/agents_and_tasks.py) (Routing manager), [`ClouderaAI/AgenticAI/chat_app/chat_utils/vector_rag_utils.py`](https://github.com/cloudera/dataserviceslabs/blob/main/ClouderaAI/AgenticAI/chat_app/chat_utils/vector_rag_utils.py) (Policy retrieval), [`ClouderaAI/LLMChatbot-GenAI/LLM-hol-user-guide/8_launch_app_final/llm_app_final.py`](https://github.com/cloudera/dataserviceslabs/blob/main/ClouderaAI/LLMChatbot-GenAI/LLM-hol-user-guide/8_launch_app_final/llm_app_final.py) (Knowledge chatbot), [`ClouderaAI/AgenticAI/chat_app/chat_utils/graph_query_wrapper.py`](https://github.com/cloudera/dataserviceslabs/blob/main/ClouderaAI/AgenticAI/chat_app/chat_utils/graph_query_wrapper.py) (Customer graph access)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_agents["Agentic customer service"]
  node_agent_ui["Support UI"]
  node_crew["Crew orchestrator"]
  node_manager["Routing manager"]
  node_policy_agent["Policy agent"]
  node_sales_agent["Sales agent"]
  node_feedback_agent["Feedback agent"]
  node_policy_tool["Policy lookup"]
  node_promo_tool["Promotion lookup"]
  node_feedback_tool["Feedback submission"]
  node_graph_wrapper["Customer graph access"]
end

subgraph group_knowledge["Knowledge chatbots"]
  node_rag_utils["Policy retrieval"]
  node_kb_ingest["Vector index ingestion"]
  node_chatbot["Knowledge chatbot<br/>[llm_app_final.py]"]
end

subgraph group_engineering["Data engineering labs"]
  node_etl["Data extraction and reporting"]
  node_etl_large["Large-data extraction"]
  node_etl_reports["Report creation"]
  node_airflow["Airflow lab"]
end

node_customer(("Customer"))
node_neo4j[("Customer graph")]
node_pinecone[("Vector index")]
node_kb_files["Knowledge files"]
node_llm["Language model service"]

node_customer -->|"submits request"| node_agent_ui
node_agent_ui -->|"launches crew"| node_crew
node_crew -->|"routes request"| node_manager
node_manager -->|"delegates"| node_policy_agent
node_manager -->|"delegates"| node_sales_agent
node_manager -->|"delegates"| node_feedback_agent
node_policy_agent -->|"looks up policy"| node_policy_tool
node_policy_tool -->|"searches context"| node_rag_utils
node_rag_utils -->|"queries vectors"| node_pinecone
node_rag_utils -->|"reads matched file"| node_kb_files
node_sales_agent -->|"gets promotions"| node_promo_tool
node_promo_tool -->|"queries customer"| node_graph_wrapper
node_feedback_agent -->|"submits feedback"| node_feedback_tool
node_feedback_tool -->|"writes feedback"| node_graph_wrapper
node_graph_wrapper -->|"runs graph queries"| node_neo4j
node_kb_ingest -->|"reads documents"| node_kb_files
node_kb_ingest -->|"upserts embeddings"| node_pinecone
node_customer -->|"asks question"| node_chatbot
node_chatbot -->|"retrieves context"| node_pinecone
node_chatbot -->|"invokes model"| node_llm

click node_agent_ui "https://github.com/cloudera/dataserviceslabs/blob/main/ClouderaAI/AgenticAI/chat_app/agents_app_launch.py"
click node_crew "https://github.com/cloudera/dataserviceslabs/blob/main/ClouderaAI/AgenticAI/chat_app/agents_app_launch.py"
click node_manager "https://github.com/cloudera/dataserviceslabs/blob/main/ClouderaAI/AgenticAI/chat_app/crew_agents/agents_and_tasks.py"
click node_policy_agent "https://github.com/cloudera/dataserviceslabs/blob/main/ClouderaAI/AgenticAI/chat_app/crew_agents/agents_and_tasks.py"
click node_sales_agent "https://github.com/cloudera/dataserviceslabs/blob/main/ClouderaAI/AgenticAI/chat_app/crew_agents/agents_and_tasks.py"
click node_feedback_agent "https://github.com/cloudera/dataserviceslabs/blob/main/ClouderaAI/AgenticAI/chat_app/crew_agents/agents_and_tasks.py"
click node_policy_tool "https://github.com/cloudera/dataserviceslabs/blob/main/ClouderaAI/AgenticAI/chat_app/tools/retrieve_policy_tool.py"
click node_promo_tool "https://github.com/cloudera/dataserviceslabs/blob/main/ClouderaAI/AgenticAI/chat_app/tools/targeted_promos_tool.py"
click node_feedback_tool "https://github.com/cloudera/dataserviceslabs/blob/main/ClouderaAI/AgenticAI/chat_app/tools/feedback_submission_tool.py"
click node_rag_utils "https://github.com/cloudera/dataserviceslabs/blob/main/ClouderaAI/AgenticAI/chat_app/chat_utils/vector_rag_utils.py"
click node_graph_wrapper "https://github.com/cloudera/dataserviceslabs/blob/main/ClouderaAI/AgenticAI/chat_app/chat_utils/graph_query_wrapper.py"
click node_kb_ingest "https://github.com/cloudera/dataserviceslabs/blob/main/ClouderaAI/LLMChatbot-GenAI/LLM-hol-user-guide/2_populate_vector_db/pinecone_vectordb_insert.py"
click node_chatbot "https://github.com/cloudera/dataserviceslabs/blob/main/ClouderaAI/LLMChatbot-GenAI/LLM-hol-user-guide/8_launch_app_final/llm_app_final.py"
click node_etl "https://github.com/cloudera/dataserviceslabs/blob/main/ClouderaDataEngineering/Resources/Scripts/Lab3B1_Data_Extraction_Sub_150k.py"
click node_etl_large "https://github.com/cloudera/dataserviceslabs/blob/main/ClouderaDataEngineering/Resources/Scripts/Lab3B2_Data_Extraction_Over_150k.py"
click node_etl_reports "https://github.com/cloudera/dataserviceslabs/blob/main/ClouderaDataEngineering/Resources/Scripts/Lab3B3_Create_Reports.py"
click node_airflow "https://github.com/cloudera/dataserviceslabs/blob/main/ClouderaDataEngineering/Resources/Scripts/Lab5_airflow_dag.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_agent_ui,node_crew,node_manager,node_policy_agent,node_sales_agent,node_feedback_agent,node_policy_tool,node_promo_tool,node_feedback_tool,node_graph_wrapper toneBlue
class node_rag_utils,node_kb_ingest,node_chatbot,node_neo4j,node_pinecone toneAmber
class node_etl,node_etl_large,node_etl_reports,node_airflow toneMint
class node_customer,node_kb_files,node_llm toneIndigo
```

## Components

### Agentic customer service

- [Support UI](https://github.com/cloudera/dataserviceslabs/blob/main/ClouderaAI/AgenticAI/chat_app/agents_app_launch.py): `ClouderaAI/AgenticAI/chat_app/agents_app_launch.py`
- [Crew orchestrator](https://github.com/cloudera/dataserviceslabs/blob/main/ClouderaAI/AgenticAI/chat_app/agents_app_launch.py): `ClouderaAI/AgenticAI/chat_app/agents_app_launch.py`
- [Routing manager](https://github.com/cloudera/dataserviceslabs/blob/main/ClouderaAI/AgenticAI/chat_app/crew_agents/agents_and_tasks.py): `ClouderaAI/AgenticAI/chat_app/crew_agents/agents_and_tasks.py`
- [Policy agent](https://github.com/cloudera/dataserviceslabs/blob/main/ClouderaAI/AgenticAI/chat_app/crew_agents/agents_and_tasks.py): `ClouderaAI/AgenticAI/chat_app/crew_agents/agents_and_tasks.py`
- [Sales agent](https://github.com/cloudera/dataserviceslabs/blob/main/ClouderaAI/AgenticAI/chat_app/crew_agents/agents_and_tasks.py): `ClouderaAI/AgenticAI/chat_app/crew_agents/agents_and_tasks.py`
- [Feedback agent](https://github.com/cloudera/dataserviceslabs/blob/main/ClouderaAI/AgenticAI/chat_app/crew_agents/agents_and_tasks.py): `ClouderaAI/AgenticAI/chat_app/crew_agents/agents_and_tasks.py`
- [Policy lookup](https://github.com/cloudera/dataserviceslabs/blob/main/ClouderaAI/AgenticAI/chat_app/tools/retrieve_policy_tool.py): `ClouderaAI/AgenticAI/chat_app/tools/retrieve_policy_tool.py`
- [Promotion lookup](https://github.com/cloudera/dataserviceslabs/blob/main/ClouderaAI/AgenticAI/chat_app/tools/targeted_promos_tool.py): `ClouderaAI/AgenticAI/chat_app/tools/targeted_promos_tool.py`
- [Feedback submission](https://github.com/cloudera/dataserviceslabs/blob/main/ClouderaAI/AgenticAI/chat_app/tools/feedback_submission_tool.py): `ClouderaAI/AgenticAI/chat_app/tools/feedback_submission_tool.py`
- [Customer graph access](https://github.com/cloudera/dataserviceslabs/blob/main/ClouderaAI/AgenticAI/chat_app/chat_utils/graph_query_wrapper.py): `ClouderaAI/AgenticAI/chat_app/chat_utils/graph_query_wrapper.py`

### Knowledge chatbots

- [Policy retrieval](https://github.com/cloudera/dataserviceslabs/blob/main/ClouderaAI/AgenticAI/chat_app/chat_utils/vector_rag_utils.py): `ClouderaAI/AgenticAI/chat_app/chat_utils/vector_rag_utils.py`
- [Vector index ingestion](https://github.com/cloudera/dataserviceslabs/blob/main/ClouderaAI/LLMChatbot-GenAI/LLM-hol-user-guide/2_populate_vector_db/pinecone_vectordb_insert.py): `ClouderaAI/LLMChatbot-GenAI/LLM-hol-user-guide/2_populate_vector_db/pinecone_vectordb_insert.py`
- [Knowledge chatbot](https://github.com/cloudera/dataserviceslabs/blob/main/ClouderaAI/LLMChatbot-GenAI/LLM-hol-user-guide/8_launch_app_final/llm_app_final.py): `ClouderaAI/LLMChatbot-GenAI/LLM-hol-user-guide/8_launch_app_final/llm_app_final.py`

### Data engineering labs

- [Data extraction and reporting](https://github.com/cloudera/dataserviceslabs/blob/main/ClouderaDataEngineering/Resources/Scripts/Lab3B1_Data_Extraction_Sub_150k.py): `ClouderaDataEngineering/Resources/Scripts/Lab3B1_Data_Extraction_Sub_150k.py`
- [Large-data extraction](https://github.com/cloudera/dataserviceslabs/blob/main/ClouderaDataEngineering/Resources/Scripts/Lab3B2_Data_Extraction_Over_150k.py): `ClouderaDataEngineering/Resources/Scripts/Lab3B2_Data_Extraction_Over_150k.py`
- [Report creation](https://github.com/cloudera/dataserviceslabs/blob/main/ClouderaDataEngineering/Resources/Scripts/Lab3B3_Create_Reports.py): `ClouderaDataEngineering/Resources/Scripts/Lab3B3_Create_Reports.py`
- [Airflow lab](https://github.com/cloudera/dataserviceslabs/blob/main/ClouderaDataEngineering/Resources/Scripts/Lab5_airflow_dag.py): `ClouderaDataEngineering/Resources/Scripts/Lab5_airflow_dag.py`

### Other components

- **Customer**
- **Customer graph**
- **Vector index**
- **Knowledge files**
- **Language model service**

## Connections

- Customer → Support UI: submits request
- Support UI → Crew orchestrator: launches crew
- Crew orchestrator → Routing manager: routes request
- Routing manager → Policy agent: delegates
- Routing manager → Sales agent: delegates
- Routing manager → Feedback agent: delegates
- Policy agent → Policy lookup: looks up policy
- Policy lookup → Policy retrieval: searches context
- Policy retrieval → Vector index: queries vectors
- Policy retrieval → Knowledge files: reads matched file
- Sales agent → Promotion lookup: gets promotions
- Promotion lookup → Customer graph access: queries customer
- Feedback agent → Feedback submission: submits feedback
- Feedback submission → Customer graph access: writes feedback
- Customer graph access → Customer graph: runs graph queries
- Vector index ingestion → Knowledge files: reads documents
- Vector index ingestion → Vector index: upserts embeddings
- Customer → Knowledge chatbot: asks question
- Knowledge chatbot → Vector index: retrieves context
- Knowledge chatbot → Language model service: invokes model