How sachin31111/wrangler works
Data Prep is a data-cleansing and transformation product: users author directives visually or manually, then apply them to records through a CDAP pipeline or service. Its runtime includes recipe execution, a library of parsing and transformation directives, and service capabilities for workspaces, connections, and source exploration. The README establishes the product workflow and directive capabilities; source-index references and excerpts support the specific service, storage, and execution links shown. Several directive and service internals were not sampled, so their detailed call paths are omitted.
Main entry points: RecipePipelineExecutor.java Recipe executor · DataPrepService.java Data Prep API · ConnectionHandler.java Connection management · DatabaseHandler.java Database discovery
Components 25
Recipe Runtime
- Recipe executor
wrangler-core/src/main/java/io/cdap/wrangler/executor/RecipePipelineExecutor.java - Recipe pipeline API
wrangler-api/src/main/java/io/cdap/wrangler/api/RecipePipeline.java - Row model
wrangler-api/src/main/java/io/cdap/wrangler/api/Row.java - Service pipeline context
wrangler-service/src/main/java/io/cdap/wrangler/service/directive/ServicePipelineContext.java - Transform context
wrangler-transform/src/main/java/io/cdap/wrangler/WranglerPipelineContext.java - Object serialization
wrangler-core/src/main/java/io/cdap/wrangler/utils/ObjectSerDe.java - Reference names
wrangler-core/src/main/java/io/cdap/wrangler/utils/ReferenceNames.java
Directive Capabilities
- Record parsers
wrangler-core/src/main/java/io/cdap/directives/parser - Record transformations
wrangler-core/src/main/java/io/cdap/directives/transformation - Column directives
wrangler-core/src/main/java/io/cdap/directives/column - Row controls
wrangler-core/src/main/java/io/cdap/directives/row - Output formatters
wrangler-core/src/main/java/io/cdap/directives/writer - Data quality
wrangler-core/src/main/java/io/cdap/wrangler/dq/TypeInference.java
Data Prep Service
- Data Prep API
wrangler-service/src/main/java/io/cdap/wrangler/service/DataPrepService.java - Request extraction
wrangler-service/src/main/java/io/cdap/wrangler/RequestExtractor.java - Source handlers
wrangler-service/src/main/java/io/cdap/wrangler/service - Connection management
wrangler-service/src/main/java/io/cdap/wrangler/service/connections/ConnectionHandler.java - Database discovery
wrangler-service/src/main/java/io/cdap/wrangler/service/database/DatabaseHandler.java - BigQuery discovery
wrangler-service/src/main/java/io/cdap/wrangler/service/bigquery/BigQueryHandler.java - Workspace operations
wrangler-service/src/main/java/io/cdap/wrangler/service/directive - Service response
wrangler-proto/src/main/java/io/cdap/wrangler/proto/ServiceResponse.java
Workspace State
- Workspace store
wrangler-storage/src/main/java/io/cdap/wrangler/dataset/workspace/WorkspaceDataset.java - Connection store
wrangler-storage/src/main/java/io/cdap/wrangler/dataset/connections/ConnectionStore.java - Connection model
wrangler-proto/src/main/java/io/cdap/wrangler/proto/connection/Connection.java
Other components
- Data Prep user
Connections 23
- Data Prep user → Data Prep API: submits requests
- Data Prep API → Source handlers: registers handlers
wrangler-service/src/main/java/io/cdap/wrangler/service/DataPrepService.java - Data Prep API → Connection management: registers handler
wrangler-service/src/main/java/io/cdap/wrangler/service/DataPrepService.java - Data Prep API → Database discovery: registers handler
wrangler-service/src/main/java/io/cdap/wrangler/service/DataPrepService.java - Data Prep API → BigQuery discovery: registers handler
wrangler-service/src/main/java/io/cdap/wrangler/service/DataPrepService.java - Data Prep API → Workspace operations: registers operations
wrangler-service/src/main/java/io/cdap/wrangler/service/DataPrepService.java - Request extraction → Connection model: parses metadata
wrangler-service/src/main/java/io/cdap/wrangler/RequestExtractor.java - Connection management → Connection store: reads and writes
wrangler-service/src/main/java/io/cdap/wrangler/service/connections/ConnectionHandler.java - Database discovery → Connection store: loads connections
- BigQuery discovery → Workspace store: uses workspace
wrangler-service/src/main/java/io/cdap/wrangler/service/bigquery/BigQueryHandler.java - Workspace operations → Workspace store: uses dataset
- Data Prep API → Service response: returns responses
- Data Prep user → Recipe executor: applies directives
- Recipe executor → Row model: processes rows
wrangler-core/src/main/java/io/cdap/wrangler/executor/RecipePipelineExecutor.java - Recipe executor → Recipe pipeline API: uses pipeline API
wrangler-core/src/main/java/io/cdap/wrangler/executor/RecipePipelineExecutor.java - Service pipeline context → Recipe executor: executes recipes
- Transform context → Recipe executor: executes recipes
- Recipe executor → Record parsers: dispatches directives
- Recipe executor → Record transformations: dispatches directives
- Recipe executor → Column directives: dispatches directives
- Recipe executor → Row controls: dispatches directives
- Recipe executor → Output formatters: dispatches directives
- Recipe executor → Data quality: supports quality checks