How donbugan/adf-ogk-learn works
This Azure Data Factory project validates a manually uploaded fleet-vehicle CSV before copying it: it reads rows, flags empty Vehicle_ID values, and branches to either a curated copy or a JSON defect report. A scheduled trigger can start the pipeline, though it is currently stopped. The graph follows the README’s documented behavior; source excerpts were unavailable, so configuration-level relationships not described there are omitted. The mapping-data-flow cleanse and database sink are planned, not built capabilities.
Main entry points: pl_copy_fleet_vehicles.json Read all rows · ds_fleet_vehicles_csv.json Source dataset · ds_fleet_vehicles_curated.json Curated dataset · trg_copy_fleet_vehicles.json Three-hour trigger
Components 15
Storage and connections
- Raw container
- Source dataset
dataset/ds_fleet_vehicles_csv.json - Blob linked service
linkedService/ls_ogk_blob.json - Curated dataset
dataset/ds_fleet_vehicles_curated.json - Curated container
- Reports container
Validation pipeline
- Fleet copy pipeline
pipeline/pl_copy_fleet_vehicles.json - Read all rows
pipeline/pl_copy_fleet_vehicles.json - Filter empty IDs
pipeline/pl_copy_fleet_vehicles.json - Defect condition
pipeline/pl_copy_fleet_vehicles.json - Copy clean file
pipeline/pl_copy_fleet_vehicles.json - Write defect report
pipeline/pl_copy_fleet_vehicles.json
Scheduling and factory
- Three-hour trigger
trigger/trg_copy_fleet_vehicles.json - Data factory
factory/adf-ogk-learn.json
Other components
- Manual uploader
Connections 13
- Manual uploader → Raw container: uploads CSV
README.md - Fleet copy pipeline → Read all rows: runs
README.md - Read all rows → Source dataset: reads rows
README.md - Source dataset → Raw container: reads from
README.md - Read all rows → Filter empty IDs: passes rows
README.md - Filter empty IDs → Defect condition: flags nulls
README.md - Defect condition → Copy clean file: clean file
README.md - Copy clean file → Curated dataset: copies file
README.md - Curated dataset → Curated container: writes to
README.md - Defect condition → Write defect report: defects found
README.md - Write defect report → Reports container: writes JSON
README.md - Three-hour trigger → Fleet copy pipeline: starts every 3 hours
README.md - Write defect report → Data factory: uses managed identity
README.md