# googlecloudplatform/training-data-analyst architecture
> Architecture diagram of the GitHub repository googlecloudplatform/training-data-analyst, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-01). It is an AI-made overview: check details against the source.
- Interactive diagram: https://gitdiagram.com/googlecloudplatform/training-data-analyst
- Repository: https://github.com/googlecloudplatform/training-data-analyst
## Overview
This repository is a collection of Google Cloud training and blog examples, not one integrated application. The sampled runtime workflows include running batch or streaming TensorFlow inference, processing radar archives into detections, and streaming traffic records into BigQuery. The graph keeps those examples separate and includes the vendored Flask/OAuth/Werkzeug runtime represented by the source index. Evidence is partial: unsampled implementations and the broader collection’s workflows are not inferred.
Main entry points: [`blogs/tf_dataflow_serving/run_pipeline.py`](https://github.com/googlecloudplatform/training-data-analyst/blob/master/blogs/tf_dataflow_serving/run_pipeline.py) (Inference runner), [`blogs/nexrad2/src/com/google/cloud/public_datasets/nexrad2/APPipeline.java`](https://github.com/googlecloudplatform/training-data-analyst/blob/master/blogs/nexrad2/src/com/google/cloud/public_datasets/nexrad2/APPipeline.java) (Radar pipeline), [`quests/dataflow_python/5_Streaming_Analytics/solution/streaming_minute_traffic_pipeline.py`](https://github.com/googlecloudplatform/training-data-analyst/blob/master/quests/dataflow_python/5_Streaming_Analytics/solution/streaming_minute_traffic_pipeline.py) (Traffic pipeline), [`courses/machine_learning/deepdive2/structured/labs/serving/application/lib/flask/json.py`](https://github.com/googlecloudplatform/training-data-analyst/blob/master/courses/machine_learning/deepdive2/structured/labs/serving/application/lib/flask/json.py) (Flask JSON)
## Diagram (Mermaid)
```mermaid
flowchart TD
subgraph group_mlserving["ML Serving"]
node_tf_runner["Inference runner
[run_pipeline.py]"]
node_batch_inference["Batch inference"]
node_stream_inference["Stream inference"]
node_beam_runner["Beam execution
[run_pipeline.py]"]
end
subgraph group_radar["Radar Processing"]
node_radar_pipeline["Radar pipeline
[APPipeline.java]"]
node_archive_list["Archive listing"]
node_archive_read["Archive reading"]
node_ap_detector["AP detection
[APDetector.java]"]
end
subgraph group_traffic["Traffic Analytics"]
node_traffic_pipeline["Traffic pipeline"]
end
subgraph group_webdeps["Web Runtime"]
node_flask["Flask framework"]
node_flask_json["Flask JSON
[json.py]"]
node_flask_logging["Flask logging
[logging.py]"]
node_werkzeug["Werkzeug utilities"]
node_werkzeug_http["HTTP utilities
[http.py]"]
node_werkzeug_wsgi["WSGI server
[serving.py]"]
node_oauth_client["OAuth client
[client.py]"]
node_oauth_helpers["OAuth helpers
[_helpers.py]"]
node_oauth_service["Service account
[service_account.py]"]
end
node_operator(("Pipeline operator"))
node_pubsub["Pub/Sub"]
node_bigquery[("BigQuery")]
node_operator -->|"runs"| node_tf_runner
node_tf_runner -->|"dispatches batch"| node_batch_inference
node_tf_runner -->|"dispatches stream"| node_stream_inference
node_tf_runner -->|"configures job"| node_beam_runner
node_radar_pipeline -->|"references"| node_archive_list
node_radar_pipeline -->|"references"| node_archive_read
node_radar_pipeline -->|"references"| node_ap_detector
node_traffic_pipeline -->|"reads messages"| node_pubsub
node_traffic_pipeline -->|"writes rows"| node_bigquery
node_flask_json -->|"uses context"| node_flask
node_flask_logging -->|"uses request context"| node_flask
node_flask_json -->|"uses HTTP helpers"| node_werkzeug_http
node_flask_logging -->|"uses local proxy"| node_werkzeug
node_oauth_client -->|"uses helpers"| node_oauth_helpers
click node_tf_runner "https://github.com/googlecloudplatform/training-data-analyst/blob/master/blogs/tf_dataflow_serving/run_pipeline.py"
click node_batch_inference "https://github.com/googlecloudplatform/training-data-analyst/tree/master/blogs/tf_dataflow_serving/pipelines"
click node_stream_inference "https://github.com/googlecloudplatform/training-data-analyst/tree/master/blogs/tf_dataflow_serving/pipelines"
click node_beam_runner "https://github.com/googlecloudplatform/training-data-analyst/blob/master/blogs/tf_dataflow_serving/run_pipeline.py"
click node_radar_pipeline "https://github.com/googlecloudplatform/training-data-analyst/blob/master/blogs/nexrad2/src/com/google/cloud/public_datasets/nexrad2/APPipeline.java"
click node_archive_list "https://github.com/googlecloudplatform/training-data-analyst/blob/master/blogs/nexrad2/src/com/google/cloud/public_datasets/nexrad2/GcsNexradL2List.java"
click node_archive_read "https://github.com/googlecloudplatform/training-data-analyst/blob/master/blogs/nexrad2/src/com/google/cloud/public_datasets/nexrad2/GcsNexradL2Read.java"
click node_ap_detector "https://github.com/googlecloudplatform/training-data-analyst/blob/master/blogs/nexrad2/src/com/google/cloud/public_datasets/nexrad2/APDetector.java"
click node_traffic_pipeline "https://github.com/googlecloudplatform/training-data-analyst/blob/master/quests/dataflow_python/5_Streaming_Analytics/solution/streaming_minute_traffic_pipeline.py"
click node_flask "https://github.com/googlecloudplatform/training-data-analyst/tree/master/courses/machine_learning/deepdive2/structured/labs/serving/application/lib/flask"
click node_flask_json "https://github.com/googlecloudplatform/training-data-analyst/blob/master/courses/machine_learning/deepdive2/structured/labs/serving/application/lib/flask/json.py"
click node_flask_logging "https://github.com/googlecloudplatform/training-data-analyst/blob/master/courses/machine_learning/deepdive2/structured/labs/serving/application/lib/flask/logging.py"
click node_werkzeug "https://github.com/googlecloudplatform/training-data-analyst/tree/master/courses/machine_learning/deepdive2/structured/labs/serving/application/lib/werkzeug"
click node_werkzeug_http "https://github.com/googlecloudplatform/training-data-analyst/blob/master/courses/machine_learning/deepdive2/structured/labs/serving/application/lib/werkzeug/http.py"
click node_werkzeug_wsgi "https://github.com/googlecloudplatform/training-data-analyst/blob/master/courses/machine_learning/deepdive2/structured/labs/serving/application/lib/werkzeug/serving.py"
click node_oauth_client "https://github.com/googlecloudplatform/training-data-analyst/blob/master/courses/machine_learning/deepdive2/structured/labs/serving/application/lib/oauth2client/client.py"
click node_oauth_helpers "https://github.com/googlecloudplatform/training-data-analyst/blob/master/courses/machine_learning/deepdive2/structured/labs/serving/application/lib/oauth2client/_helpers.py"
click node_oauth_service "https://github.com/googlecloudplatform/training-data-analyst/blob/master/courses/machine_learning/deepdive2/structured/labs/serving/application/lib/oauth2client/service_account.py"
classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_tf_runner,node_batch_inference,node_stream_inference,node_beam_runner toneBlue
class node_radar_pipeline,node_archive_list,node_archive_read,node_ap_detector,node_bigquery toneAmber
class node_traffic_pipeline toneMint
class node_flask,node_flask_json,node_flask_logging,node_werkzeug,node_werkzeug_http,node_werkzeug_wsgi,node_oauth_client,node_oauth_helpers,node_oauth_service toneRose
class node_operator,node_pubsub toneIndigo
```
## Components
### ML Serving
- [Inference runner](https://github.com/googlecloudplatform/training-data-analyst/blob/master/blogs/tf_dataflow_serving/run_pipeline.py): `blogs/tf_dataflow_serving/run_pipeline.py`
- [Batch inference](https://github.com/googlecloudplatform/training-data-analyst/tree/master/blogs/tf_dataflow_serving/pipelines): `blogs/tf_dataflow_serving/pipelines`
- [Stream inference](https://github.com/googlecloudplatform/training-data-analyst/tree/master/blogs/tf_dataflow_serving/pipelines): `blogs/tf_dataflow_serving/pipelines`
- [Beam execution](https://github.com/googlecloudplatform/training-data-analyst/blob/master/blogs/tf_dataflow_serving/run_pipeline.py): `blogs/tf_dataflow_serving/run_pipeline.py`
### Radar Processing
- [Radar pipeline](https://github.com/googlecloudplatform/training-data-analyst/blob/master/blogs/nexrad2/src/com/google/cloud/public_datasets/nexrad2/APPipeline.java): `blogs/nexrad2/src/com/google/cloud/public_datasets/nexrad2/APPipeline.java`
- [Archive listing](https://github.com/googlecloudplatform/training-data-analyst/blob/master/blogs/nexrad2/src/com/google/cloud/public_datasets/nexrad2/GcsNexradL2List.java): `blogs/nexrad2/src/com/google/cloud/public_datasets/nexrad2/GcsNexradL2List.java`
- [Archive reading](https://github.com/googlecloudplatform/training-data-analyst/blob/master/blogs/nexrad2/src/com/google/cloud/public_datasets/nexrad2/GcsNexradL2Read.java): `blogs/nexrad2/src/com/google/cloud/public_datasets/nexrad2/GcsNexradL2Read.java`
- [AP detection](https://github.com/googlecloudplatform/training-data-analyst/blob/master/blogs/nexrad2/src/com/google/cloud/public_datasets/nexrad2/APDetector.java): `blogs/nexrad2/src/com/google/cloud/public_datasets/nexrad2/APDetector.java`
### Traffic Analytics
- [Traffic pipeline](https://github.com/googlecloudplatform/training-data-analyst/blob/master/quests/dataflow_python/5_Streaming_Analytics/solution/streaming_minute_traffic_pipeline.py): `quests/dataflow_python/5_Streaming_Analytics/solution/streaming_minute_traffic_pipeline.py`
### Web Runtime
- [Flask framework](https://github.com/googlecloudplatform/training-data-analyst/tree/master/courses/machine_learning/deepdive2/structured/labs/serving/application/lib/flask): `courses/machine_learning/deepdive2/structured/labs/serving/application/lib/flask`
- [Flask JSON](https://github.com/googlecloudplatform/training-data-analyst/blob/master/courses/machine_learning/deepdive2/structured/labs/serving/application/lib/flask/json.py): `courses/machine_learning/deepdive2/structured/labs/serving/application/lib/flask/json.py`
- [Flask logging](https://github.com/googlecloudplatform/training-data-analyst/blob/master/courses/machine_learning/deepdive2/structured/labs/serving/application/lib/flask/logging.py): `courses/machine_learning/deepdive2/structured/labs/serving/application/lib/flask/logging.py`
- [Werkzeug utilities](https://github.com/googlecloudplatform/training-data-analyst/tree/master/courses/machine_learning/deepdive2/structured/labs/serving/application/lib/werkzeug): `courses/machine_learning/deepdive2/structured/labs/serving/application/lib/werkzeug`
- [HTTP utilities](https://github.com/googlecloudplatform/training-data-analyst/blob/master/courses/machine_learning/deepdive2/structured/labs/serving/application/lib/werkzeug/http.py): `courses/machine_learning/deepdive2/structured/labs/serving/application/lib/werkzeug/http.py`
- [WSGI server](https://github.com/googlecloudplatform/training-data-analyst/blob/master/courses/machine_learning/deepdive2/structured/labs/serving/application/lib/werkzeug/serving.py): `courses/machine_learning/deepdive2/structured/labs/serving/application/lib/werkzeug/serving.py`
- [OAuth client](https://github.com/googlecloudplatform/training-data-analyst/blob/master/courses/machine_learning/deepdive2/structured/labs/serving/application/lib/oauth2client/client.py): `courses/machine_learning/deepdive2/structured/labs/serving/application/lib/oauth2client/client.py`
- [OAuth helpers](https://github.com/googlecloudplatform/training-data-analyst/blob/master/courses/machine_learning/deepdive2/structured/labs/serving/application/lib/oauth2client/_helpers.py): `courses/machine_learning/deepdive2/structured/labs/serving/application/lib/oauth2client/_helpers.py`
- [Service account](https://github.com/googlecloudplatform/training-data-analyst/blob/master/courses/machine_learning/deepdive2/structured/labs/serving/application/lib/oauth2client/service_account.py): `courses/machine_learning/deepdive2/structured/labs/serving/application/lib/oauth2client/service_account.py`
### Other components
- **Pipeline operator**
- **Pub/Sub**
- **BigQuery**
## Connections
- Pipeline operator → Inference runner: runs (evidence: `blogs/tf_dataflow_serving/run_pipeline.py`)
- Inference runner → Batch inference: dispatches batch (evidence: `blogs/tf_dataflow_serving/run_pipeline.py`)
- Inference runner → Stream inference: dispatches stream (evidence: `blogs/tf_dataflow_serving/run_pipeline.py`)
- Inference runner → Beam execution: configures job (evidence: `blogs/tf_dataflow_serving/run_pipeline.py`)
- Radar pipeline → Archive listing: references (evidence: `blogs/nexrad2/src/com/google/cloud/public_datasets/nexrad2/APPipeline.java`)
- Radar pipeline → Archive reading: references (evidence: `blogs/nexrad2/src/com/google/cloud/public_datasets/nexrad2/APPipeline.java`)
- Radar pipeline → AP detection: references (evidence: `blogs/nexrad2/src/com/google/cloud/public_datasets/nexrad2/APPipeline.java`)
- Traffic pipeline → Pub/Sub: reads messages (evidence: `quests/dataflow_python/5_Streaming_Analytics/solution/streaming_minute_traffic_pipeline.py`)
- Traffic pipeline → BigQuery: writes rows (evidence: `quests/dataflow_python/5_Streaming_Analytics/solution/streaming_minute_traffic_pipeline.py`)
- Flask JSON → Flask framework: uses context (evidence: `courses/machine_learning/deepdive2/structured/labs/serving/application/lib/flask/json.py`)
- Flask logging → Flask framework: uses request context (evidence: `courses/machine_learning/deepdive2/structured/labs/serving/application/lib/flask/logging.py`)
- Flask JSON → HTTP utilities: uses HTTP helpers (evidence: `courses/machine_learning/deepdive2/structured/labs/serving/application/lib/flask/json.py`)
- Flask logging → Werkzeug utilities: uses local proxy (evidence: `courses/machine_learning/deepdive2/structured/labs/serving/application/lib/flask/logging.py`)
- OAuth client → OAuth helpers: uses helpers (evidence: `courses/machine_learning/deepdive2/structured/labs/serving/application/lib/oauth2client/client.py`)