# thanhdat24020062/bigdata-cloud-lab1 architecture

> Architecture diagram of the GitHub repository thanhdat24020062/bigdata-cloud-lab1, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/thanhdat24020062/bigdata-cloud-lab1
- Repository: https://github.com/thanhdat24020062/bigdata-cloud-lab1

## Overview

This repository is a cloud-native big-data governance lab bundle, not a single deployable application. Its user workflow is to configure Kubernetes guardrails and storage, establish S3 identities and access controls, run probes and benchmarks, test recovery, and review the resulting evidence. The README documents task-specific manifests and evidence; sampled scripts show identity generation, S3 operations, and a final evidence/governance audit. Task D/E implementation details are represented at their documented directory level because their source was not sampled.

Main entry points: [`scripts/audit_role_e.py`](https://github.com/thanhdat24020062/bigdata-cloud-lab1/blob/main/scripts/audit_role_e.py) (Role E auditor), [`scripts/task-b/s3lab.py`](https://github.com/thanhdat24020062/bigdata-cloud-lab1/blob/main/scripts/task-b/s3lab.py) (S3 probe client), [`evidence`](https://github.com/thanhdat24020062/bigdata-cloud-lab1/tree/main/evidence) (Task evidence), [`manifests/task-b`](https://github.com/thanhdat24020062/bigdata-cloud-lab1/tree/main/manifests/task-b) (Storage deployment)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_platform["Platform Setup"]
  node_guardrails["Guardrails &amp; quotas"]
  node_cluster_storage["Storage deployment"]
  node_storage_service[("S3-compatible store")]
end

subgraph group_access["Storage &amp; Access"]
  node_identity_gen["Identity generator<br/>[make_identities.py]"]
  node_task_b_identity_gen["Task B identity generator<br/>[make_identities.py]"]
  node_identity_files["Identity &amp; policy files"]
  node_access_controls["S3, RBAC &amp; network controls"]
end

subgraph group_validation["Experiments &amp; Recovery"]
  node_s3_client["S3 probe client<br/>[s3lab.py]"]
  node_benchmark["Performance experiments"]
  node_recovery["Reliability &amp; recovery"]
  node_evidence["Task evidence"]
end

subgraph group_governance["Governance &amp; Review"]
  node_governance_config["Governance records"]
  node_audit["Role E auditor<br/>[audit_role_e.py]"]
  node_results["Audit &amp; security results"]
end

node_operator(("Lab operator"))

node_operator -->|"configures"| node_guardrails
node_operator -->|"deploys"| node_cluster_storage
node_cluster_storage -.->|"deploys"| node_storage_service
node_operator -.->|"runs"| node_identity_gen
node_operator -.->|"runs"| node_task_b_identity_gen
node_identity_gen -->|"writes policies"| node_identity_files
node_task_b_identity_gen -->|"writes policies"| node_identity_files
node_operator -->|"configures"| node_access_controls
node_operator -->|"runs probes"| node_s3_client
node_s3_client -->|"sends S3 requests"| node_storage_service
node_operator -->|"runs experiments"| node_benchmark
node_operator -->|"tests recovery"| node_recovery
node_s3_client -.->|"emits JSONL"| node_evidence
node_operator -->|"records outcomes"| node_evidence
node_operator -->|"maintains records"| node_governance_config
node_audit -->|"checks artifacts"| node_evidence
node_audit -->|"validates records"| node_governance_config
node_audit -->|"checks results"| node_results
node_audit -.->|"reports findings"| node_operator

click node_guardrails "https://github.com/thanhdat24020062/bigdata-cloud-lab1/tree/main/manifests/task-a"
click node_cluster_storage "https://github.com/thanhdat24020062/bigdata-cloud-lab1/tree/main/manifests/task-b"
click node_identity_gen "https://github.com/thanhdat24020062/bigdata-cloud-lab1/blob/main/scripts/make_identities.py"
click node_task_b_identity_gen "https://github.com/thanhdat24020062/bigdata-cloud-lab1/blob/main/scripts/task-b/make_identities.py"
click node_identity_files "https://github.com/thanhdat24020062/bigdata-cloud-lab1/blob/main/governance/policies-redacted.json"
click node_access_controls "https://github.com/thanhdat24020062/bigdata-cloud-lab1/tree/main/manifests/task-c"
click node_s3_client "https://github.com/thanhdat24020062/bigdata-cloud-lab1/blob/main/scripts/task-b/s3lab.py"
click node_benchmark "https://github.com/thanhdat24020062/bigdata-cloud-lab1/tree/main/manifests/task-d"
click node_recovery "https://github.com/thanhdat24020062/bigdata-cloud-lab1/tree/main/manifests/task-e"
click node_evidence "https://github.com/thanhdat24020062/bigdata-cloud-lab1/tree/main/evidence"
click node_governance_config "https://github.com/thanhdat24020062/bigdata-cloud-lab1/tree/main/governance"
click node_audit "https://github.com/thanhdat24020062/bigdata-cloud-lab1/blob/main/scripts/audit_role_e.py"
click node_results "https://github.com/thanhdat24020062/bigdata-cloud-lab1/blob/main/security-results.csv"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_guardrails,node_cluster_storage,node_storage_service toneBlue
class node_identity_gen,node_task_b_identity_gen,node_identity_files,node_access_controls toneAmber
class node_s3_client,node_benchmark,node_recovery,node_evidence toneMint
class node_governance_config,node_audit,node_results toneRose
class node_operator toneIndigo
```

## Components

### Platform Setup

- [Guardrails & quotas](https://github.com/thanhdat24020062/bigdata-cloud-lab1/tree/main/manifests/task-a): `manifests/task-a`
- [Storage deployment](https://github.com/thanhdat24020062/bigdata-cloud-lab1/tree/main/manifests/task-b): `manifests/task-b`
- **S3-compatible store**

### Storage & Access

- [Identity generator](https://github.com/thanhdat24020062/bigdata-cloud-lab1/blob/main/scripts/make_identities.py): `scripts/make_identities.py`
- [Task B identity generator](https://github.com/thanhdat24020062/bigdata-cloud-lab1/blob/main/scripts/task-b/make_identities.py): `scripts/task-b/make_identities.py`
- [Identity & policy files](https://github.com/thanhdat24020062/bigdata-cloud-lab1/blob/main/governance/policies-redacted.json): `governance/policies-redacted.json`
- [S3, RBAC & network controls](https://github.com/thanhdat24020062/bigdata-cloud-lab1/tree/main/manifests/task-c): `manifests/task-c`

### Experiments & Recovery

- [S3 probe client](https://github.com/thanhdat24020062/bigdata-cloud-lab1/blob/main/scripts/task-b/s3lab.py): `scripts/task-b/s3lab.py`
- [Performance experiments](https://github.com/thanhdat24020062/bigdata-cloud-lab1/tree/main/manifests/task-d): `manifests/task-d`
- [Reliability & recovery](https://github.com/thanhdat24020062/bigdata-cloud-lab1/tree/main/manifests/task-e): `manifests/task-e`
- [Task evidence](https://github.com/thanhdat24020062/bigdata-cloud-lab1/tree/main/evidence): `evidence`

### Governance & Review

- [Governance records](https://github.com/thanhdat24020062/bigdata-cloud-lab1/tree/main/governance): `governance`
- [Role E auditor](https://github.com/thanhdat24020062/bigdata-cloud-lab1/blob/main/scripts/audit_role_e.py): `scripts/audit_role_e.py`
- [Audit & security results](https://github.com/thanhdat24020062/bigdata-cloud-lab1/blob/main/security-results.csv): `security-results.csv`

### Other components

- **Lab operator**

## Connections

- Lab operator → Guardrails & quotas: configures (evidence: `README.md`)
- Lab operator → Storage deployment: deploys (evidence: `README.md`)
- Storage deployment → S3-compatible store: deploys
- Lab operator → Identity generator: runs
- Lab operator → Task B identity generator: runs
- Identity generator → Identity & policy files: writes policies (evidence: `scripts/make_identities.py`)
- Task B identity generator → Identity & policy files: writes policies (evidence: `scripts/task-b/make_identities.py`)
- Lab operator → S3, RBAC & network controls: configures (evidence: `README.md`)
- Lab operator → S3 probe client: runs probes (evidence: `scripts/task-b/s3lab.py`)
- S3 probe client → S3-compatible store: sends S3 requests (evidence: `scripts/task-b/s3lab.py`)
- Lab operator → Performance experiments: runs experiments (evidence: `README.md`)
- Lab operator → Reliability & recovery: tests recovery (evidence: `README.md`)
- S3 probe client → Task evidence: emits JSONL
- Lab operator → Task evidence: records outcomes (evidence: `README.md`)
- Lab operator → Governance records: maintains records (evidence: `README.md`)
- Role E auditor → Task evidence: checks artifacts (evidence: `scripts/audit_role_e.py`)
- Role E auditor → Governance records: validates records (evidence: `scripts/audit_role_e.py`)
- Role E auditor → Audit & security results: checks results (evidence: `scripts/audit_role_e.py`)
- Role E auditor → Lab operator: reports findings
