How thanhdat24020062/bigdata-cloud-lab1 works
This repository is a cloud-native big-data governance lab bundle, not a single deployable application. Its user workflow is to configure Kubernetes guardrails and storage, establish S3 identities and access controls, run probes and benchmarks, test recovery, and review the resulting evidence. The README documents task-specific manifests and evidence; sampled scripts show identity generation, S3 operations, and a final evidence/governance audit. Task D/E implementation details are represented at their documented directory level because their source was not sampled.
Main entry points: audit_role_e.py Role E auditor · s3lab.py S3 probe client · evidence Task evidence · task-b Storage deployment
Components 15
Platform Setup
- Guardrails & quotas
manifests/task-a - Storage deployment
manifests/task-b - S3-compatible store
Storage & Access
- Identity generator
scripts/make_identities.py - Task B identity generator
scripts/task-b/make_identities.py - Identity & policy files
governance/policies-redacted.json - S3, RBAC & network controls
manifests/task-c
Experiments & Recovery
- S3 probe client
scripts/task-b/s3lab.py - Performance experiments
manifests/task-d - Reliability & recovery
manifests/task-e - Task evidence
evidence
Governance & Review
- Governance records
governance - Role E auditor
scripts/audit_role_e.py - Audit & security results
security-results.csv
Other components
- Lab operator
Connections 19
- Lab operator → Guardrails & quotas: configures
README.md - Lab operator → Storage deployment: deploys
README.md - Storage deployment → S3-compatible store: deploys
- Lab operator → Identity generator: runs
- Lab operator → Task B identity generator: runs
- Identity generator → Identity & policy files: writes policies
scripts/make_identities.py - Task B identity generator → Identity & policy files: writes policies
scripts/task-b/make_identities.py - Lab operator → S3, RBAC & network controls: configures
README.md - Lab operator → S3 probe client: runs probes
scripts/task-b/s3lab.py - S3 probe client → S3-compatible store: sends S3 requests
scripts/task-b/s3lab.py - Lab operator → Performance experiments: runs experiments
README.md - Lab operator → Reliability & recovery: tests recovery
README.md - S3 probe client → Task evidence: emits JSONL
- Lab operator → Task evidence: records outcomes
README.md - Lab operator → Governance records: maintains records
README.md - Role E auditor → Task evidence: checks artifacts
scripts/audit_role_e.py - Role E auditor → Governance records: validates records
scripts/audit_role_e.py - Role E auditor → Audit & security results: checks results
scripts/audit_role_e.py - Role E auditor → Lab operator: reports findings