# praneetsahoo/hcl_tech_hackathon-team_name- architecture

> Architecture diagram of the GitHub repository praneetsahoo/hcl_tech_hackathon-team_name-, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/praneetsahoo/hcl_tech_hackathon-team_name-
- Repository: https://github.com/praneetsahoo/hcl_tech_hackathon-team_name-

## Overview

RetailBank Analytics turns Day 1/Day 2 raw transaction and master-data files into a cleaned, audited MySQL star schema and 13 KPI views, then presents them in a password-protected dashboard. The principal workflow is batch orchestration → read and cleanse → bronze/quarantine handling → transactional dimension and fact loading → KPI reporting. The graph also preserves data profiling and operational logging. Relationships are limited to calls shown in excerpts or capabilities explicitly documented in the README; unsampled implementation details are not assumed.

Main entry points: [`pipeline/run_pipeline.py`](https://github.com/praneetsahoo/hcl_tech_hackathon-team_name-/blob/main/pipeline/run_pipeline.py) (Batch Orchestrator), [`dashboard/app.py`](https://github.com/praneetsahoo/hcl_tech_hackathon-team_name-/blob/main/dashboard/app.py) (Analytics Dashboard), [`pipeline/db.py`](https://github.com/praneetsahoo/hcl_tech_hackathon-team_name-/blob/main/pipeline/db.py) (Database Access), [`pipeline/readers.py`](https://github.com/praneetsahoo/hcl_tech_hackathon-team_name-/blob/main/pipeline/readers.py) (Source Readers)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_pipeline["Batch Pipeline"]
  node_orchestrator["Batch Orchestrator<br/>[run_pipeline.py]"]
  node_settings["Pipeline Settings<br/>[config.py]"]
  node_reader["Source Readers<br/>[readers.py]"]
  node_bronze["Bronze Ingest<br/>[ingest.py]"]
  node_profiler["Data Profiler<br/>[profile_data.py]"]
end

subgraph group_quality["Data Quality"]
  node_cleanser["Entity Cleansing<br/>[cleanse.py]"]
  node_rules["Cleaning Rules<br/>[rules.py]"]
  node_quarantine[("Quarantine Records<br/>[cleanse.py]")]
  node_dq_audit["DQ and Audit Writes<br/>[load.py]"]
end

subgraph group_storage["Analytics Storage"]
  node_gold_loader["Gold Loader<br/>[load.py]"]
  node_db_access["Database Access<br/>[db.py]"]
  node_mysql[("RetailBank MySQL")]
  node_schema["Star Schema<br/>[schema.sql]"]
  node_kpi_views["KPI Views<br/>[kpi_views.sql]"]
end

subgraph group_product["Dashboard and Operations"]
  node_dashboard["Analytics Dashboard<br/>[app.py]"]
  node_ssm[("AWS Parameter Store")]
  node_cloudwatch[("CloudWatch Logs")]
end

node_source_files["Raw Batch Files"]
node_operator(("Pipeline Operator"))

node_operator -->|"starts batch"| node_orchestrator
node_orchestrator -->|"loads settings"| node_settings
node_orchestrator -->|"reads files"| node_reader
node_reader -->|"parses"| node_source_files
node_orchestrator -->|"uploads raw"| node_bronze
node_bronze -.->|"uses settings"| node_settings
node_profiler -->|"profiles"| node_source_files
node_profiler -->|"salvages JSON"| node_reader
node_orchestrator -->|"processes entities"| node_cleanser
node_cleanser -->|"applies rules"| node_rules
node_orchestrator -->|"routes rejects"| node_quarantine
node_orchestrator -->|"merges and loads"| node_gold_loader
node_orchestrator -->|"writes scorecard"| node_dq_audit
node_orchestrator -->|"opens connection"| node_db_access
node_db_access -->|"connects to"| node_mysql
node_schema -->|"defines tables"| node_mysql
node_kpi_views -->|"defines KPI views"| node_mysql
node_dashboard -->|"queries KPIs"| node_mysql
node_dashboard -->|"runs batch"| node_orchestrator
node_dashboard -->|"gets login secret"| node_ssm
node_db_access -->|"gets DB secret"| node_ssm
node_orchestrator -->|"logs errors"| node_cloudwatch
node_dashboard -->|"shares pipeline logs"| node_cloudwatch

click node_source_files "https://github.com/praneetsahoo/hcl_tech_hackathon-team_name-/tree/main/data/raw"
click node_orchestrator "https://github.com/praneetsahoo/hcl_tech_hackathon-team_name-/blob/main/pipeline/run_pipeline.py"
click node_settings "https://github.com/praneetsahoo/hcl_tech_hackathon-team_name-/blob/main/pipeline/config.py"
click node_reader "https://github.com/praneetsahoo/hcl_tech_hackathon-team_name-/blob/main/pipeline/readers.py"
click node_bronze "https://github.com/praneetsahoo/hcl_tech_hackathon-team_name-/blob/main/pipeline/ingest.py"
click node_profiler "https://github.com/praneetsahoo/hcl_tech_hackathon-team_name-/blob/main/pipeline/profile_data.py"
click node_cleanser "https://github.com/praneetsahoo/hcl_tech_hackathon-team_name-/blob/main/pipeline/cleanse.py"
click node_rules "https://github.com/praneetsahoo/hcl_tech_hackathon-team_name-/blob/main/pipeline/rules.py"
click node_quarantine "https://github.com/praneetsahoo/hcl_tech_hackathon-team_name-/blob/main/pipeline/cleanse.py"
click node_gold_loader "https://github.com/praneetsahoo/hcl_tech_hackathon-team_name-/blob/main/pipeline/load.py"
click node_dq_audit "https://github.com/praneetsahoo/hcl_tech_hackathon-team_name-/blob/main/pipeline/load.py"
click node_db_access "https://github.com/praneetsahoo/hcl_tech_hackathon-team_name-/blob/main/pipeline/db.py"
click node_schema "https://github.com/praneetsahoo/hcl_tech_hackathon-team_name-/blob/main/sql/schema.sql"
click node_kpi_views "https://github.com/praneetsahoo/hcl_tech_hackathon-team_name-/blob/main/sql/kpi_views.sql"
click node_dashboard "https://github.com/praneetsahoo/hcl_tech_hackathon-team_name-/blob/main/dashboard/app.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_orchestrator,node_settings,node_reader,node_bronze,node_profiler toneBlue
class node_cleanser,node_rules,node_quarantine,node_dq_audit toneAmber
class node_gold_loader,node_db_access,node_mysql,node_schema,node_kpi_views toneMint
class node_dashboard,node_ssm,node_cloudwatch toneRose
class node_source_files toneTeal
class node_operator toneIndigo
```

## Components

### Batch Pipeline

- [Batch Orchestrator](https://github.com/praneetsahoo/hcl_tech_hackathon-team_name-/blob/main/pipeline/run_pipeline.py): `pipeline/run_pipeline.py`
- [Pipeline Settings](https://github.com/praneetsahoo/hcl_tech_hackathon-team_name-/blob/main/pipeline/config.py): `pipeline/config.py`
- [Source Readers](https://github.com/praneetsahoo/hcl_tech_hackathon-team_name-/blob/main/pipeline/readers.py): `pipeline/readers.py`
- [Bronze Ingest](https://github.com/praneetsahoo/hcl_tech_hackathon-team_name-/blob/main/pipeline/ingest.py): `pipeline/ingest.py`
- [Data Profiler](https://github.com/praneetsahoo/hcl_tech_hackathon-team_name-/blob/main/pipeline/profile_data.py): `pipeline/profile_data.py`

### Data Quality

- [Entity Cleansing](https://github.com/praneetsahoo/hcl_tech_hackathon-team_name-/blob/main/pipeline/cleanse.py): `pipeline/cleanse.py`
- [Cleaning Rules](https://github.com/praneetsahoo/hcl_tech_hackathon-team_name-/blob/main/pipeline/rules.py): `pipeline/rules.py`
- [Quarantine Records](https://github.com/praneetsahoo/hcl_tech_hackathon-team_name-/blob/main/pipeline/cleanse.py): `pipeline/cleanse.py`
- [DQ and Audit Writes](https://github.com/praneetsahoo/hcl_tech_hackathon-team_name-/blob/main/pipeline/load.py): `pipeline/load.py`

### Analytics Storage

- [Gold Loader](https://github.com/praneetsahoo/hcl_tech_hackathon-team_name-/blob/main/pipeline/load.py): `pipeline/load.py`
- [Database Access](https://github.com/praneetsahoo/hcl_tech_hackathon-team_name-/blob/main/pipeline/db.py): `pipeline/db.py`
- **RetailBank MySQL**
- [Star Schema](https://github.com/praneetsahoo/hcl_tech_hackathon-team_name-/blob/main/sql/schema.sql): `sql/schema.sql`
- [KPI Views](https://github.com/praneetsahoo/hcl_tech_hackathon-team_name-/blob/main/sql/kpi_views.sql): `sql/kpi_views.sql`

### Dashboard and Operations

- [Analytics Dashboard](https://github.com/praneetsahoo/hcl_tech_hackathon-team_name-/blob/main/dashboard/app.py): `dashboard/app.py`
- **AWS Parameter Store**
- **CloudWatch Logs**

### Other components

- [Raw Batch Files](https://github.com/praneetsahoo/hcl_tech_hackathon-team_name-/tree/main/data/raw): `data/raw`
- **Pipeline Operator**

## Connections

- Pipeline Operator → Batch Orchestrator: starts batch (evidence: `README.md`)
- Batch Orchestrator → Pipeline Settings: loads settings (evidence: `pipeline/run_pipeline.py`)
- Batch Orchestrator → Source Readers: reads files (evidence: `pipeline/run_pipeline.py`)
- Source Readers → Raw Batch Files: parses (evidence: `pipeline/readers.py`)
- Batch Orchestrator → Bronze Ingest: uploads raw (evidence: `pipeline/run_pipeline.py`)
- Bronze Ingest → Pipeline Settings: uses settings (evidence: `pipeline/ingest.py`)
- Data Profiler → Raw Batch Files: profiles (evidence: `pipeline/profile_data.py`)
- Data Profiler → Source Readers: salvages JSON (evidence: `pipeline/profile_data.py`)
- Batch Orchestrator → Entity Cleansing: processes entities (evidence: `pipeline/run_pipeline.py`)
- Entity Cleansing → Cleaning Rules: applies rules (evidence: `pipeline/cleanse.py`)
- Batch Orchestrator → Quarantine Records: routes rejects (evidence: `README.md`)
- Batch Orchestrator → Gold Loader: merges and loads (evidence: `pipeline/run_pipeline.py`)
- Batch Orchestrator → DQ and Audit Writes: writes scorecard (evidence: `pipeline/run_pipeline.py`)
- Batch Orchestrator → Database Access: opens connection (evidence: `pipeline/run_pipeline.py`)
- Database Access → RetailBank MySQL: connects to (evidence: `pipeline/db.py`)
- Star Schema → RetailBank MySQL: defines tables (evidence: `README.md`)
- KPI Views → RetailBank MySQL: defines KPI views (evidence: `README.md`)
- Analytics Dashboard → RetailBank MySQL: queries KPIs (evidence: `README.md`)
- Analytics Dashboard → Batch Orchestrator: runs batch (evidence: `dashboard/app.py`)
- Analytics Dashboard → AWS Parameter Store: gets login secret (evidence: `dashboard/app.py`)
- Database Access → AWS Parameter Store: gets DB secret (evidence: `pipeline/db.py`)
- Batch Orchestrator → CloudWatch Logs: logs errors (evidence: `README.md`)
- Analytics Dashboard → CloudWatch Logs: shares pipeline logs (evidence: `dashboard/app.py`)
