# rajpandeygithub/automated-bilingual-complaint-system architecture

> Architecture diagram of the GitHub repository rajpandeygithub/automated-bilingual-complaint-system, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rajpandeygithub/automated-bilingual-complaint-system
- Repository: https://github.com/rajpandeygithub/automated-bilingual-complaint-system

## Overview

The repository implements a bilingual banking-complaint system: users submit English or Hindi complaints, which the API validates and transforms, checks for drift, classifies by product and department, and records. Separate Airflow data-preparation and Kubeflow model-training workflows support the serving system. The README also describes language-aware agent assignment, but the supplied source does not show its implementation. Edges reflect shown calls or documented workflow; unsampled subsystems are included without asserting unverified connections.

Main entry points: [`train/submit_train_pipeline.py`](https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/train/submit_train_pipeline.py) (Training orchestrator), [`data_preprocessing_pipeline/dags/airflow.py`](https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/data_preprocessing_pipeline/dags/airflow.py) (Preprocessing DAGs), [`backend/server.py`](https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/backend/server.py) (Prediction API), [`frontend/frontend.py`](https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/frontend/frontend.py) (Complaint portal)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_serving["Complaint serving"]
  node_portal["Complaint portal<br/>[frontend.py]"]
  node_frontend_api["API client<br/>[backend.py]"]
  node_api["Prediction API<br/>[server.py]"]
  node_transform["Text validation and cleanup<br/>[preprocessing.py]"]
  node_inference["Model inference<br/>[inference.py]"]
  node_vertex["Product and department models"]
  node_prediction_store[("Prediction records")]
  node_bq_ops["BigQuery writer"]
  node_agent_assignment["Language-aware agent assignment"]
end

subgraph group_prep["Data preparation"]
  node_airflow["Preprocessing DAGs<br/>[airflow.py]"]
  node_quality["Data quality checks<br/>[data_quality.py]"]
  node_deidentify["Data de-identification"]
  node_batch_clean["Batch cleaning<br/>[preprocessing.py]"]
  node_statistics["Statistics generation"]
  node_agent_data["Agent dataset preparation<br/>[agent_dataset.py]"]
end

subgraph group_training["Model lifecycle"]
  node_train_pipeline["Training orchestrator"]
  node_get_data["Training data acquisition<br/>[get_data.py]"]
  node_prepare_data["Tokenized dataset preparation<br/>[prepare_data.py]"]
  node_model_train["Model training<br/>[hf_model_train.py]"]
  node_model_test["Model evaluation<br/>[hf_model_test.py]"]
  node_select_model["Best model selection"]
  node_bias["Bias assessment<br/>[bias_detection.py]"]
  node_registry["Model registry<br/>[model_registry.py]"]
  node_deployment["Model deployment"]
end

subgraph group_monitoring["Monitoring and operations"]
  node_drift["Drift detection"]
  node_drift_english["English drift support<br/>[Data_drift.py]"]
  node_airflow_alerts["Pipeline email alerts<br/>[failure_email.py]"]
  node_success_alerts["Pipeline success alerts<br/>[success_email.py]"]
  node_slack(("Slack notifications"))
end

node_customer(("Complaint submitter"))

node_customer -->|"submits complaint"| node_portal
node_portal -->|"sends text"| node_frontend_api
node_frontend_api -->|"posts complaint"| node_api
node_api -->|"validates and cleans"| node_transform
node_api -->|"requests predictions"| node_inference
node_inference -->|"calls endpoints"| node_vertex
node_api -->|"writes prediction"| node_bq_ops
node_bq_ops -->|"inserts record"| node_prediction_store
node_api -.->|"checks complaint"| node_drift
node_api -.->|"checks complaint"| node_drift_english
node_airflow -->|"runs checks"| node_quality
node_airflow -->|"runs de-identification"| node_deidentify
node_airflow -->|"runs cleaning"| node_batch_clean
node_airflow -->|"generates statistics"| node_statistics
node_airflow -->|"sends failure alerts"| node_airflow_alerts
node_airflow -->|"sends success alerts"| node_success_alerts
node_airflow -.->|"prepares agent data"| node_agent_data
node_train_pipeline -->|"acquires data"| node_get_data
node_train_pipeline -->|"prepares datasets"| node_prepare_data
node_train_pipeline -->|"trains models"| node_model_train
node_train_pipeline -->|"tests models"| node_model_test
node_train_pipeline -->|"selects best model"| node_select_model
node_train_pipeline -->|"assesses bias"| node_bias
node_train_pipeline -.->|"registers model"| node_registry
node_train_pipeline -->|"deploys conditionally"| node_deployment
node_train_pipeline -.->|"notifies pipeline"| node_slack
node_portal -.->|"supports assignment"| node_agent_assignment

click node_portal "https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/frontend/frontend.py"
click node_frontend_api "https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/frontend/backend.py"
click node_api "https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/backend/server.py"
click node_transform "https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/backend/preprocessing.py"
click node_inference "https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/backend/inference.py"
click node_bq_ops "https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/backend/bigquery_operations.py"
click node_airflow "https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/data_preprocessing_pipeline/dags/airflow.py"
click node_quality "https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/data_preprocessing_pipeline/dags/scripts/data_quality.py"
click node_deidentify "https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/data_preprocessing_pipeline/dags/scripts/deidentification.py"
click node_batch_clean "https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/data_preprocessing_pipeline/dags/scripts/preprocessing.py"
click node_statistics "https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/data_preprocessing_pipeline/dags/scripts/statistics_generation.py"
click node_agent_data "https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/data_preprocessing_pipeline/dags/scripts/agent_dataset.py"
click node_train_pipeline "https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/train/submit_train_pipeline.py"
click node_get_data "https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/train/components/get_data.py"
click node_prepare_data "https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/train/components/prepare_data.py"
click node_model_train "https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/train/components/hf_model_train.py"
click node_model_test "https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/train/components/hf_model_test.py"
click node_select_model "https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/train/components/select_best_model.py"
click node_bias "https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/train/components/bias_detection.py"
click node_registry "https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/train/components/model_registry.py"
click node_deployment "https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/train/components/model_deployment.py"
click node_drift "https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/data_drift/drift_cloud_fn_hindi.py"
click node_drift_english "https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/data_drift/Data_drift.py"
click node_airflow_alerts "https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/data_preprocessing_pipeline/dags/scripts/failure_email.py"
click node_success_alerts "https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/data_preprocessing_pipeline/dags/scripts/success_email.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_portal,node_frontend_api,node_api,node_transform,node_inference,node_vertex,node_prediction_store,node_bq_ops,node_agent_assignment toneBlue
class node_airflow,node_quality,node_deidentify,node_batch_clean,node_statistics,node_agent_data toneAmber
class node_train_pipeline,node_get_data,node_prepare_data,node_model_train,node_model_test,node_select_model,node_bias,node_registry,node_deployment toneMint
class node_drift,node_drift_english,node_airflow_alerts,node_success_alerts,node_slack toneRose
class node_customer toneIndigo
```

## Components

### Complaint serving

- [Complaint portal](https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/frontend/frontend.py): `frontend/frontend.py`
- [API client](https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/frontend/backend.py): `frontend/backend.py`
- [Prediction API](https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/backend/server.py): `backend/server.py`
- [Text validation and cleanup](https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/backend/preprocessing.py): `backend/preprocessing.py`
- [Model inference](https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/backend/inference.py): `backend/inference.py`
- **Product and department models**
- **Prediction records**
- [BigQuery writer](https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/backend/bigquery_operations.py): `backend/bigquery_operations.py`
- **Language-aware agent assignment**

### Data preparation

- [Preprocessing DAGs](https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/data_preprocessing_pipeline/dags/airflow.py): `data_preprocessing_pipeline/dags/airflow.py`
- [Data quality checks](https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/data_preprocessing_pipeline/dags/scripts/data_quality.py): `data_preprocessing_pipeline/dags/scripts/data_quality.py`
- [Data de-identification](https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/data_preprocessing_pipeline/dags/scripts/deidentification.py): `data_preprocessing_pipeline/dags/scripts/deidentification.py`
- [Batch cleaning](https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/data_preprocessing_pipeline/dags/scripts/preprocessing.py): `data_preprocessing_pipeline/dags/scripts/preprocessing.py`
- [Statistics generation](https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/data_preprocessing_pipeline/dags/scripts/statistics_generation.py): `data_preprocessing_pipeline/dags/scripts/statistics_generation.py`
- [Agent dataset preparation](https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/data_preprocessing_pipeline/dags/scripts/agent_dataset.py): `data_preprocessing_pipeline/dags/scripts/agent_dataset.py`

### Model lifecycle

- [Training orchestrator](https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/train/submit_train_pipeline.py): `train/submit_train_pipeline.py`
- [Training data acquisition](https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/train/components/get_data.py): `train/components/get_data.py`
- [Tokenized dataset preparation](https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/train/components/prepare_data.py): `train/components/prepare_data.py`
- [Model training](https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/train/components/hf_model_train.py): `train/components/hf_model_train.py`
- [Model evaluation](https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/train/components/hf_model_test.py): `train/components/hf_model_test.py`
- [Best model selection](https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/train/components/select_best_model.py): `train/components/select_best_model.py`
- [Bias assessment](https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/train/components/bias_detection.py): `train/components/bias_detection.py`
- [Model registry](https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/train/components/model_registry.py): `train/components/model_registry.py`
- [Model deployment](https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/train/components/model_deployment.py): `train/components/model_deployment.py`

### Monitoring and operations

- [Drift detection](https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/data_drift/drift_cloud_fn_hindi.py): `data_drift/drift_cloud_fn_hindi.py`
- [English drift support](https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/data_drift/Data_drift.py): `data_drift/Data_drift.py`
- [Pipeline email alerts](https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/data_preprocessing_pipeline/dags/scripts/failure_email.py): `data_preprocessing_pipeline/dags/scripts/failure_email.py`
- [Pipeline success alerts](https://github.com/rajpandeygithub/automated-bilingual-complaint-system/blob/main/data_preprocessing_pipeline/dags/scripts/success_email.py): `data_preprocessing_pipeline/dags/scripts/success_email.py`
- **Slack notifications**

### Other components

- **Complaint submitter**

## Connections

- Complaint submitter → Complaint portal: submits complaint (evidence: `README.md`)
- Complaint portal → API client: sends text (evidence: `frontend/frontend.py`)
- API client → Prediction API: posts complaint (evidence: `frontend/backend.py`)
- Prediction API → Text validation and cleanup: validates and cleans (evidence: `backend/server.py`)
- Prediction API → Model inference: requests predictions (evidence: `backend/server.py`)
- Model inference → Product and department models: calls endpoints (evidence: `backend/inference.py`)
- Prediction API → BigQuery writer: writes prediction (evidence: `backend/server.py`)
- BigQuery writer → Prediction records: inserts record (evidence: `backend/bigquery_operations.py`)
- Prediction API → Drift detection: checks complaint
- Prediction API → English drift support: checks complaint
- Preprocessing DAGs → Data quality checks: runs checks (evidence: `data_preprocessing_pipeline/dags/airflow.py`)
- Preprocessing DAGs → Data de-identification: runs de-identification (evidence: `data_preprocessing_pipeline/dags/airflow.py`)
- Preprocessing DAGs → Batch cleaning: runs cleaning (evidence: `data_preprocessing_pipeline/dags/airflow.py`)
- Preprocessing DAGs → Statistics generation: generates statistics (evidence: `data_preprocessing_pipeline/dags/airflow.py`)
- Preprocessing DAGs → Pipeline email alerts: sends failure alerts (evidence: `data_preprocessing_pipeline/dags/airflow.py`)
- Preprocessing DAGs → Pipeline success alerts: sends success alerts (evidence: `data_preprocessing_pipeline/dags/airflow.py`)
- Preprocessing DAGs → Agent dataset preparation: prepares agent data
- Training orchestrator → Training data acquisition: acquires data (evidence: `train/submit_train_pipeline.py`)
- Training orchestrator → Tokenized dataset preparation: prepares datasets (evidence: `train/submit_train_pipeline.py`)
- Training orchestrator → Model training: trains models (evidence: `train/submit_train_pipeline.py`)
- Training orchestrator → Model evaluation: tests models (evidence: `train/submit_train_pipeline.py`)
- Training orchestrator → Best model selection: selects best model (evidence: `train/submit_train_pipeline.py`)
- Training orchestrator → Bias assessment: assesses bias (evidence: `train/submit_train_pipeline.py`)
- Training orchestrator → Model registry: registers model (evidence: `train/submit_train_pipeline.py`)
- Training orchestrator → Model deployment: deploys conditionally (evidence: `train/submit_train_pipeline.py`)
- Training orchestrator → Slack notifications: notifies pipeline
- Complaint portal → Language-aware agent assignment: supports assignment