# bipeen-ranjan/automated-data-quality-data-observability-platform architecture

> Architecture diagram of the GitHub repository bipeen-ranjan/automated-data-quality-data-observability-platform, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bipeen-ranjan/automated-data-quality-data-observability-platform
- Repository: https://github.com/bipeen-ranjan/automated-data-quality-data-observability-platform

## Overview

The repository is a data-quality and observability application: users upload CSV or Excel datasets through a Streamlit dashboard or FastAPI endpoint, then receive profiling, issue checks, quality dimensions, and a weighted score. The analyzer coordinates loading, profiling, checks, and scoring; the API also persists run summaries in PostgreSQL. The README is empty, so capability descriptions rely on the supplied source excerpts and index; profiler implementation details were not sampled.

Main entry points: [`app/analyzer.py`](https://github.com/bipeen-ranjan/automated-data-quality-data-observability-platform/blob/main/app/analyzer.py) (Analysis Orchestrator), [`app/quality_engine.py`](https://github.com/bipeen-ranjan/automated-data-quality-data-observability-platform/blob/main/app/quality_engine.py) (Quality Engine), [`app/api/main.py`](https://github.com/bipeen-ranjan/automated-data-quality-data-observability-platform/blob/main/app/api/main.py) (FastAPI Upload API), [`app/quality_checks.py`](https://github.com/bipeen-ranjan/automated-data-quality-data-observability-platform/blob/main/app/quality_checks.py) (Quality Checks)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interfaces["User Interfaces"]
  node_dashboard["Streamlit Dashboard<br/>[dashboard.py]"]
  node_api["FastAPI Upload API<br/>[main.py]"]
end

subgraph group_analysis["Analysis Workflow"]
  node_analyzer["Analysis Orchestrator<br/>[analyzer.py]"]
  node_loader["Dataset Loader<br/>[data_loader.py]"]
  node_profiler["Dataset Profiler<br/>[profiler.py]"]
end

subgraph group_quality["Quality Assessment"]
  node_engine["Quality Engine<br/>[quality_engine.py]"]
  node_classifier["Column Classifier"]
  node_checks["Quality Checks<br/>[quality_checks.py]"]
  node_rules["Validation Rules"]
  node_result["Check Result Model<br/>[quality_result.py]"]
  node_scoring["Quality Scoring<br/>[scoring.py]"]
end

subgraph group_persistence["Run Persistence"]
  node_repository["Analysis Repository"]
  node_database["Database Connector<br/>[database.py]"]
end

node_user(("Dataset User"))
node_postgres[("PostgreSQL")]

node_user -->|"uploads dataset"| node_dashboard
node_user -->|"uploads dataset"| node_api
node_dashboard -->|"runs analysis"| node_analyzer
node_api -->|"runs analysis"| node_analyzer
node_api -->|"saves run summary"| node_repository
node_analyzer -->|"loads dataset"| node_loader
node_analyzer -->|"profiles data"| node_profiler
node_analyzer -->|"runs checks"| node_engine
node_analyzer -->|"calculates scores"| node_scoring
node_engine -->|"classifies columns"| node_classifier
node_engine -->|"runs checks"| node_checks
node_checks -->|"applies validators"| node_rules
node_checks -->|"creates results"| node_result
node_engine -->|"summarizes results"| node_result
node_repository -->|"gets connection"| node_database
node_database -->|"connects to"| node_postgres

click node_dashboard "https://github.com/bipeen-ranjan/automated-data-quality-data-observability-platform/blob/main/app/dashboard.py"
click node_api "https://github.com/bipeen-ranjan/automated-data-quality-data-observability-platform/blob/main/app/api/main.py"
click node_analyzer "https://github.com/bipeen-ranjan/automated-data-quality-data-observability-platform/blob/main/app/analyzer.py"
click node_loader "https://github.com/bipeen-ranjan/automated-data-quality-data-observability-platform/blob/main/app/data_loader.py"
click node_profiler "https://github.com/bipeen-ranjan/automated-data-quality-data-observability-platform/blob/main/app/profiler.py"
click node_engine "https://github.com/bipeen-ranjan/automated-data-quality-data-observability-platform/blob/main/app/quality_engine.py"
click node_classifier "https://github.com/bipeen-ranjan/automated-data-quality-data-observability-platform/blob/main/app/column_classifier.py"
click node_checks "https://github.com/bipeen-ranjan/automated-data-quality-data-observability-platform/blob/main/app/quality_checks.py"
click node_rules "https://github.com/bipeen-ranjan/automated-data-quality-data-observability-platform/blob/main/app/validation_rules.py"
click node_result "https://github.com/bipeen-ranjan/automated-data-quality-data-observability-platform/blob/main/app/quality_result.py"
click node_scoring "https://github.com/bipeen-ranjan/automated-data-quality-data-observability-platform/blob/main/app/scoring.py"
click node_repository "https://github.com/bipeen-ranjan/automated-data-quality-data-observability-platform/blob/main/app/repositories/analysis_repository.py"
click node_database "https://github.com/bipeen-ranjan/automated-data-quality-data-observability-platform/blob/main/app/database.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_dashboard,node_api,node_user toneBlue
class node_analyzer,node_loader,node_profiler,node_postgres toneAmber
class node_engine,node_classifier,node_checks,node_rules,node_result,node_scoring toneMint
class node_repository,node_database toneRose
```

## Components

### User Interfaces

- [Streamlit Dashboard](https://github.com/bipeen-ranjan/automated-data-quality-data-observability-platform/blob/main/app/dashboard.py): `app/dashboard.py`
- [FastAPI Upload API](https://github.com/bipeen-ranjan/automated-data-quality-data-observability-platform/blob/main/app/api/main.py): `app/api/main.py`

### Analysis Workflow

- [Analysis Orchestrator](https://github.com/bipeen-ranjan/automated-data-quality-data-observability-platform/blob/main/app/analyzer.py): `app/analyzer.py`
- [Dataset Loader](https://github.com/bipeen-ranjan/automated-data-quality-data-observability-platform/blob/main/app/data_loader.py): `app/data_loader.py`
- [Dataset Profiler](https://github.com/bipeen-ranjan/automated-data-quality-data-observability-platform/blob/main/app/profiler.py): `app/profiler.py`

### Quality Assessment

- [Quality Engine](https://github.com/bipeen-ranjan/automated-data-quality-data-observability-platform/blob/main/app/quality_engine.py): `app/quality_engine.py`
- [Column Classifier](https://github.com/bipeen-ranjan/automated-data-quality-data-observability-platform/blob/main/app/column_classifier.py): `app/column_classifier.py`
- [Quality Checks](https://github.com/bipeen-ranjan/automated-data-quality-data-observability-platform/blob/main/app/quality_checks.py): `app/quality_checks.py`
- [Validation Rules](https://github.com/bipeen-ranjan/automated-data-quality-data-observability-platform/blob/main/app/validation_rules.py): `app/validation_rules.py`
- [Check Result Model](https://github.com/bipeen-ranjan/automated-data-quality-data-observability-platform/blob/main/app/quality_result.py): `app/quality_result.py`
- [Quality Scoring](https://github.com/bipeen-ranjan/automated-data-quality-data-observability-platform/blob/main/app/scoring.py): `app/scoring.py`

### Run Persistence

- [Analysis Repository](https://github.com/bipeen-ranjan/automated-data-quality-data-observability-platform/blob/main/app/repositories/analysis_repository.py): `app/repositories/analysis_repository.py`
- [Database Connector](https://github.com/bipeen-ranjan/automated-data-quality-data-observability-platform/blob/main/app/database.py): `app/database.py`

### Other components

- **Dataset User**
- **PostgreSQL**

## Connections

- Dataset User → Streamlit Dashboard: uploads dataset (evidence: `app/dashboard.py`)
- Dataset User → FastAPI Upload API: uploads dataset (evidence: `app/api/main.py`)
- Streamlit Dashboard → Analysis Orchestrator: runs analysis (evidence: `app/dashboard.py`)
- FastAPI Upload API → Analysis Orchestrator: runs analysis (evidence: `app/api/main.py`)
- FastAPI Upload API → Analysis Repository: saves run summary (evidence: `app/api/main.py`)
- Analysis Orchestrator → Dataset Loader: loads dataset (evidence: `app/analyzer.py`)
- Analysis Orchestrator → Dataset Profiler: profiles data (evidence: `app/analyzer.py`)
- Analysis Orchestrator → Quality Engine: runs checks (evidence: `app/analyzer.py`)
- Analysis Orchestrator → Quality Scoring: calculates scores (evidence: `app/analyzer.py`)
- Quality Engine → Column Classifier: classifies columns (evidence: `app/quality_engine.py`)
- Quality Engine → Quality Checks: runs checks (evidence: `app/quality_engine.py`)
- Quality Checks → Validation Rules: applies validators (evidence: `app/quality_checks.py`)
- Quality Checks → Check Result Model: creates results (evidence: `app/quality_checks.py`)
- Quality Engine → Check Result Model: summarizes results (evidence: `app/quality_engine.py`)
- Analysis Repository → Database Connector: gets connection (evidence: `app/repositories/analysis_repository.py`)
- Database Connector → PostgreSQL: connects to (evidence: `app/database.py`)
