How bipeen-ranjan/automated-data-quality-data-observability-platform works
The repository is a data-quality and observability application: users upload CSV or Excel datasets through a Streamlit dashboard or FastAPI endpoint, then receive profiling, issue checks, quality dimensions, and a weighted score. The analyzer coordinates loading, profiling, checks, and scoring; the API also persists run summaries in PostgreSQL. The README is empty, so capability descriptions rely on the supplied source excerpts and index; profiler implementation details were not sampled.
Main entry points: analyzer.py Analysis Orchestrator · quality_engine.py Quality Engine · main.py FastAPI Upload API · quality_checks.py Quality Checks
Components 15
User Interfaces
- Streamlit Dashboard
app/dashboard.py - FastAPI Upload API
app/api/main.py
Analysis Workflow
- Analysis Orchestrator
app/analyzer.py - Dataset Loader
app/data_loader.py - Dataset Profiler
app/profiler.py
Quality Assessment
- Quality Engine
app/quality_engine.py - Column Classifier
app/column_classifier.py - Quality Checks
app/quality_checks.py - Validation Rules
app/validation_rules.py - Check Result Model
app/quality_result.py - Quality Scoring
app/scoring.py
Run Persistence
- Analysis Repository
app/repositories/analysis_repository.py - Database Connector
app/database.py
Other components
- Dataset User
- PostgreSQL
Connections 16
- Dataset User → Streamlit Dashboard: uploads dataset
app/dashboard.py - Dataset User → FastAPI Upload API: uploads dataset
app/api/main.py - Streamlit Dashboard → Analysis Orchestrator: runs analysis
app/dashboard.py - FastAPI Upload API → Analysis Orchestrator: runs analysis
app/api/main.py - FastAPI Upload API → Analysis Repository: saves run summary
app/api/main.py - Analysis Orchestrator → Dataset Loader: loads dataset
app/analyzer.py - Analysis Orchestrator → Dataset Profiler: profiles data
app/analyzer.py - Analysis Orchestrator → Quality Engine: runs checks
app/analyzer.py - Analysis Orchestrator → Quality Scoring: calculates scores
app/analyzer.py - Quality Engine → Column Classifier: classifies columns
app/quality_engine.py - Quality Engine → Quality Checks: runs checks
app/quality_engine.py - Quality Checks → Validation Rules: applies validators
app/quality_checks.py - Quality Checks → Check Result Model: creates results
app/quality_checks.py - Quality Engine → Check Result Model: summarizes results
app/quality_engine.py - Analysis Repository → Database Connector: gets connection
app/repositories/analysis_repository.py - Database Connector → PostgreSQL: connects to
app/database.py