# satishkumarkolamudi/snowflake architecture

> Architecture diagram of the GitHub repository satishkumarkolamudi/snowflake, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/satishkumarkolamudi/snowflake
- Repository: https://github.com/satishkumarkolamudi/snowflake

## Overview

##### Project Type and Purpose
- KalaYatra is a full-stack, data-intensive web application.
- Frontend: Streamlit-based interactive UI
- Backend/Data Layer: Snowflake Data Cloud for storage, transformation, and geospatial processing
- Data Sources: Public datasets (e.g., data.gov.in) ingested via CSV stages

##### File Structure
Top-level files and directories:
- `.sql` scripts: define raw tables (`TOURISM_STATS_RAW.sql`, `CULTURAL_SITES.sql`, etc.) and curated views (`VW_UNTOUCHED_GEMS.sql`, `VW_TOURISM_SEASONALITY.sql`) in Snowflake
- `kalayatra_app.py`: Streamlit app entry point
- `popularity_score.py`: Python module for computing custom metrics
- `requirements.txt`: Python dependencies (Snowflake connector, Snowpark, Streamlit)
- `secrets.toml`: Snowflake credentials (in `.streamlit/`)
Architectural hints:
- SQL files imply an ETL/ELT pipeline within Snowflake
- Python scripts connect to Snowflake, fetch data, and feed the UI

##### Extract Key Insights from README
- Data ingestion: CSVs → Snowflake stage → raw tables
- Transformation: SQL scripts → curated tables and views in `CURATED_DATA` schema
- UI: Streamlit queries curated views, renders maps, charts, filters
- Responsible-tourism tips and popularity scores are additional overlays

##### Main Components
a. External Data Sources
– data.gov.in CSV exports
b. Snowflake Data Warehouse
– Stage (`@KALAYATRA_DB.RAW_DATA.KALAYATRA_STAGE`)
– Schemas: `RAW_DATA`, `CURATED_DATA`
– Raw tables (`TOURISM_STATS_RAW`, `ART_FORMS`, …)
– Curated views (`VW_UNTOUCHED_GEMS`, `VW_TOURISM_SEASONALITY`, …)
c. Python Processing Layer
– `popularity_score.py` (Snowpark or connector-based enrichment)
d. Streamlit Frontend
– `kalayatra_app.py`
– Renders interactive map, filters, charts, tips
e. User’s Browser

##### Relationships and Data Flow
4. Python Processing (`popularity_score.py`) reading Curated Views and writing back metrics or passing to UI (bi-directional)

##### Architectural Patterns and Technologies
- ELT pattern: load raw CSVs → transform in-database with SQL
- Snowflake: Data storage, compute (Snowpark), geospatial capabilities
- Streamlit: declarative Python UI framework for data apps
- Modular Python code: separate metric computations (`popularity_score.py`) from UI logic
- Configuration via `secrets.toml` (Snowflake credentials)

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Data Ingestion & Storage"
        direction TB
        CSV["External CSVs (data.gov.in CSVs)"]:::ingestion
        Stage["Snowflake Stage"]:::ingestion
        subgraph "RAW_DATA Schema (Raw Tables)"
            direction TB
            RT1["TOURISM_STATS_RAW"]:::ingestion
            RT2["CULTURAL_SITES"]:::ingestion
            RT3["ART_FORMS"]:::ingestion
            RT4["RESPONSIBLE_TOURISM_TIPS"]:::ingestion
            RT5["TOURISM_TRENDS"]:::ingestion
        end
        subgraph "CURATED_DATA Schema (Curated Views)"
            direction TB
            CV1["VW_UNTOUCHED_GEMS"]:::ingestion
            CV2["VW_TOURISM_SEASONALITY"]:::ingestion
        end
    end

    subgraph "Processing Layer"
        direction TB
        PP["Popularity Score Module"]:::processing
    end

    subgraph "Presentation Layer"
        direction TB
        UI["Streamlit App"]:::presentation
        Req["requirements.txt"]:::presentation
        Sec["secrets.toml"]:::presentation
        UB["User Browser"]:::presentation
    end

    CSV -->|"Upload via COPY"| Stage
    Stage -->|"SQL COPY"| RT1
    Stage -->|"SQL COPY"| RT2
    Stage -->|"SQL COPY"| RT3
    Stage -->|"SQL COPY"| RT4
    Stage -->|"SQL COPY"| RT5

    RT1 -->|"Transform via SQL"| CV1
    RT2 -->|"Transform via SQL"| CV1
    RT3 -->|"Transform via SQL"| CV1
    RT4 -->|"Transform via SQL"| CV1
    RT5 -->|"Transform via SQL"| CV2

    CV1 -->|"Snowflake Connector Query"| PP
    CV2 -->|"Snowflake Connector Query"| PP

    PP -->|"Enriched Metrics/DataFrames"| UI
    UI -->|"Query Snowflake"| CV1
    UI -->|"Query Snowflake"| CV2
    UI -->|"Renders Maps & Charts"| UB

    click Stage "https://github.com/satishkumarkolamudi/snowflake/blob/main/KALAYATRA_DB.sql"
    click RT1 "https://github.com/satishkumarkolamudi/snowflake/blob/main/TOURISM_STATS_RAW.sql"
    click RT2 "https://github.com/satishkumarkolamudi/snowflake/blob/main/CULTURAL_SITES.sql"
    click RT3 "https://github.com/satishkumarkolamudi/snowflake/blob/main/ART_FORMS.sql"
    click RT4 "https://github.com/satishkumarkolamudi/snowflake/blob/main/RESPONSIBLE_TOURISM_TIPS.sql"
    click RT5 "https://github.com/satishkumarkolamudi/snowflake/blob/main/TOURISM_TRENDS.sql"
    click CV1 "https://github.com/satishkumarkolamudi/snowflake/blob/main/VW_UNTOUCHED_GEMS.sql"
    click CV2 "https://github.com/satishkumarkolamudi/snowflake/blob/main/VW_TOURISM_SEASONALITY.sql"
    click PP "https://github.com/satishkumarkolamudi/snowflake/blob/main/popularity_score.py"
    click UI "https://github.com/satishkumarkolamudi/snowflake/blob/main/kalayatra_app.py"
    click Req "https://github.com/satishkumarkolamudi/snowflake/blob/main/requirements.txt"
    click Sec "https://github.com/satishkumarkolamudi/snowflake/blob/main/secrets.toml"

    classDef ingestion fill:#E8F1F2,stroke:#107080,stroke-width:2px
    classDef processing fill:#F2EBE9,stroke:#A3423F,stroke-width:2px
    classDef presentation fill:#F3F4E7,stroke:#8C841C,stroke-width:2px
```
