# pt-indonesia-comnets-plus/database-management-web architecture

> Architecture diagram of the GitHub repository pt-indonesia-comnets-plus/database-management-web, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pt-indonesia-comnets-plus/database-management-web
- Repository: https://github.com/pt-indonesia-comnets-plus/database-management-web

## Overview

##### Project type and purpose
- This is a containerized, data‐driven Streamlit web application with two principal user interfaces (“Home” and “Admin”) plus an ETL pipeline and AI‐powered services (RAG, agent graph, Airflow triggers, etc.).
- It supports user management, asset data ingestion, dashboard views, chat/search features, and email notifications.

##### File structure
- Top level:
- Dockerfile/.dockerignore – container definition
- Main_Page.py & pages/ – Streamlit entry points
- etl_proces.py & ETL_proces.ipynb – batch ETL jobs ingesting data.xlsx, PDF → database
- core/ – business-logic package
· configs/ (YAML data & tools configurations)
· models/ (ORM-style classes: User, Asset)
· services/ (AssetDataService, UserService, EmailService, RAG, plus an agent_graph subpackage with tool connectors)
· utils/ (DB connection, Firebase, cookies, CSS loader)
- features/ – MVC-style controllers & views for admin and home flows
- static/ – CSS/image assets
- data/ – raw data files
- init.sql & schema.rb – database schema
- Patterns: layered MVC (features/controllers → services → models → utils/database), plus a separate ETL module.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Users
    USER["User (Browser)"]:::user

    subgraph "Docker Container"
      subgraph "Streamlit Server"
        subgraph "UI Layer" 
          MAIN["Main_Page.py"]:::ui
          HOME["Home Page (pages/1 Home Page.py)"]:::ui
          ADMIN["Admin Page (pages/2 Admin Page.py)"]:::ui
          STATIC["static/css & images"]:::ui
        end
        subgraph "Controllers"
          HC["features/home/controller.py"]:::controller
          AC["features/admin/controller.py"]:::controller
        end
        subgraph "Services"
          ADS["AssetDataService"]:::service
          US["UserService"]:::service
          UDS["UserDataService"]:::service
          ES["EmailService"]:::service
          RAG["RAG"]:::service
          subgraph "agent_graph"
            AG_B["build_graph"]:::service
            AG_BE["agent_backend"]:::service
            AG_AFT["tool_airflow_trigger"]:::service
            AG_RAG["tool_rag"]:::service
            AG_SQL["tool_sql_agent"]:::service
            AG_TAV["tool_tavily_search"]:::service
            AG_VIS["tool_visualization"]:::service
          end
        end
        subgraph "Models Layer"
          M_ALL["models.py"]:::model
          M_USER["user_model.py"]:::model
        end
        subgraph "Utils Layer"
          U_DB["database.py"]:::util
          U_FB["firebase_config.py"]:::util
          U_LC["load_config & load_data_configs"]:::util
          U_CSS["load_css"]:::util
          U_COOK["cookies"]:::util
        end
        subgraph "Configs"
          CFG_DATA["data_configs.yml"]:::config
          CFG_TOOLS["tools_configs.yml"]:::config
          CFG_TOML["config.toml"]:::config
          CFG_SEC["secret_example.toml"]:::config
        end
      end
    end

    %% ETL Pipeline
    subgraph "ETL Pipeline"
      ETL_PY["etl_proces.py"]:::etl
      ETL_NB["ETL_proces.ipynb"]:::etl
      RAW_XLS["data.xlsx"]:::data
      RAW_PDF["data knowledge data aset.pdf"]:::data
    end

    %% Data Store
    subgraph "Data Store"
      DB["Relational DB"]:::db
      INIT["init.sql"]:::files
      SCHEMA["schema.rb"]:::files
    end

    %% External Services
    SMTP["Email Provider"]:::external
    AIAPI["AI API (OpenAI)"]:::external
    AIRFLOW["Airflow Orchestrator"]:::external
    FIREBASE_AUTH["Firebase Auth"]:::external

    %% User Flow
    USER -->|"HTTP"| MAIN
    MAIN -->|routes| HOME
    MAIN -->|routes| ADMIN
    HOME -->|calls| HC
    ADMIN -->|calls| AC
    HC -->|uses| ADS
    HC -->|uses| UDS
    HC -->|uses| RAG
    AC -->|uses| RAG
    AC -->|uses| ES

    %% Service to Model & Utils & External
    ADS -->|reads/writes| M_ALL
    US -->|reads/writes| M_USER
    UDS -->|reads/writes| M_ALL
    ES -->|sends| SMTP
    RAG -->|calls| AIAPI
    RAG -->|stores vectors| DB
    AG_AFT -->|triggers| AIRFLOW
    AG_SQL -->|queries| DB

    %% Model to Utils and DB
    M_ALL -->|SQL| U_DB
    M_USER -->|SQL| U_DB
    U_DB -->|connects| DB

    %% Configs loaded
    CFG_DATA -->|loaded by| U_LC
    CFG_TOOLS -->|loaded by| U_LC
    CFG_TOML -->|streamlit| MAIN
    CFG_SEC -->|streamlit| MAIN

    %% ETL Flow
    RAW_XLS -->|input| ETL_PY
    RAW_PDF -->|input| ETL_PY
    ETL_PY -->|batch load| DB
    ETL_NB -->|prototype| ETL_PY

    %% DB schema files
    INIT -->|defines| DB
    SCHEMA -->|defines| DB

    %% Styles
    classDef user fill:#B3E5FC,stroke:#0298B0,color:#000
    classDef ui fill:#BBDEFB,stroke:#1976D2,color:#000
    classDef controller fill:#C8E6C9,stroke:#388E3C,color:#000
    classDef service fill:#A5D6A7,stroke:#2E7D32,color:#000
    classDef model fill:#FFE0B2,stroke:#EF6C00,color:#000
    classDef util fill:#DCEDC8,stroke:#558B2F,color:#000
    classDef config fill:#E1BEE7,stroke:#8E24AA,color:#000
    classDef etl fill:#FFCCBC,stroke:#E64A19,color:#000
    classDef data fill:#FFECB3,stroke:#FFA000,color:#000
    classDef db fill:#FFE082,stroke:#FF8F00,color:#000
    classDef files fill:#FFF9C4,stroke:#FBC02D,color:#000
    classDef external fill:#E1F5FE,stroke:#0288D1,color:#000

    %% Click Events
    click MAIN "https://github.com/pt-indonesia-comnets-plus/database-management-web/blob/main/Main_Page.py"
    click HOME "https://github.com/pt-indonesia-comnets-plus/database-management-web/blob/main/pages/1 Home Page.py"
    click ADMIN "https://github.com/pt-indonesia-comnets-plus/database-management-web/blob/main/pages/2 Admin Page.py"
    click ETL_PY "https://github.com/pt-indonesia-comnets-plus/database-management-web/blob/main/etl_proces.py"
    click ETL_NB "https://github.com/pt-indonesia-comnets-plus/database-management-web/blob/main/ETL_proces.ipynb"
    click CFG_DATA "https://github.com/pt-indonesia-comnets-plus/database-management-web/blob/main/core/configs/data_configs.yml"
    click CFG_TOOLS "https://github.com/pt-indonesia-comnets-plus/database-management-web/blob/main/core/configs/tools_configs.yml"
    click CFG_TOML "https://github.com/pt-indonesia-comnets-plus/database-management-web/blob/main/.streamlit/config.toml"
    click CFG_SEC "https://github.com/pt-indonesia-comnets-plus/database-management-web/blob/main/.streamlit/secret_example.toml"
    click INIT "https://github.com/pt-indonesia-comnets-plus/database-management-web/blob/main/init.sql"
    click SCHEMA "https://github.com/pt-indonesia-comnets-plus/database-management-web/blob/main/schema.rb"
    click RAW_XLS "https://github.com/pt-indonesia-comnets-plus/database-management-web/blob/main/data/data.xlsx"
    click RAW_PDF "https://github.com/pt-indonesia-comnets-plus/database-management-web/blob/main/data/data knowledge data aset.pdf"
    click M_ALL "https://github.com/pt-indonesia-comnets-plus/database-management-web/blob/main/core/models/models.py"
    click M_USER "https://github.com/pt-indonesia-comnets-plus/database-management-web/blob/main/core/models/user_model.py"
    click ADS "https://github.com/pt-indonesia-comnets-plus/database-management-web/blob/main/core/services/AssetDataService.py"
    click US "https://github.com/pt-indonesia-comnets-plus/database-management-web/blob/main/core/services/UserService.py"
    click UDS "https://github.com/pt-indonesia-comnets-plus/database-management-web/blob/main/core/services/UserDataService.py"
    click ES "https://github.com/pt-indonesia-comnets-plus/database-management-web/blob/main/core/services/EmailService.py"
    click RAG "https://github.com/pt-indonesia-comnets-plus/database-management-web/blob/main/core/services/RAG.py"
    click AG_BE "https://github.com/pt-indonesia-comnets-plus/database-management-web/blob/main/core/services/agent_graph/agent_backend.py"
    click AG_B "https://github.com/pt-indonesia-comnets-plus/database-management-web/blob/main/core/services/agent_graph/build_graph.py"
    click AG_AFT "https://github.com/pt-indonesia-comnets-plus/database-management-web/blob/main/core/services/agent_graph/tool_airflow_trigger.py"
    click AG_RAG "https://github.com/pt-indonesia-comnets-plus/database-management-web/blob/main/core/services/agent_graph/tool_rag.py"
    click AG_SQL "https://github.com/pt-indonesia-comnets-plus/database-management-web/blob/main/core/services/agent_graph/tool_sql_agent.py"
    click AG_TAV "https://github.com/pt-indonesia-comnets-plus/database-management-web/blob/main/core/services/agent_graph/tool_tavily_search.py"
    click AG_VIS "https://github.com/pt-indonesia-comnets-plus/database-management-web/blob/main/core/services/agent_graph/tool_visualization.py"
    click U_DB "https://github.com/pt-indonesia-comnets-plus/database-management-web/blob/main/core/utils/database.py"
    click U_FB "https://github.com/pt-indonesia-comnets-plus/database-management-web/blob/main/core/utils/firebase_config.py"
    click U_LC "https://github.com/pt-indonesia-comnets-plus/database-management-web/blob/main/core/utils/load_config.py"
    click U_LC "https://github.com/pt-indonesia-comnets-plus/database-management-web/blob/main/core/utils/load_data_configs.py"
    click U_CSS "https://github.com/pt-indonesia-comnets-plus/database-management-web/blob/main/core/utils/load_css.py"
    click U_COOK "https://github.com/pt-indonesia-comnets-plus/database-management-web/blob/main/core/utils/cookies.py"
    click HC "https://github.com/pt-indonesia-comnets-plus/database-management-web/blob/main/features/home/controller.py"
    click AC "https://github.com/pt-indonesia-comnets-plus/database-management-web/blob/main/features/admin/controller.py"
```
