# cmu-hgc-mac/hgc_db_postgres architecture

> Architecture diagram of the GitHub repository cmu-hgc-mac/hgc_db_postgres, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/cmu-hgc-mac/hgc_db_postgres
- Repository: https://github.com/cmu-hgc-mac/hgc_db_postgres

## Overview

##### Main Components Identification
- Database Server – This is the PostgreSQL-15 database which is the core of the system. The README and numerous configuration files (e.g., dbase_info/conn_example.yaml) indicate that accurate connection details (ports, hostnames, user credentials) are crucial.
- Database Setup Modules – The “create” directory contains scripts (create_database.py, create_tables.py) to initially create the database and configure tables.
- Database Information & Configuration – The “dbase_info” directory holds connection examples, tables definitions (YAML and CSV files) for various parts (e.g., baseplate, sensor, module, etc.), showing the schema design.
- Data Export Functionality – The “export_data” folder includes scripts and various utility modules for exporting data (especially generating XMLs and CSV files). There are clearly defined utilities for module assembly, sensor test data, and protomodule exports.
- Data Import Functionality – The “import_data” directory includes scripts to retrieve parts data and sensor IV data from external sources (including a reference to CERN parts information), implementing the data ingestion pipeline.
- Table Modification & Housekeeping – The “modify” and “housekeeping” directories include scripts for updating table hierarchies, verifying received components, and updating foreign keys. These are critical for ongoing data integrity and DB maintenance.
- MAC Public Information – The “mac_public” folder contains scripts to extract and process data (e.g., get_macs_data.py) that are likely used for public reporting or dashboard purposes.
- Control Panel Interface – The script “postgres_control_panel.py” serves as a central GUI-based tool to trigger operations like table modifications and refreshes, integrating the various database operations.

##### Relationships and Interactions
- The Control Panel (postgres_control_panel.py) is the user-facing layer that triggers the database setup, modification, and data import/export operations.
- The Setup modules (create and dbase_info) interact directly with the PostgreSQL server to create/modify the database schema.
- Import (import_data) and Export (export_data) components perform bidirectional data flows: data is imported from and exported to external sources (e.g., CERN’s API and file systems), ensuring the database stays synchronized.
- Housekeeping and Modify scripts perform periodic verification and update tasks to maintain consistency between various tables and relationships within the database.
- There is also an integration point with a browser-based tool (AdminerEvo) that is accessed via PHP installations, emphasizing external interaction for data viewing and editing.

##### Architectural Patterns & Technologies
- Modular design is evident – the project separates functionality into distinct folders (create, import_data, export_data, modify, etc.). This modularity supports maintainability and clear separation of concerns.
- Configuration-Driven Setup – YAML files and CSV templates define database connection parameters and table structures.
- While this is a backend database management system, the GUI elements (via the control panel) indicate an integrated application layer aimed at both automation and user-triggered interventions.

## Diagram (Mermaid)

```mermaid
flowchart TD
    CP["Control Panel (postgres_control_panel.py)"]:::interface
    subgraph "Database Operations"
        DS_Create["Create Scripts (create)"]:::module
        DS_Config["DB Config (dbase_info)"]:::module
        DE["Data Export Module (export_data)"]:::module
        DI["Data Import Module (import_data)"]:::module
        MM["Maintenance Module (modify/ & housekeeping)"]:::module
        MP["MAC Public Module (mac_public)"]:::module
    end
    DB["PostgreSQL-15 Database Server"]:::database
    Adminer["AdminerEvo/pgAdmin4 (PHP interface)"]:::external
    CERN["CERN API"]:::external

    CP -->|"triggers"| DS_Create
    CP -->|"triggers"| DS_Config
    CP -->|"initiates"| DE
    CP -->|"initiates"| DI
    CP -->|"initiates"| MM
    CP -->|"initiates"| MP

    DS_Create -->|"configures"| DB
    DS_Config -->|"applies_config"| DB
    DE <-->|"data_flow"| DB
    DI <-->|"data_flow"| DB
    MM -->|"modifies"| DB
    MP -->|"reports"| DB

    Adminer -->|"manages"| DB
    CERN -->|"feeds_data"| DI

    click CP "https://github.com/cmu-hgc-mac/hgc_db_postgres/blob/main/postgres_control_panel.py"
    click DS_Create "https://github.com/cmu-hgc-mac/hgc_db_postgres/blob/main/create/create_database.py"
    click DS_Create "https://github.com/cmu-hgc-mac/hgc_db_postgres/blob/main/create/create_tables.py"
    click DS_Config "https://github.com/cmu-hgc-mac/hgc_db_postgres/tree/main/dbase_info/"
    click DE "https://github.com/cmu-hgc-mac/hgc_db_postgres/tree/main/export_data/"
    click DI "https://github.com/cmu-hgc-mac/hgc_db_postgres/tree/main/import_data/"
    click MM "https://github.com/cmu-hgc-mac/hgc_db_postgres/tree/main/modify/"
    click MM "https://github.com/cmu-hgc-mac/hgc_db_postgres/tree/main/housekeeping/"
    click MP "https://github.com/cmu-hgc-mac/hgc_db_postgres/tree/main/mac_public/"
    click DB "https://github.com/cmu-hgc-mac/hgc_db_postgres/blob/main/dbase_info/conn_example.yaml"
    click DB "https://github.com/cmu-hgc-mac/hgc_db_postgres/blob/main/mac_public/macs_db_conn.yaml"

classDef interface fill:#AED6F1,stroke:#1B4F72,stroke-width:2px;
classDef module fill:#F9E79F,stroke:#B9770E,stroke-width:2px;
classDef database fill:#A3E4D7,stroke:#1D8348,stroke-width:2px;
classDef external fill:#F5CBA7,stroke:#D35400,stroke-width:2px;
```
