# luchoss10/data_eng_project architecture

> Architecture diagram of the GitHub repository luchoss10/data_eng_project, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-02-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/luchoss10/data_eng_project
- Repository: https://github.com/luchoss10/data_eng_project

## Overview

##### Main Components

a) Data Source Layer:
- 'upload_files' directory containing the three CSV files:
- departments.csv
- hired_employees.csv
- jobs.csv

b) API Service Layer:
- Main.py as the central API service component
- Load_local_data.py as a data loading utility

c) Testing Component:
- Test_main.py as the testing framework

##### Top Section (Data Sources)
- Three rectangles representing each CSV file

##### Middle Section (Processing)
- Load_local_data.py as a supporting component

##### Bottom Section (Storage/Output)
- Database or storage representation
- Data flow from processing to storage

##### Additional Elements
- Requirements.txt as a configuration note
- Test_main.py with coverage indicators

##### Key Relationships to Highlight

→ CSV Files → load_local_data.py → main.py API → Database

With test_main.py showing coverage across components

##### Important Notes

- Keep the API service (main.py) as the central focus
- Data flow from CSV files through the system
- Where data validation and processing occur
- Test coverage scope

##### Technical Details

- Python environment indicators
- API endpoints (if any)
- Data transformation points
- File processing sequence

- All symbols used
- Component relationships

## Diagram (Mermaid)

```mermaid
graph TB
    %% Style Definitions
    classDef source fill:#b3d1ff,stroke:#333
    classDef api fill:#98FB98,stroke:#333
    classDef test fill:#FFE4B5,stroke:#333
    classDef config fill:#DCDCDC,stroke:#333
    classDef storage fill:#FFB6C1,stroke:#333

    %% Legend
    subgraph Legend
        S1[Data Source]:::source
        A1[API Component]:::api
        T1[Test Component]:::test
        C1[Configuration]:::config
        DB1[(Storage)]:::storage
    end

    %% Data Source Layer
    subgraph DataSources["Data Source Layer"]
        CSV1[departments.csv]:::source
        CSV2[hired_employees.csv]:::source
        CSV3[jobs.csv]:::source
    end

    %% Processing Layer
    subgraph ProcessingLayer["Processing Layer"]
        LoadData[load_local_data.py]:::api
        MainAPI[main.py]:::api
        Config[requirements.txt]:::config
    end

    %% Testing Layer
    subgraph TestLayer["Testing Layer"]
        TestMain[test_main.py]:::test
    end

    %% Storage Layer
    subgraph StorageLayer["Storage Layer"]
        DB[(Database)]:::storage
    end

    %% Relationships
    CSV1 --> LoadData
    CSV2 --> LoadData
    CSV3 --> LoadData
    LoadData --> MainAPI
    MainAPI --> DB
    TestMain -.-> MainAPI
    TestMain -.-> LoadData
    Config -.-> MainAPI

    %% Click Events
    click CSV1 "https://github.com/luchoss10/data_eng_project/blob/main/upload_files/departments.csv"
    click CSV2 "https://github.com/luchoss10/data_eng_project/blob/main/upload_files/hired_employees.csv"
    click CSV3 "https://github.com/luchoss10/data_eng_project/blob/main/upload_files/jobs.csv"
    click LoadData "https://github.com/luchoss10/data_eng_project/blob/main/load_local_data.py"
    click MainAPI "https://github.com/luchoss10/data_eng_project/blob/main/main.py"
    click TestMain "https://github.com/luchoss10/data_eng_project/blob/main/test_main.py"
    click Config "https://github.com/luchoss10/data_eng_project/blob/main/requirements.txt"
```
