# metecan/wape architecture

> Architecture diagram of the GitHub repository metecan/wape, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/metecan/wape
- Repository: https://github.com/metecan/wape

## Overview

##### Main Components

A. Services Layer:
- API Service (Port 8000)
- Parser Service (Port 8001)
- Beanstalkd Queue Service
- PostgreSQL Database
- Supabase Storage (S3 Bucket)

##### Specific Component Details

- API Service       |

- File Upload Handler  |
- Queue Producer      |
- S3 Upload Helper    |

- Parser Service      |

- Queue Consumer      |
- File Parser         |
- DB Insert Helper    |

##### Flow Directions

→ Client to API: HTTP POST /page (file upload)
→ API to S3: File storage
→ API to Queue: File metadata
→ Queue to Parser: Processing tasks
→ Parser to DB: Parsed data storage

##### Important Notes

a) Environment Considerations:
- Docker container boundaries
- Port mappings (8000, 8001)
- Environment variables connection points

b) Technical Stack Callouts:
- Gofiber framework
- Beanstalkd queue
- Supabase Storage
- PostgreSQL database

##### Parsed data stored → PostgreSQL

Remember to indicate:
- Docker container boundaries
- Network connections between services
- External service dependencies
- Data flow directions
- API endpoints
- Queue channels

This system design should effectively communicate both the high-level architecture and the specific implementation details of the WAPE microservice system.

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph Client
        HTTP[HTTP Clients]:::client
    end

    subgraph "API Service Container"
        API[API Service]:::service
        Upload[File Upload Handler]:::handler
        QueueProd[Queue Producer]:::helper
        S3Helper[S3 Upload Helper]:::helper
        FileHelper[File Helper]:::helper
        Routes[Routes Config]:::config
        
        API --> Upload
        API --> QueueProd
        API --> S3Helper
        API --> FileHelper
        API --> Routes
    end

    subgraph "Parser Service Container"
        Parser[Parser Service]:::service
        QueueCons[Queue Consumer]:::helper
        FileParser[File Parser]:::helper
        DBHelper[DB Insert Helper]:::helper
        ParserHandler[Parser Handler]:::handler
        DBConn[Database Connection]:::config
        
        Parser --> QueueCons
        Parser --> FileParser
        Parser --> DBHelper
        Parser --> ParserHandler
        Parser --> DBConn
    end

    subgraph "Infrastructure"
        Queue[(Beanstalkd Queue)]:::queue
        S3[(Supabase Storage)]:::storage
        DB[(PostgreSQL)]:::database
    end

    HTTP --> |HTTP POST /page|API
    API --> |Store Files|S3
    API --> |Queue Tasks|Queue
    Queue --> |Consume Tasks|Parser
    Parser --> |Store Data|DB
    Parser --> |Read Files|S3

    %% Click Events
    click API "https://github.com/metecan/wape/tree/master/api"
    click Upload "https://github.com/metecan/wape/blob/master/api/handler/page.go"
    click QueueProd "https://github.com/metecan/wape/blob/master/api/helper/queue.go"
    click S3Helper "https://github.com/metecan/wape/blob/master/api/helper/bucket.go"
    click FileHelper "https://github.com/metecan/wape/blob/master/api/helper/file.go"
    click Routes "https://github.com/metecan/wape/blob/master/api/routes/setup.go"
    click Parser "https://github.com/metecan/wape/tree/master/parser"
    click QueueCons "https://github.com/metecan/wape/blob/master/parser/helper/queue.go"
    click FileParser "https://github.com/metecan/wape/blob/master/parser/helper/parser.go"
    click DBHelper "https://github.com/metecan/wape/blob/master/parser/helper/insert.go"
    click ParserHandler "https://github.com/metecan/wape/blob/master/parser/handler/parser.go"
    click DBConn "https://github.com/metecan/wape/blob/master/parser/db/connect.go"

    %% Styling
    classDef client fill:#4B96FF,stroke:#333,stroke-width:2px
    classDef service fill:#90EE90,stroke:#333,stroke-width:2px
    classDef handler fill:#FFB6C1,stroke:#333,stroke-width:2px
    classDef helper fill:#FFE4B5,stroke:#333,stroke-width:2px
    classDef config fill:#E6E6FA,stroke:#333,stroke-width:2px
    classDef queue fill:#FFA500,stroke:#333,stroke-width:2px
    classDef storage fill:#DDA0DD,stroke:#333,stroke-width:2px
    classDef database fill:#20B2AA,stroke:#333,stroke-width:2px

    %% Legend
    subgraph Legend
        L1[Client]:::client
        L2[Service]:::service
        L3[Handler]:::handler
        L4[Helper]:::helper
        L5[Config]:::config
        L6[Queue]:::queue
        L7[Storage]:::storage
        L8[Database]:::database
    end
```
