# nilavra/mturk-crowdsourcing-pipeline architecture

> Architecture diagram of the GitHub repository nilavra/mturk-crowdsourcing-pipeline, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nilavra/mturk-crowdsourcing-pipeline
- Repository: https://github.com/nilavra/mturk-crowdsourcing-pipeline

## Overview

##### Main Components

a) Client-Side Components (Frontend):
- Image Viewer Component (using iviewer)
- Annotation Interface
- jQuery-based UI Components

b) Server-Side Components:
- External HIT Handler
- Configuration Management

c) Database Layer:
- A database symbol labeled "Database System" (MySQL/SQLite)
- Connection to image list storage (referenced in sql_script)

d) MTurk Integration Layer:
- HIT Management
- Worker Management
- Results Processing

e) Processing Scripts:
- A container for "Processing Scripts" showing:
- Python Parser
- Shell Script Automation

##### Important Details

a) External Services:
- Amazon Mechanical Turk (as an external cloud service)
- Web Server (hosting environment)

b) File System Components:
- Image Storage
- Results Storage
- Configuration Files

##### Architectural Notes

- Notes about the technology stack:
- Frontend: jQuery, jQuery UI, Custom JS
- Backend: PHP
- Database: MySQL/SQLite
- Integration: MTurk CLI Tools
- Processing: Python, Shell Scripts

##### Additional Elements

- A security boundary showing public/private components
- Configuration management flow
- Worker interaction flow
- Data persistence paths

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph Web_Interface
        IV[Image Viewer]:::frontend
        AI[Annotation Interface]:::frontend
        JQ[jQuery UI Components]:::frontend
    end

    subgraph PHP_Backend
        EH[External HIT Handler]:::backend
        CM[Configuration Management]:::backend
    end

    subgraph Database_System
        DB[(Database)]:::database
        IMG_STORE[(Image List Storage)]:::database
    end

    subgraph MTurk_Management
        HM[HIT Management]:::mturk
        WM[Worker Management]:::mturk
        RP[Results Processing]:::mturk
    end

    subgraph Processing_Scripts
        PP[Python Parser]:::processing
        SA[Shell Automation]:::processing
    end

    subgraph External_Services
        AMTURK[Amazon MTurk]:::external
    end

    %% Connections
    Web_Interface <--> PHP_Backend
    PHP_Backend <--> Database_System
    MTurk_Management <--> AMTURK
    MTurk_Management --> Processing_Scripts
    Processing_Scripts --> Database_System

    %% Internal connections
    IV <--> AI
    AI <--> JQ
    EH <--> CM
    DB <--> IMG_STORE
    HM <--> WM
    WM <--> RP
    PP <--> SA

    %% Click events
    click IV "https://github.com/nilavra/mturk-crowdsourcing-pipeline/blob/master/src/php_html_css/js/iviewer/jquery.iviewer.js"
    click AI "https://github.com/nilavra/mturk-crowdsourcing-pipeline/blob/master/src/php_html_css/external_hit.php"
    click JQ "https://github.com/nilavra/mturk-crowdsourcing-pipeline/tree/master/src/php_html_css/js/jquery-ui-1.12.1/"
    click EH "https://github.com/nilavra/mturk-crowdsourcing-pipeline/blob/master/src/php_html_css/external_hit.php"
    click CM "https://github.com/nilavra/mturk-crowdsourcing-pipeline/blob/master/src/php_html_css/config.ini.php"
    click HM "https://github.com/nilavra/mturk-crowdsourcing-pipeline/tree/master/src/mturk_files/"
    click WM "https://github.com/nilavra/mturk-crowdsourcing-pipeline/blob/master/src/shell_scripts/block_worker.cmd"
    click RP "https://github.com/nilavra/mturk-crowdsourcing-pipeline/blob/master/src/shell_scripts/getResults.cmd"
    click PP "https://github.com/nilavra/mturk-crowdsourcing-pipeline/blob/master/src/python_scripts/parser_result_mturk.py"
    click SA "https://github.com/nilavra/mturk-crowdsourcing-pipeline/blob/master/src/shell_scripts/run.cmd"
    click IMG_STORE "https://github.com/nilavra/mturk-crowdsourcing-pipeline/blob/master/src/sql_script/pd_popl_img_list.sql"

    %% Styling
    classDef frontend fill:#2374ab
    classDef backend fill:#2a9d8f
    classDef database fill:#e76f51
    classDef mturk fill:#7209b7
    classDef processing fill:#fca311
    classDef external fill:#grey

    %% Legend
    subgraph Legend
        F[Frontend]:::frontend
        B[Backend]:::backend
        D[Database]:::database
        M[MTurk]:::mturk
        P[Processing]:::processing
        E[External]:::external
    end
```
