# frozsgy/personality-analysis-framework architecture

> Architecture diagram of the GitHub repository frozsgy/personality-analysis-framework, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/frozsgy/personality-analysis-framework
- Repository: https://github.com/frozsgy/personality-analysis-framework

## Overview

##### Project type and purpose
- This is a full-stack web framework (“Personality Analysis Framework”) that ingests Turkish tweets, processes them via NLP and vector models, then predicts Big-Five personality scores.
- It exposes:
- A React-based frontend (in /frontend)
- A Flask REST backend (main.py, router.py, web/service.py)
- Two auxiliary microservices: Zemberek (gRPC) and Word2Vec
- MySQL/MariaDB for persistence
- A CLI wrapper (main.py) and training scripts

##### File structure
- Top-level directories:
- frontend/ – React UI code
- twitter/ – tweet retrieval abstractions
- vector/ – boundary generation & Word2Vec wrappers
- predictors/ – TPOT-based model training & inference
- utils/ – preprocessing, Big-Five calculators, stopword lists
- web/ – Flask DB connectors (web/db.py), service endpoints (web/service.py), plotting
- zemberek/ – protobufs and JAR for gRPC NLP service
- data/ – tweet storage
- Scripts to stand up services: run_zemberek.sh, run_word2vec.sh, train_model.sh
- Config files: requirements.txt, setup.py, tsconfig.json, package.json

##### README for additional insights
- Four distinct runtime services:
1. Zemberek gRPC server (Java NLP)
2. Word2Vec vector service (Python)
3. Flask REST API (Python backend)
4. React frontend (Node/React)
- Optional CLI mode via main.py
- Persistent storage via MySQL/MariaDB

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Actors
    Browser("User Browser"):::external
    CLI("CLI"):::external

    %% Frontend Layer
    subgraph "Frontend Layer"
        ReactApp("React App"):::frontend
    end

    %% Backend Layer
    subgraph "Backend Layer"
        subgraph "Flask REST API"
            FLMain("main.py"):::backend
            Router("router.py"):::backend
            Service("Service Endpoints"):::backend
            DBLayer("DB Layer"):::backend
            Plot("Plotting"):::backend
            ID("Identifier"):::backend
        end
        subgraph "Backend Modules"
            subgraph "Twitter Module"
                DL("download.py"):::module
                TweetAb("tweet.py"):::module
            end
            subgraph "Preprocessing Utils"
                Preproc("preprocess.py"):::module
                Emoji("emoji.py"):::module
                Turkish("turkish.py"):::module
                Stopwords("stopwords.py"):::module
                Ranks("ranks-nl-stopwords"):::module
                Discret("discretization.py"):::module
            end
            subgraph "Big-Five Calculation"
                OCEAN("ocean_calculator.py"):::module
                Selenium("selenium_calculator.py"):::module
            end
            subgraph "Vectorization Logic"
                W2VWrapper("word2vec.py"):::module
                GenSet("generate_set.py"):::module
                GetBdry("get_boundary.py"):::module
                CoreVec("vector.py"):::module
                TrainVec("train.py"):::module
            end
            subgraph "Predictive Modeling"
                Predict("predict.py"):::module
                GenVec("generate_vectors.py"):::module
                Teapot("teapot.py"):::module
            end
        end
    end

    %% Microservices
    subgraph "Microservices"
        Zemberek("Zemberek NLP gRPC"):::micro
        Word2VecServ("Word2Vec Service"):::micro
    end

    %% Data Stores
    subgraph "Data Stores"
        MySQL("MySQL/MariaDB"):::db
        RawTweets("Raw Tweets (CSV)"):::storage
    end

    %% External Integration
    subgraph "External Systems"
        TwitterAPI("Twitter API"):::external
    end

    %% Scripts
    subgraph "Scripts"
        RunZemb("run_zemberek.sh"):::script
        RunW2V("run_word2vec.sh"):::script
        TrainScr("train_model.sh"):::script
    end

    %% Connections
    Browser -->|HTTPS| ReactApp
    CLI -->|Invoke| FLMain
    ReactApp -->|HTTP| Router
    Router -->|routes to| Service
    Service -->|uses| DBLayer
    Service -->|uses| Plot
    Service -->|uses| ID
    Service -->|imports| DL
    Service -->|imports| TweetAb
    Service -->|calls gRPC| Zemberek
    Service -->|calls HTTP/gRPC| Word2VecServ
    Service -->|imports| Preproc
    Service -->|imports| Emoji
    Service -->|imports| Turkish
    Service -->|imports| Stopwords
    Service -->|imports| Ranks
    Service -->|imports| Discret
    Service -->|imports| OCEAN
    Service -->|imports| Selenium
    Service -->|imports| W2VWrapper
    Service -->|imports| GenSet
    Service -->|imports| GetBdry
    Service -->|imports| CoreVec
    Service -->|imports| TrainVec
    Service -->|imports| Predict
    Service -->|imports| GenVec
    Service -->|imports| Teapot
    DL -->|HTTP| TwitterAPI
    Predict -->|read/write| MySQL
    Predict -->|read| RawTweets

    RunZemb -->|launch| Zemberek
    RunW2V -->|launch| Word2VecServ
    TrainScr -->|train| TrainVec
    TrainScr -->|train| GenVec
    TrainScr -->|train| Teapot

    %% Click Events
    click ReactApp "https://github.com/frozsgy/personality-analysis-framework/tree/master/frontend/"
    click FLMain "https://github.com/frozsgy/personality-analysis-framework/blob/master/main.py"
    click Router "https://github.com/frozsgy/personality-analysis-framework/blob/master/router.py"
    click Service "https://github.com/frozsgy/personality-analysis-framework/blob/master/web/service.py"
    click DBLayer "https://github.com/frozsgy/personality-analysis-framework/blob/master/web/db.py"
    click Plot "https://github.com/frozsgy/personality-analysis-framework/blob/master/web/plot.py"
    click ID "https://github.com/frozsgy/personality-analysis-framework/blob/master/web/id.py"
    click DL "https://github.com/frozsgy/personality-analysis-framework/blob/master/twitter/download.py"
    click TweetAb "https://github.com/frozsgy/personality-analysis-framework/blob/master/twitter/tweet.py"
    click Preproc "https://github.com/frozsgy/personality-analysis-framework/blob/master/utils/preprocess.py"
    click Emoji "https://github.com/frozsgy/personality-analysis-framework/blob/master/utils/emoji.py"
    click Turkish "https://github.com/frozsgy/personality-analysis-framework/blob/master/utils/turkish.py"
    click Stopwords "https://github.com/frozsgy/personality-analysis-framework/blob/master/utils/stopwords.py"
    click Ranks "https://github.com/frozsgy/personality-analysis-framework/tree/master/utils/ranks-nl-stopwords/"
    click Discret "https://github.com/frozsgy/personality-analysis-framework/blob/master/utils/discretization.py"
    click OCEAN "https://github.com/frozsgy/personality-analysis-framework/blob/master/utils/bigfive/ocean_calculator.py"
    click Selenium "https://github.com/frozsgy/personality-analysis-framework/blob/master/utils/bigfive/selenium_calculator.py"
    click W2VWrapper "https://github.com/frozsgy/personality-analysis-framework/blob/master/vector/word2vec.py"
    click GenSet "https://github.com/frozsgy/personality-analysis-framework/blob/master/vector/boundary/generate_set.py"
    click GetBdry "https://github.com/frozsgy/personality-analysis-framework/blob/master/vector/boundary/get_boundary.py"
    click CoreVec "https://github.com/frozsgy/personality-analysis-framework/blob/master/vector/vector.py"
    click TrainVec "https://github.com/frozsgy/personality-analysis-framework/blob/master/vector/train.py"
    click Predict "https://github.com/frozsgy/personality-analysis-framework/blob/master/predictors/predict.py"
    click GenVec "https://github.com/frozsgy/personality-analysis-framework/blob/master/predictors/tpot/generate_vectors.py"
    click Teapot "https://github.com/frozsgy/personality-analysis-framework/blob/master/predictors/tpot/teapot.py"
    click RunZemb "https://github.com/frozsgy/personality-analysis-framework/blob/master/run_zemberek.sh"
    click RunW2V "https://github.com/frozsgy/personality-analysis-framework/blob/master/run_word2vec.sh"
    click TrainScr "https://github.com/frozsgy/personality-analysis-framework/blob/master/train_model.sh"

    %% Styles
    classDef frontend fill:#D0E6FF,stroke:#0366d6,color:#0366d6
    classDef backend fill:#E8F5E9,stroke:#2E7D32,color:#2E7D32
    classDef module fill:#FFF3E0,stroke:#FB8C00,color:#E65100
    classDef micro fill:#F3E5F5,stroke:#8E24AA,color:#6A1B9A
    classDef db fill:#E8F5E9,stroke:#43A047,color:#2E7D32
    classDef storage fill:#E0F7FA,stroke:#00ACC1,color:#006064
    classDef external fill:#ECEFF1,stroke:#607D8B,color:#37474F,stroke-dasharray: 5 5
    classDef script fill:none,stroke:#757575,stroke-dasharray: 2 2,color:#37474F
```
