# shoumiksaha/ai-polished-text architecture

> Architecture diagram of the GitHub repository shoumiksaha/ai-polished-text, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/shoumiksaha/ai-polished-text
- Repository: https://github.com/shoumiksaha/ai-polished-text

## Overview

##### Project type and purpose
- APT-Eval is an open-source research tool / CLI pipeline for evaluating AI “polishers” and detectors on a human-written text (HWT) dataset.
- It ingests raw and AI-polished text samples, runs a suite of detection models/metrics, computes evaluation metrics, and outputs structured results for analysis.

##### File structure
Top-level directories & files:
- data/ – raw and polished CSV/JSON datasets and preprocessing scripts (cal_distance.py, json_to_csv.py, polish scripts).
- detectors/ – detector “plugins” subdivided by model name (radar, detectgpt, gltr, binoculars, commercial wrappers like ZeroGPT).
- raid/ – core helper code inherited from RAID (detect.py, evaluate.py, utils).
- results/ – stored predictions, accuracy/FPR results, domain- and polish-type breakdowns, and plotting scripts.
- CLI scripts – detect_cli.py, evaluate_cli.py, evaluate_for_hybrid.py.
- run_*.sh & set_api_keys.sh – orchestration and environment setup.
- environment.yml – Conda environment specification.

##### README
- Describes dataset (15K samples), directory layouts (`data/polished`, `data/polished_json`).
- Usage: CLI workflows (`detect_cli.py`, `evaluate_cli.py`, `evaluate_for_hybrid.py`, plus helper shell scripts).
- Lists included detectors and how to add new ones.
- Points to RAID library integration.

a. Main components:
1. Data Ingestion & Preprocessing
- Raw HWT CSV (data/HWT_original_data.csv)
- Polished data CSV/JSON (data/polished & data/polished_json)
- Utilities: data/json_to_csv.py, data/cal_distance.py
2. Detector Engine
- CLI front-end: detect_cli.py
- Plugin loader: detectors/detector.py
- Detector modules: detectors/models/{binoculars, detectgpt, fast_detectgpt, gltr, llmdet, radar, gpt2_detector, commercial wrappers}
- External APIs: ZeroGPT, GPTZero, huggingface datasets
3. Evaluation Engine
- CLI front-end: evaluate_cli.py & evaluate_for_hybrid.py
- RAID integration: raid/detect.py, raid/evaluate.py, raid/utils.py
- Threshold calculation and hybrid evaluation
4. Results & Analysis
- JSON result files (predictions.json, results_acc.json, results_fpr.json under results/<model>/)
- Domain- and polish-type breakdowns (results/domain_results, results/domain_results_ratio)
- Plotting scripts: results/plot_domain_results.py, results/plots/, results/domain_results/…
5. Orchestration & Environment
- Shell scripts (run_detector_basic.sh, run_detector_hybrid_polish_prctg.sh, run_detector_hybrid_polish_type.sh)
- Environment setup: environment.yml, set_api_keys.sh

1. Data Ingestion → Detector Engine
- CSV/JSON → detect_cli.py → detectors/detector.py loads specific model → model returns prediction scores
- Predictions written to JSON under results/<model>/
2. Detector Engine → Evaluation Engine
- predictions.json + original CSV → evaluate_cli.py / evaluate_for_hybrid.py → evaluation metrics JSON
3. Evaluation Engine → Results & Analysis
- results_acc.json, results_fpr.json → further breakdown and plot generation scripts → charts & domain/polish tables
4. Orchestration
- Shell scripts sequence detection + evaluation for various polish-types and models
5. Environment
- environment.yml & set_api_keys.sh support reproducibility and external API credential injection

c. Architectural patterns & design principles
- Plugin architecture for detectors: detectors/models folder structure allows easy extension of new detectors.
- Clear separation of concerns: Data layer, Detection layer, Evaluation layer, and Analysis layer.
- Reuse of external library (RAID) for core detection/evaluation logic.
- CLI-driven workflows enabling batch automation via shell scripts.

d. Key technologies/frameworks
- Python 3.x
- Conda environment (environment.yml)
- Huggingface datasets (optional data loading)
- PyTorch / TensorFlow (underlying detector models)
- External APIs: OpenAI (ChatGPT-based detectors), ZeroGPT, GPTZero
- Data formats: CSV, JSON
- Plotting: Matplotlib/Seaborn PDFs

##### Detail level
- Shell scripts triggering end-to-end pipelines
- Environment setup step feeding API keys into detector modules
- Two evaluation paths: basic (evaluate_cli.py) vs. hybrid (evaluate_for_hybrid.py)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Ingestion & Preprocessing
    subgraph "Data Ingestion & Preprocessing"
        HWT["HWT_original_data.csv"]:::dataStore
        PolishedCSV["polished/"]:::dataStore
        PolishedJSON["polished_json/"]:::dataStore
        Json2CSV["json_to_csv.py"]:::dataModule
        CalDist["cal_distance.py"]:::dataModule
        PolishHWT["polish_hwt.py"]:::dataModule
        UtilsData["utils.py"]:::dataModule
    end

    %% Detector Engine
    subgraph "Detector Engine"
        DetectCLI["detect_cli.py"]:::cli
        PluginLoader["detectors/detector.py"]:::module
        mergeResults["merge_results.py"]:::module
        subgraph "Detector Plugins"
            binoc["binoculars.py"]:::module
            chatgpt["chatgpt_detector.py"]:::module
            detectgpt["detectgpt.py"]:::module
            fastdgpt["fast_detectgpt.py"]:::module
            gltr["gltr.py"]:::module
            gpt2["gpt2_detector.py"]:::module
            gptzero["gptzero.py"]:::module
            llmdet["llmdet.py"]:::module
            originality["originality_ai.py"]:::module
            pangram["pangram.py"]:::module
            radar["radar.py"]:::module
            winston["winston_ai.py"]:::module
            zerogpt["zerogpt.py"]:::module
        end
        ExternalAPIs["External APIs"]:::external
    end

    %% Evaluation Engine
    subgraph "Evaluation Engine"
        EvalCLI["evaluate_cli.py"]:::cli
        EvalHybrid["evaluate_for_hybrid.py"]:::cli
        RAID_Detect["raid/detect.py"]:::module
        RAID_Eval["raid/evaluate.py"]:::module
        RAID_Utils["raid/utils.py"]:::module
    end

    %% Results & Analysis
    subgraph "Results & Analysis"
        ResultsFolder["results/"]:::resultStore
        PlotDomain["plot_domain_results.py"]:::resultMod
        ComparePol["compare_polisher.py"]:::resultMod
        ExtractRes["extract_results.py"]:::resultMod
        DomainRes["domain_results.py"]:::resultMod
        PlotChange["plot_domain_change.py"]:::resultMod
        PlotsFolder["results/plots/"]:::resultStore
        ResultsUtils["results/utils.py"]:::resultMod
    end

    %% Orchestration & Environment
    subgraph "Orchestration & Environment"
        RunBasic["run_detector_basic.sh"]:::shell
        RunHybridPrctg["run_detector_hybrid_polish_prctg.sh"]:::shell
        RunHybridType["run_detector_hybrid_polish_type.sh"]:::shell
        SetKeys["set_api_keys.sh"]:::shell
        Env["environment.yml"]:::env
        Gitignore[".gitignore"]:::env
        Readme["README.md"]:::env
        ProjectUtils["utils.py"]:::module
    end

    %% Data Flow
    HWT -->|ingest| DetectCLI
    PolishedCSV -->|ingest| DetectCLI
    PolishedJSON -->|ingest| DetectCLI
    DetectCLI -->|loads| PluginLoader
    PluginLoader -->|runs| binoc & chatgpt & detectgpt & fastdgpt & gltr & gpt2 & gptzero & llmdet & originality & pangram & radar & winston & zerogpt
    binoc & chatgpt & detectgpt & fastdgpt & gltr & gpt2 & gptzero & llmdet & originality & pangram & radar & winston & zerogpt -->|predictions| mergeResults
    mergeResults -->|writes JSON| ResultsFolder

    ResultsFolder -->|input| EvalCLI
    HWT -->|input| EvalCLI
    EvalCLI -->|calls| RAID_Detect
    RAID_Detect -->|runs| RAID_Eval
    RAID_Eval -->|uses| RAID_Utils
    RAID_Eval -->|metrics JSON| ResultsFolder

    ResultsFolder -->|analysis| PlotDomain & ComparePol & ExtractRes & DomainRes & PlotChange & ResultsUtils
    PlotDomain & ComparePol & ExtractRes & DomainRes & PlotChange -->|outputs| PlotsFolder

    %% Orchestration & Environment Flow
    Env & SetKeys --> DetectCLI
    Env & SetKeys --> EvalCLI
    Env & SetKeys --> EvalHybrid
    RunBasic --> DetectCLI
    RunBasic --> EvalCLI
    RunHybridPrctg --> DetectCLI
    RunHybridPrctg --> EvalHybrid
    RunHybridType --> DetectCLI
    RunHybridType --> EvalHybrid

    ExternalAPIs --> PluginLoader

    %% Click Events
    click HWT "https://github.com/shoumiksaha/ai-polished-text/blob/main/data/HWT_original_data.csv"
    click PolishedCSV "https://github.com/shoumiksaha/ai-polished-text/tree/main/data/polished/"
    click PolishedJSON "https://github.com/shoumiksaha/ai-polished-text/tree/main/data/polished_json/"
    click Json2CSV "https://github.com/shoumiksaha/ai-polished-text/blob/main/data/json_to_csv.py"
    click CalDist "https://github.com/shoumiksaha/ai-polished-text/blob/main/data/cal_distance.py"
    click PolishHWT "https://github.com/shoumiksaha/ai-polished-text/blob/main/data/polish_hwt.py"
    click UtilsData "https://github.com/shoumiksaha/ai-polished-text/blob/main/data/utils.py"
    click DetectCLI "https://github.com/shoumiksaha/ai-polished-text/blob/main/detect_cli.py"
    click PluginLoader "https://github.com/shoumiksaha/ai-polished-text/blob/main/detectors/detector.py"
    click mergeResults "https://github.com/shoumiksaha/ai-polished-text/blob/main/detectors/merge_results.py"
    click binoc "https://github.com/shoumiksaha/ai-polished-text/blob/main/detectors/models/binoculars/binoculars.py"
    click chatgpt "https://github.com/shoumiksaha/ai-polished-text/blob/main/detectors/models/chatgpt_roberta_detector/chatgpt_detector.py"
    click detectgpt "https://github.com/shoumiksaha/ai-polished-text/blob/main/detectors/models/detectgpt/detectgpt.py"
    click fastdgpt "https://github.com/shoumiksaha/ai-polished-text/blob/main/detectors/models/fast_detectgpt/fast_detectgpt.py"
    click gltr "https://github.com/shoumiksaha/ai-polished-text/blob/main/detectors/models/gltr/gltr.py"
    click gpt2 "https://github.com/shoumiksaha/ai-polished-text/blob/main/detectors/models/gpt2_detector/gpt2_detector.py"
    click gptzero "https://github.com/shoumiksaha/ai-polished-text/blob/main/detectors/models/gptzero/gptzero.py"
    click llmdet "https://github.com/shoumiksaha/ai-polished-text/blob/main/detectors/models/llmdet/llmdet.py"
    click originality "https://github.com/shoumiksaha/ai-polished-text/blob/main/detectors/models/originality_ai/originality_ai.py"
    click pangram "https://github.com/shoumiksaha/ai-polished-text/blob/main/detectors/models/pangram/pangram.py"
    click radar "https://github.com/shoumiksaha/ai-polished-text/blob/main/detectors/models/radar/radar.py"
    click winston "https://github.com/shoumiksaha/ai-polished-text/blob/main/detectors/models/winston_ai/winston_ai.py"
    click zerogpt "https://github.com/shoumiksaha/ai-polished-text/blob/main/detectors/models/zerogpt/zerogpt.py"
    click EvalCLI "https://github.com/shoumiksaha/ai-polished-text/blob/main/evaluate_cli.py"
    click EvalHybrid "https://github.com/shoumiksaha/ai-polished-text/blob/main/evaluate_for_hybrid.py"
    click RAID_Detect "https://github.com/shoumiksaha/ai-polished-text/blob/main/raid/detect.py"
    click RAID_Eval "https://github.com/shoumiksaha/ai-polished-text/blob/main/raid/evaluate.py"
    click RAID_Utils "https://github.com/shoumiksaha/ai-polished-text/blob/main/raid/utils.py"
    click ResultsFolder "https://github.com/shoumiksaha/ai-polished-text/tree/main/results/"
    click PlotDomain "https://github.com/shoumiksaha/ai-polished-text/blob/main/results/plot_domain_results.py"
    click ComparePol "https://github.com/shoumiksaha/ai-polished-text/blob/main/results/compare_polisher.py"
    click ExtractRes "https://github.com/shoumiksaha/ai-polished-text/blob/main/results/extract_results.py"
    click DomainRes "https://github.com/shoumiksaha/ai-polished-text/blob/main/results/domain_results.py"
    click PlotChange "https://github.com/shoumiksaha/ai-polished-text/blob/main/results/plot_domain_change.py"
    click PlotsFolder "https://github.com/shoumiksaha/ai-polished-text/tree/main/results/plots/"
    click ResultsUtils "https://github.com/shoumiksaha/ai-polished-text/blob/main/results/utils.py"
    click RunBasic "https://github.com/shoumiksaha/ai-polished-text/blob/main/run_detector_basic.sh"
    click RunHybridPrctg "https://github.com/shoumiksaha/ai-polished-text/blob/main/run_detector_hybrid_polish_prctg.sh"
    click RunHybridType "https://github.com/shoumiksaha/ai-polished-text/blob/main/run_detector_hybrid_polish_type.sh"
    click SetKeys "https://github.com/shoumiksaha/ai-polished-text/blob/main/set_api_keys.sh"
    click Env "https://github.com/shoumiksaha/ai-polished-text/blob/main/environment.yml"
    click Gitignore "https://github.com/shoumiksaha/ai-polished-text/blob/main/.gitignore"
    click Readme "https://github.com/shoumiksaha/ai-polished-text/blob/main/README.md"
    click ProjectUtils "https://github.com/shoumiksaha/ai-polished-text/blob/main/utils.py"

    %% Styles
    classDef dataStore fill:#e0f7fa,stroke:#000
    classDef dataModule fill:#b3e5fc,stroke:#000
    classDef module fill:#c8e6c9,stroke:#000
    classDef cli fill:#a5d6a7,stroke:#000
    classDef shell fill:#eeeeee,stroke:#000
    classDef env fill:#eeeeee,stroke:#000
    classDef external fill:#ffe0b2,stroke:#000,shape:octagon
    classDef resultStore fill:#ffe0b2,stroke:#000
    classDef resultMod fill:#ffcc80,stroke:#000
```
