# amitbcp/multilingual_profanity architecture

> Architecture diagram of the GitHub repository amitbcp/multilingual_profanity, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/amitbcp/multilingual_profanity
- Repository: https://github.com/amitbcp/multilingual_profanity

## Overview

##### Project type and purpose
- This is an open-source benchmarking toolkit (“SweEval-Bench”) for evaluating LLMs on safety by prompting them with multilingual swear-word instructions and measuring harmful outputs. It is not a web or mobile app but rather a data-science/ML evaluation pipeline comprised of Python scripts, Jupyter notebooks, data files, and metric outputs.

##### File structure
- notebooks/
- dataset/Swear_Words_Dataset.ipynb – exploration and construction of the prompt dataset
- inference/multi_experiment.ipynb, multi_experiment_2.ipynb – interactive model inference and result visualization
- src/
- downloader.py – retrieves models or data artifacts if needed
- load_dataset.py, dataset_config.py – loads prompt files and case-specific JSONs (case_1.json, case_2.json, case_3.json)
- load_model.py, infer_model.py – instantiates LLMs (e.g., via HuggingFace or API) and runs inference
- annotate_inference.py, eval_utils.py – applies manual or semi-automated annotation logic to mark harmful responses
- calculate_metrics.py, utils.py – computes harmful rates and other metrics, writes to Excel (src/metrics/*.xlsx)
- paths.py – centralizes directory paths for dataset, outputs, logs
- src/metrics/ – stores raw metric spreadsheets for each case
- requirements.txt – lists Python dependencies
- README.md – project overview, methodology, results, and proposed final structure

##### README for additional insights
- Architecture is implied as a linear data-processing pipeline: Dataset formation → Model inference → Manual annotation → Metric computation → Visualization/analysis.

## Diagram (Mermaid)

```mermaid
graph TB

  subgraph "Data Layer"
    Configs["Case Configs (JSON)"]:::data
    click Configs "https://github.com/amitbcp/multilingual_profanity/tree/main/src/details"
    DatasetNotebook["Swear Words Dataset Notebook"]:::viz
    click DatasetNotebook "https://github.com/amitbcp/multilingual_profanity/blob/main/notebooks/dataset/Swear_Words_Dataset.ipynb"
    LoadDataset["load_dataset.py"]:::internal
    click LoadDataset "https://github.com/amitbcp/multilingual_profanity/blob/main/src/load_dataset.py"
    DatasetConfig["dataset_config.py"]:::internal
    click DatasetConfig "https://github.com/amitbcp/multilingual_profanity/blob/main/src/dataset_config.py"
    Paths["paths.py"]:::internal
    click Paths "https://github.com/amitbcp/multilingual_profanity/blob/main/src/paths.py"

    Configs -->|"JSON prompts"| LoadDataset
    DatasetConfig --> LoadDataset
    Paths --> LoadDataset
  end

  subgraph "Inference Layer"
    Downloader["downloader.py"]:::internal
    click Downloader "https://github.com/amitbcp/multilingual_profanity/blob/main/src/downloader.py"
    LoadModel["load_model.py"]:::internal
    click LoadModel "https://github.com/amitbcp/multilingual_profanity/blob/main/src/load_model.py"
    InferModel["infer_model.py"]:::internal
    click InferModel "https://github.com/amitbcp/multilingual_profanity/blob/main/src/infer_model.py"
    ExternalLLM["LLM Providers"]:::external
    Storage["Storage"]:::external

    LoadDataset --> Downloader
    Downloader --> LoadModel
    LoadModel --> InferModel
    ExternalLLM --> InferModel
    InferModel --> Storage
  end

  subgraph "Annotation Layer"
    Annotate["annotate_inference.py"]:::internal
    click Annotate "https://github.com/amitbcp/multilingual_profanity/blob/main/src/annotate_inference.py"
    EvalUtils["eval_utils.py"]:::internal
    click EvalUtils "https://github.com/amitbcp/multilingual_profanity/blob/main/src/eval_utils.py"

    InferModel -->|"raw LLM responses"| Annotate
    EvalUtils --> Annotate
  end

  subgraph "Metrics Layer"
    CalculateMetrics["calculate_metrics.py"]:::internal
    click CalculateMetrics "https://github.com/amitbcp/multilingual_profanity/blob/main/src/calculate_metrics.py"
    Utils["utils.py"]:::internal
    click Utils "https://github.com/amitbcp/multilingual_profanity/blob/main/src/utils.py"
    MetricsStore["Metrics Data (Excel)"]:::data
    click MetricsStore "https://github.com/amitbcp/multilingual_profanity/tree/main/src/metrics"

    Annotate -->|"annotated CSV"| CalculateMetrics
    Utils --> CalculateMetrics
    CalculateMetrics -->|"metrics.xlsx"| MetricsStore
  end

  subgraph "Visualization Layer"
    Notebook1["multi_experiment.ipynb"]:::viz
    click Notebook1 "https://github.com/amitbcp/multilingual_profanity/blob/main/notebooks/inference/multi_experiment.ipynb"
    Notebook2["multi_experiment_2.ipynb"]:::viz
    click Notebook2 "https://github.com/amitbcp/multilingual_profanity/blob/main/notebooks/inference/multi_experiment_2.ipynb"

    MetricsStore -->|"metrics data"| Notebook1
    MetricsStore -->|"metrics data"| Notebook2
    Notebook1 -->|"insights"| DatasetConfig
  end

  subgraph "Project Config & Dependencies"
    Requirements["requirements.txt"]:::internal
    click Requirements "https://github.com/amitbcp/multilingual_profanity/blob/main/requirements.txt"
    Readme["README.md"]:::internal
    click Readme "https://github.com/amitbcp/multilingual_profanity/blob/main/README.md"
    GitignoreRoot[".gitignore"]:::internal
    click GitignoreRoot "https://github.com/amitbcp/multilingual_profanity/blob/main/.gitignore"
    GitignoreSrc["src/.gitignore"]:::internal
    click GitignoreSrc "https://github.com/amitbcp/multilingual_profanity/blob/main/src/.gitignore"
    License["LICENSE"]:::internal
    click License "https://github.com/amitbcp/multilingual_profanity/tree/main/LICENSE"
  end

  classDef internal fill:#D6EAF8,stroke:#1B4F72;
  classDef data fill:#D5F5E3,stroke:#145A32;
  classDef external fill:#FDEBD0,stroke:#B9770E;
  classDef viz fill:#E8DAEF,stroke:#6C3483;
```
