# cabooseexia/dsa4264-problem-2 architecture

> Architecture diagram of the GitHub repository cabooseexia/dsa4264-problem-2, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/cabooseexia/dsa4264-problem-2
- Repository: https://github.com/cabooseexia/dsa4264-problem-2

## Overview

This repository is a Reddit hate-speech detection and analysis project, combining interactive data preparation and labeling, model experimentation, topic and behavioral analysis, and an explanatory dashboard. The documented data workflow begins with Reddit threads and labeled examples, then develops analysis outputs that the dashboard presents through filters, trends, post summaries, and model explanations. The tree and README identify broader notebook capabilities, but their internal calls and data handoffs were not sampled; those links are therefore omitted rather than inferred. Dashboard-to-shared helper usage and the two interactive labeling scripts are visible in the excerpts.

Main entry points: [`dashboard-reddit/app.py`](https://github.com/cabooseexia/dsa4264-problem-2/blob/main/dashboard-reddit/app.py) (Shiny Analysis Dashboard), [`for_expertly_labelling/setup_for_expertly_labelled_data.py`](https://github.com/cabooseexia/dsa4264-problem-2/blob/main/for_expertly_labelling/setup_for_expertly_labelled_data.py) (Expert Sample Preparation), [`for_expertly_labelling/expert_data_labeller.py`](https://github.com/cabooseexia/dsa4264-problem-2/blob/main/for_expertly_labelling/expert_data_labeller.py) (Expert Labeling), [`phase_2/Reddit_Report4.ipynb`](https://github.com/cabooseexia/dsa4264-problem-2/blob/main/phase_2/Reddit_Report4.ipynb) (Analysis Reporting)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_labeling["Data Preparation &amp; Labeling"]
  node_expertprep["Expert Sample Preparation"]
  node_expertlabel["Expert Labeling"]
  node_labelnotebook["Data Labeling<br/>[Labelling.ipynb]"]
end

subgraph group_modeling["Modeling &amp; Topic Analysis"]
  node_modeltests["Model Experiments"]
  node_apitests["API Model Experiments"]
  node_redditbert["Reddit BERT Analysis<br/>[Reddit_BERT.ipynb]"]
  node_postextract["Post Title Extraction"]
  node_postprocess["Post Topic Processing"]
  node_report["Analysis Reporting"]
end

subgraph group_dashboard["Interactive Dashboard"]
  node_dashboard["Shiny Analysis Dashboard<br/>[app.py]"]
  node_dashboardhelpers["Model &amp; Explanation Helpers<br/>[shared.py]"]
end

subgraph group_insights["Behavior &amp; Explainability"]
  node_topicseries["Topic Time Series"]
  node_userbehavior["User Behavior Analysis"]
  node_xai["Model Explainability<br/>[XAI.ipynb]"]
end

node_analyst(("Analyst / Labeler"))
node_threaddata["Reddit Thread Data"]
node_dashboarduser(("Dashboard User"))

node_threaddata -->|"samples and cleans"| node_expertprep
node_expertprep -->|"requests selection"| node_analyst
node_expertprep -.->|"saves selected rows"| node_expertlabel
node_analyst -->|"supplies labels"| node_expertlabel
node_expertlabel -->|"writes labeled CSV"| node_threaddata
node_dashboarduser -.->|"filters and explores"| node_dashboard
node_dashboard -->|"uses model helpers"| node_dashboardhelpers
node_dashboardhelpers -->|"returns predictions and explanations"| node_dashboard
node_labelnotebook -.->|"informs experiments"| node_modeltests
node_modeltests -.->|"supports Reddit modeling"| node_redditbert
node_postextract -.->|"feeds topic analysis"| node_postprocess
node_postprocess -.->|"enables trend analysis"| node_topicseries
node_redditbert -.->|"informs reporting"| node_report
node_topicseries -.->|"informs reporting"| node_report
node_userbehavior -.->|"informs reporting"| node_report
node_xai -.->|"provides explainability capability"| node_dashboard

click node_expertprep "https://github.com/cabooseexia/dsa4264-problem-2/blob/main/for_expertly_labelling/setup_for_expertly_labelled_data.py"
click node_expertlabel "https://github.com/cabooseexia/dsa4264-problem-2/blob/main/for_expertly_labelling/expert_data_labeller.py"
click node_labelnotebook "https://github.com/cabooseexia/dsa4264-problem-2/blob/main/phase_1/Labelling.ipynb"
click node_modeltests "https://github.com/cabooseexia/dsa4264-problem-2/blob/main/phase_1/BERT_and_Llama_model_testing.ipynb"
click node_apitests "https://github.com/cabooseexia/dsa4264-problem-2/blob/main/phase_1/api_model_testing.ipynb"
click node_redditbert "https://github.com/cabooseexia/dsa4264-problem-2/blob/main/phase_2/Reddit_BERT.ipynb"
click node_postextract "https://github.com/cabooseexia/dsa4264-problem-2/blob/main/phase_2/Reddit_Post_Title_Extraction2.ipynb"
click node_postprocess "https://github.com/cabooseexia/dsa4264-problem-2/blob/main/phase_2/Reddit_Data_Processing_%26_Post_Topics2.ipynb"
click node_report "https://github.com/cabooseexia/dsa4264-problem-2/blob/main/phase_2/Reddit_Report4.ipynb"
click node_topicseries "https://github.com/cabooseexia/dsa4264-problem-2/blob/main/phase_3/TopicModelling_TimeSeries.ipynb"
click node_userbehavior "https://github.com/cabooseexia/dsa4264-problem-2/blob/main/phase_3/User_Behaviour.ipynb"
click node_xai "https://github.com/cabooseexia/dsa4264-problem-2/blob/main/phase_4/XAI.ipynb"
click node_dashboard "https://github.com/cabooseexia/dsa4264-problem-2/blob/main/dashboard-reddit/app.py"
click node_dashboardhelpers "https://github.com/cabooseexia/dsa4264-problem-2/blob/main/dashboard-reddit/shared.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_expertprep,node_expertlabel,node_labelnotebook,node_dashboarduser toneBlue
class node_modeltests,node_apitests,node_redditbert,node_postextract,node_postprocess,node_report toneAmber
class node_dashboard,node_dashboardhelpers toneMint
class node_topicseries,node_userbehavior,node_xai toneRose
class node_analyst,node_threaddata toneIndigo
```

## Components

### Data Preparation & Labeling

- [Expert Sample Preparation](https://github.com/cabooseexia/dsa4264-problem-2/blob/main/for_expertly_labelling/setup_for_expertly_labelled_data.py): `for_expertly_labelling/setup_for_expertly_labelled_data.py`
- [Expert Labeling](https://github.com/cabooseexia/dsa4264-problem-2/blob/main/for_expertly_labelling/expert_data_labeller.py): `for_expertly_labelling/expert_data_labeller.py`
- [Data Labeling](https://github.com/cabooseexia/dsa4264-problem-2/blob/main/phase_1/Labelling.ipynb): `phase_1/Labelling.ipynb`

### Modeling & Topic Analysis

- [Model Experiments](https://github.com/cabooseexia/dsa4264-problem-2/blob/main/phase_1/BERT_and_Llama_model_testing.ipynb): `phase_1/BERT_and_Llama_model_testing.ipynb`
- [API Model Experiments](https://github.com/cabooseexia/dsa4264-problem-2/blob/main/phase_1/api_model_testing.ipynb): `phase_1/api_model_testing.ipynb`
- [Reddit BERT Analysis](https://github.com/cabooseexia/dsa4264-problem-2/blob/main/phase_2/Reddit_BERT.ipynb): `phase_2/Reddit_BERT.ipynb`
- [Post Title Extraction](https://github.com/cabooseexia/dsa4264-problem-2/blob/main/phase_2/Reddit_Post_Title_Extraction2.ipynb): `phase_2/Reddit_Post_Title_Extraction2.ipynb`
- [Post Topic Processing](https://github.com/cabooseexia/dsa4264-problem-2/blob/main/phase_2/Reddit_Data_Processing_%26_Post_Topics2.ipynb): `phase_2/Reddit_Data_Processing_&_Post_Topics2.ipynb`
- [Analysis Reporting](https://github.com/cabooseexia/dsa4264-problem-2/blob/main/phase_2/Reddit_Report4.ipynb): `phase_2/Reddit_Report4.ipynb`

### Interactive Dashboard

- [Shiny Analysis Dashboard](https://github.com/cabooseexia/dsa4264-problem-2/blob/main/dashboard-reddit/app.py): `dashboard-reddit/app.py`
- [Model & Explanation Helpers](https://github.com/cabooseexia/dsa4264-problem-2/blob/main/dashboard-reddit/shared.py): `dashboard-reddit/shared.py`

### Behavior & Explainability

- [Topic Time Series](https://github.com/cabooseexia/dsa4264-problem-2/blob/main/phase_3/TopicModelling_TimeSeries.ipynb): `phase_3/TopicModelling_TimeSeries.ipynb`
- [User Behavior Analysis](https://github.com/cabooseexia/dsa4264-problem-2/blob/main/phase_3/User_Behaviour.ipynb): `phase_3/User_Behaviour.ipynb`
- [Model Explainability](https://github.com/cabooseexia/dsa4264-problem-2/blob/main/phase_4/XAI.ipynb): `phase_4/XAI.ipynb`

### Other components

- **Analyst / Labeler**
- **Reddit Thread Data**
- **Dashboard User**

## Connections

- Reddit Thread Data → Expert Sample Preparation: samples and cleans (evidence: `for_expertly_labelling/setup_for_expertly_labelled_data.py`)
- Expert Sample Preparation → Analyst / Labeler: requests selection (evidence: `for_expertly_labelling/setup_for_expertly_labelled_data.py`)
- Expert Sample Preparation → Expert Labeling: saves selected rows
- Analyst / Labeler → Expert Labeling: supplies labels (evidence: `for_expertly_labelling/expert_data_labeller.py`)
- Expert Labeling → Reddit Thread Data: writes labeled CSV (evidence: `for_expertly_labelling/expert_data_labeller.py`)
- Dashboard User → Shiny Analysis Dashboard: filters and explores
- Shiny Analysis Dashboard → Model & Explanation Helpers: uses model helpers (evidence: `dashboard-reddit/app.py`)
- Model & Explanation Helpers → Shiny Analysis Dashboard: returns predictions and explanations (evidence: `dashboard-reddit/shared.py`)
- Data Labeling → Model Experiments: informs experiments
- Model Experiments → Reddit BERT Analysis: supports Reddit modeling
- Post Title Extraction → Post Topic Processing: feeds topic analysis
- Post Topic Processing → Topic Time Series: enables trend analysis
- Reddit BERT Analysis → Analysis Reporting: informs reporting
- Topic Time Series → Analysis Reporting: informs reporting
- User Behavior Analysis → Analysis Reporting: informs reporting
- Model Explainability → Shiny Analysis Dashboard: provides explainability capability
