# humanai-foundation/issr architecture

> Architecture diagram of the GitHub repository humanai-foundation/issr, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/humanai-foundation/issr
- Repository: https://github.com/humanai-foundation/issr

## Overview

##### Overview & Project Purpose
- ISSR is an umbrella repository supporting multiple research projects in the social sciences. Each project focuses on different aspects—from analyzing language evolution on the Dark Web (using clustering, topic modeling, and deep learning) to gender roles analysis in educational choices, mixed-method data prediction, and fatigue/distracted driving detection using computer vision.
- The purpose is to provide reproducible analyses, state-of-the-art ML pipelines (e.g., BERT models, LSTM, LightGBM), and data visualizations that support interdisciplinary research.

##### Main Architectural Components
a. Projects as Modules:
- ISSR_Dark_Web_Domenico_Lacavalla: Contains sub-components for content analysis, thread analysis, topic modeling (BERTopic, LLAMA), and even computer vision aspects (for visual topic models).
- ISSR_Gender_Roles_Career_Rashi_Gupta: Houses analysis notebooks, reports, and presentations related to grade-wise educational data.
- ISSR_Improve_Accuracy_Mixed_Data_Shao_Jin: Comprises regression and classification analyses on mixed data, including raw dataset input and output analysis notebooks.
- ISSR_Fatigue_detection: Focuses on driver fatigue detection using computer vision; it includes YOLO, Mediapipe, and CLIP-based pipelines.

b. Common Components Across Projects:
- Data Sources & Datasets: Each project includes its own raw/input data files as well as processed datasets (e.g., CSVs, Excel files, and specialized .safetensors for ML models).
- Preprocessing & Utility Modules: The Util directory includes utility scripts for text preprocessing, clustering, and model helper functions.
- ML Models & Analysis Scripts: Multiple directories (such as MLModelsBERT, MergedModelBERT, and notebooks in each project) show the separation of training, saved models, and inference pipelines.
- Visualization & Reporting: The ShowResultsHTML directory and presentation files (e.g., PowerPoints) indicate a reporting layer used to share reproducible results.
- Experimentation & Grid Search: Directories like ResultsGridSearchBERTopic indicate components dedicated to hyperparameter tuning and analysis.

##### Relationships & Data Flow
- Data is ingested from various datasets (raw text files, annotated CSVs from computer vision experiments, XLSX files for quantitative analysis).
- Data Preprocessing & Utility Layer:
- Raw data is cleaned/preprocessed using scripts in the Util directory and separate notebooks in each analysis folder.
- ML Model Training & Analysis:
- Preprocessed data is fed into topic modeling (BERTopic, LLAMA), deep learning models (LSTM, YOLO, LightGBM, etc.), and other classification/regression models. Each project’s “Models” or “SavedModels” directories indicate training outputs.
- There’s further merging of results, as seen in the MergedModelBERT module, where content and thread analysis outcomes are combined.
- Result Generation & Visualization:
- Outputs (CSV results, HTML reproducibility reports, and interactive notebooks) are then visualized and shared. The ShowResultsHTML directory and various Jupyter notebooks support this stage.
- Feedback Loops:

## Diagram (Mermaid)

```mermaid
flowchart TD
    RawData["Raw Data Sources"]:::rawData
    Prep["Preprocessing & Utilities"]:::preprocess

    RawData -->|"data ingestion"| Prep

    %% Dark Web Project Subgraph
    subgraph "ISSR_Dark_Web_Domenico_Lacavalla"
        DWProj["Dark Web Project"]:::project
        CA["Content Analysis"]:::ml
        TA["Thread Analysis"]:::ml
        U["Utility Modules"]:::preprocess
        ML1["MLModelsBERT"]:::ml
        ML2["MergedModelBERT"]:::ml
        CML["Combined ML Models"]:::ml
        CV["Computer Vision (Visual Topic Models)"]:::ml
        VR["Visualization & Reporting"]:::vis

        DWProj --> CA
        DWProj --> TA
        DWProj --> U

        CA --> ML1
        TA --> ML1
        U  --> ML1
        ML1 --> ML2
        ML2 --> CML
        CV --> CML
        CML --> VR
        %% Feedback loop from visualization back to preprocessing for iterative refinement
        VR ---|feedback| Prep
    end

    %% Other Project Modules
    FD["Fatigue Detection (CV/YOLO/CLIP)"]:::project
    GR["Gender Roles Analysis"]:::project
    MD["Mixed Data Analysis"]:::project

    Prep --> FD
    Prep --> GR
    Prep --> MD

    %% Click Events for Mapped Components
    click DWProj "https://github.com/humanai-foundation/issr/tree/main/ISSR_Dark_Web_Domenico_Lacavalla"
    click CA "https://github.com/humanai-foundation/issr/tree/main/ISSR_Dark_Web_Domenico_Lacavalla/Analyze_files/CombiningAnalysisCompleteDataset/ContentAnalysis"
    click TA "https://github.com/humanai-foundation/issr/tree/main/ISSR_Dark_Web_Domenico_Lacavalla/Analyze_files/CombiningAnalysisCompleteDataset/ThreadAnalysis"
    click ML1 "https://github.com/humanai-foundation/issr/tree/main/ISSR_Dark_Web_Domenico_Lacavalla/MLModelsBERT"
    click ML2 "https://github.com/humanai-foundation/issr/tree/main/ISSR_Dark_Web_Domenico_Lacavalla/MergedModelBERT"
    click CV "https://github.com/humanai-foundation/issr/tree/main/ISSR_Dark_Web_Domenico_Lacavalla/ComputerVision"
    click VR "https://github.com/humanai-foundation/issr/tree/main/ISSR_Dark_Web_Domenico_Lacavalla/ShowResultsHTML"
    click U "https://github.com/humanai-foundation/issr/tree/main/ISSR_Dark_Web_Domenico_Lacavalla/Util"
    click FD "https://github.com/humanai-foundation/issr/tree/main/ISSR_Fatigue_detection"
    click GR "https://github.com/humanai-foundation/issr/tree/main/ISSR_Gender_Roles_Career_Rashi_Gupta"
    click MD "https://github.com/humanai-foundation/issr/tree/main/ISSR_Improve_Accuracy_Mixed_Data_Shao_Jin"

    %% Styles
    classDef rawData fill:#ADD8E6,stroke:#333,stroke-width:2px;
    classDef preprocess fill:#7FFFD4,stroke:#333,stroke-width:2px;
    classDef ml fill:#90EE90,stroke:#333,stroke-width:2px;
    classDef vis fill:#FFD700,stroke:#333,stroke-width:2px;
    classDef project fill:#D3D3D3,stroke:#333,stroke-width:2px;
```
