# loid122/secure_ai-powered_content_filtering_system architecture

> Architecture diagram of the GitHub repository loid122/secure_ai-powered_content_filtering_system, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/loid122/secure_ai-powered_content_filtering_system
- Repository: https://github.com/loid122/secure_ai-powered_content_filtering_system

## Overview

##### Project Type and Purpose
- This is a multi-component Secure AI-Powered Content Filtering System, combining:
- URL threat analysis (Task1)
- Secure client–server encryption demo (Task2)
- AI-based URL classification (Task3 – BERT & Random Forest)
- Production-style Flask API + a CLI blocking tool (Task4)
- Its goal is to ingest arbitrary URLs, securely transport them, evaluate them with traditional & AI-driven risk models, and expose results via an API or a CLI utility.

##### File-Structure Highlights
- Top-level Tasks folder splits each capability into its own subproject.
- Task1: standalone Python script aggregating Google Safe Browsing, VirusTotal, and IPQualityScore.
- Task2: client.py and server.py demonstrating AES-GCM + RSA encryption over Flask.
- Task3: two training pipelines under “Bert Model” and “Random Forest Model” (Jupyter notebooks + datasets + saved weights).
- Task4: “API” directory (Flaskapp.py + API client) and “CLI Tool” (tool.py) modifying the hosts file.

##### Key Components
a. Clients & Interfaces
- Task1 CLI script (URL analyzer)
- Task2 secure client (encrypts URL/data)
- Task4 API client (posts URLs for risk check)
- Task4 CLI tool (hosts-file blocker)
b. Backend Services
- Flask Server for Task2 (decrypts & returns data)
- Flaskapp (Task4) – URL risk‐evaluation API
c. External Threat Services
- Google Safe Browsing API
- VirusTotal API
- IPQualityScore API
d. AI Model Servers
- BERT inference service (loads bert_model2.pth)
- Random Forest inference (if exposed via API/notebook)
e. Data Stores & Artifacts
- Trained model files (bert_model2.pth)
- CSV/text datasets (in Task3 folders)
f. Training Environments (Dev Pipeline)
- Jupyter notebooks for training & testing

##### Interactions & Data Flows
1. Secure Client (Task2 client.py)
→ Encrypts payload with AES-GCM, wraps key with RSA public key
→ POST → Flask server (Task2)
← Decrypted data returned
2. URL Risk Evaluation (Task4 API)
- Client (Task4 client.py) POSTs URL → Flaskapp →
– Task1 module: aggregates third-party API calls
– Task3 BERT service: returns ML risk score
– (Optionally) Task3 Random Forest service
← Combined risk score/classification
3. CLI Tool (Task4/tool.py)
– Reads classification (safe/unsafe), modifies hosts file

##### Architectural Patterns & Principles
- Micro-modular design: each Task is a self-contained component.
- Layered approach:
1. Presentation/Client Layer (CLI & API clients)
2. Security Layer (Task2 encryption)
3. Risk-Evaluation Layer (Task1 aggregator & Task3 classifiers)
4. Data Storage (models, datasets)
- Service abstraction: ML models are served via REST (Flask) for testing/inference.
- Separation of concerns: training pipelines (notebooks) vs. serving code (Flaskapp.py).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Clients & Interfaces
    subgraph "Clients Layer"
        direction TB
        Task1CLI["Task1 CLI Script\n(URL Analyzer)"]:::client
        SecureClient["Task2 Secure Client\n(encrypts URL/data)"]:::client
        APIClient["Task4 API Client\n(posts URLs)"]:::client
        CLITool["Task4 CLI Tool\n(hosts-file blocker)"]:::client
    end

    %% Security Layer
    subgraph "Security Layer"
        direction TB
        DecryptService["Task2 Flask Server\n(POST /decrypt)"]:::service
    end

    %% Risk-Evaluation Layer
    subgraph "Risk-Evaluation Service"
        direction TB
        RiskAPI["Task4 URL Risk-Evaluation API\n(POST /check)"]:::service
        Task1Module["Task1 Aggregator Module"]:::service
        BERTService["BERT Inference Service\n(REST)"]:::ml
        RFService["Random Forest Service\n(REST)"]:::ml
    end

    %% External Threat Services
    subgraph "External Third-Party Services"
        direction TB
        GSB["Google Safe Browsing API"]:::external
        VirusTotal["VirusTotal API"]:::external
        IPQS["IPQualityScore API"]:::external
    end

    %% Data Stores & Artifacts
    subgraph "Data Stores & Artifacts"
        direction TB
        BERTModel["bert_model2.pth"]:::artifact
        DatasetsBERT["BERT Datasets\n(.csv, .txt)"]:::artifact
        DatasetsRF["RF Datasets\n(.csv, .txt)"]:::artifact
    end

    %% Training Pipeline (Dev)
    subgraph "Training Pipeline"
        direction TB
        TrainBERT["Train_Bert_Model.ipynb"]:::dev
        TestBERT["Test_Bertmodel.ipynb"]:::dev
        TrainRF["randforest.ipynb"]:::dev
    end

    %% Flows
    SecureClient -->|"AES-GCM + RSA over HTTPS"| DecryptService
    DecryptService -->|"Decrypted Data JSON"| SecureClient

    APIClient -->|"POST /check via REST"| RiskAPI
    RiskAPI -->|"calls aggregator"| Task1Module
    Task1Module -->|"HTTPS API calls"| GSB
    Task1Module -->|"HTTPS API calls"| VirusTotal
    Task1Module -->|"HTTPS API calls"| IPQS
    RiskAPI -->|"REST inference"| BERTService
    RiskAPI -.->|"optional REST inference"| RFService
    BERTService -->|"risk score JSON"| RiskAPI
    RFService -->|"risk score JSON"| RiskAPI
    RiskAPI -->|"combined score JSON"| APIClient
    APIClient -->|"classification"| CLITool
    CLITool -->|"modifies hosts file"| HostsNote[/"Requires admin rights"/]

    %% Training->Artifacts
    TrainBERT --> BERTModel
    TestBERT --> BERTModel
    TrainBERT --> DatasetsBERT
    TrainBERT --> DatasetsRF
    TrainRF --> DatasetsRF
    TrainRF --> DatasetsBERT

    %% Legend
    classDef client fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef service fill:#90EE90,stroke:#333,stroke-width:1px
    classDef external fill:#FFA500,stroke:#333,stroke-width:1px
    classDef ml fill:#DDA0DD,stroke:#333,stroke-width:1px
    classDef artifact fill:#DDA0DD,stroke-dasharray: 5 5,stroke:#333,stroke-width:1px
    classDef dev fill:#EEEEEE,stroke:#333,stroke-width:1px

    %% Click Events
    click Task1CLI "https://github.com/loid122/secure_ai-powered_content_filtering_system/blob/main/Task1/Task1.py"
    click SecureClient "https://github.com/loid122/secure_ai-powered_content_filtering_system/blob/main/Task2/client.py"
    click DecryptService "https://github.com/loid122/secure_ai-powered_content_filtering_system/blob/main/Task2/server.py"
    click APIClient "https://github.com/loid122/secure_ai-powered_content_filtering_system/blob/main/Task4/API/client.py"
    click RiskAPI "https://github.com/loid122/secure_ai-powered_content_filtering_system/blob/main/Task4/API/Flaskapp.py"
    click CLITool "https://github.com/loid122/secure_ai-powered_content_filtering_system/blob/main/Task4/CLI Tool/tool.py"
    click TrainBERT "https://github.com/loid122/secure_ai-powered_content_filtering_system/blob/main/Task3/Bert Model/Train_Bert_Model.ipynb"
    click TestBERT "https://github.com/loid122/secure_ai-powered_content_filtering_system/blob/main/Task3/Bert Model/Test_Bertmodel.ipynb"
    click TrainRF "https://github.com/loid122/secure_ai-powered_content_filtering_system/blob/main/Task3/Random Forest Model/randforest.ipynb"
    click DatasetsBERT "https://github.com/loid122/secure_ai-powered_content_filtering_system/blob/main/Task3/Bert Model/data.txt"
    click DatasetsRF "https://github.com/loid122/secure_ai-powered_content_filtering_system/blob/main/Task3/Random Forest Model/PhiUSIIL_Phishing_URL_Dataset.csv"
```
