How loid122/secure_ai-powered_content_filtering_system works
Project Type and Purpose
• This is a multi-component Secure AI-Powered Content Filtering System, combining:
• URL threat analysis (Task1)
• Secure client–server encryption demo (Task2)
• AI-based URL classification (Task3 – BERT & Random Forest)
• Production-style Flask API + a CLI blocking tool (Task4)
• Its goal is to ingest arbitrary URLs, securely transport them, evaluate them with traditional & AI-driven risk models, and expose results via an API or a CLI utility.
File-Structure Highlights
• Top-level Tasks folder splits each capability into its own subproject.
• Task1: standalone Python script aggregating Google Safe Browsing, VirusTotal, and IPQualityScore.
• Task2: client.py and server.py demonstrating AES-GCM + RSA encryption over Flask.
• Task3: two training pipelines under “Bert Model” and “Random Forest Model” (Jupyter notebooks + datasets + saved weights).
• Task4: “API” directory (Flaskapp.py + API client) and “CLI Tool” (tool.py) modifying the hosts file.
Key Components
a. Clients & Interfaces
• Task1 CLI script (URL analyzer)
• Task2 secure client (encrypts URL/data)
• Task4 API client (posts URLs for risk check)
• Task4 CLI tool (hosts-file blocker)
b. Backend Services
• Flask Server for Task2 (decrypts & returns data)
• Flaskapp (Task4) – URL risk‐evaluation API
c. External Threat Services
• Google Safe Browsing API
• VirusTotal API
• IPQualityScore API
d. AI Model Servers
• BERT inference service (loads bert_model2.pth)
• Random Forest inference (if exposed via API/notebook)
e. Data Stores & Artifacts
• Trained model files (bert_model2.pth)
• CSV/text datasets (in Task3 folders)
f. Training Environments (Dev Pipeline)
• Jupyter notebooks for training & testing
Interactions & Data Flows
1. Secure Client (Task2 client.py)
→ Encrypts payload with AES-GCM, wraps key with RSA public key
→ POST → Flask server (Task2)
← Decrypted data returned
2. URL Risk Evaluation (Task4 API)
• Client (Task4 client.py) POSTs URL → Flaskapp →
– Task1 module: aggregates third-party API calls
– Task3 BERT service: returns ML risk score
– (Optionally) Task3 Random Forest service
← Combined risk score/classification
3. CLI Tool (Task4/tool.py)
– Reads classification (safe/unsafe), modifies hosts file
Architectural Patterns & Principles
• Micro-modular design: each Task is a self-contained component.
• Layered approach:
1. Presentation/Client Layer (CLI & API clients)
2. Security Layer (Task2 encryption)
3. Risk-Evaluation Layer (Task1 aggregator & Task3 classifiers)
4. Data Storage (models, datasets)
• Service abstraction: ML models are served via REST (Flask) for testing/inference.
• Separation of concerns: training pipelines (notebooks) vs. serving code (Flaskapp.py).