How haroon423/harmsafe-detection works
Project Type and Purpose
• This is a Python‑based single‑page “full‑stack” application built with Streamlit.
• Purpose: provide a user‑friendly UI for content moderation, combining text‑based and vision‑language models to detect harmful content in text and images.
File Structure
• Top‑level files (README.md, app.py, requirements.txt) suggest a monolithic codebase.
• The entry point is app.py (the Streamlit script).
• Dependencies in requirements.txt will include Streamlit, Hugging Face Transformers, Torch, etc.
• No separate frontend or backend folders—UI and logic live together in one script.
Key Insights from README
• Text Moderation: uses cardiffnlp/twitter-roberta-base-offensive.
• Image Captioning: uses BLIP.
• Image Moderation: the caption is re‑evaluated by the text model.
• Streamlit for interface, so all request handling, model loading, inference, and result rendering occur in the same process.
Main Components
a. Streamlit UI Layer
• Text Input widget
• File Upload widget
• Result display panels
b. Application Logic / Orchestrator
• Input validation
• Invocation of moderation pipelines
• Confidence‑score aggregation
c. Text Moderation Module
• Loads the Roberta‑based offensive classification model
• Accepts raw text → returns { label, confidence }
d. Image Captioning Module
• Loads the BLIP model
• Accepts image bytes → returns generated caption
e. Image Moderation Module
• Feeds the BLIP‑generated caption into the Text Moderation Module
• Returns { label, confidence } for image content
f. External Model Repository
• Hugging Face model hub (for on‑the‑fly downloads or versioned Docker image)
g. (Optional) Logging / Metrics
• Writes moderation outcomes and performance metrics (could be to file or external monitoring)
Relationships and Data Flow
• The Image pipeline branches: UI → Logic → Captioning → Logic → Text Moderation → Logic → UI.
Architectural Patterns and Principles
• Modular pipeline: clearly separated Text vs. Image pipelines.
• Single process (monolith) but logically layered (UI vs. Orchestrator vs. Model Inference).
• Lazy loading or caching of models to minimize startup time.
• Stateless request handling (per‑input moderation, no session state beyond Streamlit).
Technologies, Frameworks, Libraries
• Python 3.x, Streamlit
• Hugging Face Transformers, Torch / TensorFlow backend
• BLIP Vision‑Language Model
• CardiffNLP Roberta offensive classification
• (Optionally) Docker for containerization; GitHub Actions for CI/CD
Diagramming Recommendations
• Use clear, concise labels: “Streamlit UI,” “Text Moderation Module,” etc.
• UI components in blue
• Orchestrator / app logic in green
• Model inference modules in orange
• External services in gray