How karthik-md07/ai-web-vuln_scanner works
What this project is (type + purpose)
This is a Django monolith web application that provides an AI-assisted web vulnerability scanning service. Users submit a target URL via a web UI, the server crawls/pages the target, runs a set of vulnerability checks, applies an ML model (Random Forest) to classify/score risk, stores scan results/history in the DB, and generates a PDF report.
• Web UI (templates + JS) → Django web app → Scanning/ML/PDF subsystems → DB + files (+ external target websites)
Core Django project
• manage.py — entry point
• config/ — Django project config: settings, URLs, WSGI
• config/settings.py — installed apps, DB config (sqlite), static/templates config
• config/urls.py — root routing (likely includes scanner/urls.py)
Django apps (important: there appear to be two)
• scanner/ — looks like the primary app containing the actual scanner logic:
• views.py, urls.py — HTTP endpoints / request handling
• models.py — persistence for scans/results/history
• crawler.py — discovers pages/links on the target
• scanners.py — likely rule-based vulnerability checks (the 5 vuln types)
• ml_engine.py — feature extraction + model inference
• pdf_report.py — PDF generation (ReportLab)
• scanner/ml_data/training_data.csv — training dataset (not runtime-critical unless retraining)
• scanners/ — a second Django app (name is confusingly close to scanner/):
• apps.py, models.py, views.py, tests.py
Data/artifacts
• db.sqlite3 — runtime persistence
• ml_data/vuln_model.pkl — packaged trained model artifact used by ML engine
• .env — environment config (secrets, toggles)
• templates/dashboard.html — UI entry point
User Browser
• Loads dashboard.html
• Submits target URL (form POST or AJAX)
Django Routing
• config/urls.py → scanner/urls.py → scanner/views.py
Scan Orchestration (inside scanner app)
• scanner/views.py calls into scanning pipeline, typically:
• crawler.py to fetch and enumerate pages/links
• scanners.py to run vulnerability checks per URL/page
• ml_engine.py to compute risk score / classify vulnerabilities using ml_data/vuln_model.pkl
Persistence
• scanner/models.py writes Scan + Findings + Score + Timestamps to db.sqlite3
Response
• Returns JSON/HTML with results and risk score, updates UI
External dependency boundary
• All crawling/scanning uses:
• requests (HTTP)
• BeautifulSoup4 (HTML parsing)
• The target website(s) are external systems and must be drawn as a separate boundary.
Architectural patterns/design principles to emphasize (specific to this project)
• Monolithic Django architecture: single deployable unit; internal modules separate concerns.
• Pipeline pattern for scanning:
• Crawl → Detect (rules) → Score (ML) → Persist → Report
• Separation of concerns:
• Views/URLs handle HTTP
• Dedicated modules for crawler/scanners/ML/PDF
• Models isolate DB schema
If there’s “real-time scanning”, represent it as:
• synchronous request (browser waits) or
• pseudo-real-time updates (polling endpoints / AJAX)
Since no Celery/Redis is visible, assume synchronous + frontend updates, unless you confirm otherwise.
• Browser → view → crawler → scanners → ml_engine → models → view → Browser
• Scan, Finding, Report etc. (names to be confirmed by reading the file)