How mhbhuiyan99/seo_audit_tool works
1. What this repo is
- Sitewell is a single-service SEO audit application built on Django 5 plus Playwright.
- Its core job is to take one target URL, crawl/render it in a real Chromium browser, grade SEO and performance signals, store the result, and return a JSON report that the browser dashboard renders.
2. Top-level runtime shape
- The app runs as a containerized Django server with a Playwright-ready base image.
- There is no separate frontend build, worker queue, or microservice split; the main boundary is between the HTTP request layer, the browser automation worker thread, and the persistence layer.
3. Request entry and routing
- config/urls.py is the top-level router.
- It sends the dashboard page to Django templates and mounts the analyzer API under /api/.
- analyzer/urls.py exposes the audit and history endpoints used by the dashboard.
4. User-facing UI boundary
- templates/dashboard.html is a self-contained single-page interface.
- It uses vanilla JavaScript and Tailwind CDN, with no React/Vue or client-side build pipeline.
- The UI submits audit requests via fetch, renders the returned JSON, animates inline SVG score gauges, and can refresh historical scans from the API.
5. Audit execution pipeline
- The main audit endpoint is in analyzer/views.py.
- A request does not run the scan inline on the Django thread; it submits work to a module-level ThreadPoolExecutor and waits with a timeout.
- This is the key concurrency boundary: Django handles the request, a worker thread runs the browser scan, then the result comes back as JSON.
6. Browser automation and crawling
- analyzer/scanner.py is the scanning engine.
- It uses Playwright’s sync API to launch Chromium, open the target page, and collect page signals.
- The scanner includes practical guardrails such as navigation retries, content-type checks, and a robots-related fetch before scanning.
7. What the scanner extracts
- The scan is not a general crawler; it focuses on on-page SEO and performance signals for a single URL.
- Extracted data includes metadata, headings, structured data / JSON-LD, social metadata, images, links, and timing/performance information.
- The result is a structured “raw scan” dictionary that becomes the input to the rule engine.
8. Grading and diagnostics
- analyzer/seo_rules.py turns scan output into a scored report.
- It applies a fixed set of SEO rules and classifies each one as PASS, WARNING, or FAIL.
- Each check carries a score, severity, evidence snippet, and recommendation, which is what the dashboard displays.
9. Persistence and scan history
- analyzer/models.py defines the scan history model stored in SQLite.
- The full report is persisted in a JSONField, so the app can show previous audits without re-running the scan.
- The migrations directory shows this is a normal Django ORM-backed history store rather than an external analytics database.
10. API/data flow back to the browser
- The scan result is converted to a JSON response from the view layer.
- The dashboard then renders the overall score, category scores, rule cards, and filterable status tabs from that payload.
- A separate history endpoint lets the UI reload prior audits and selected scan details.
11. Important operational boundary
- The architecture is intentionally built around blocking Playwright work isolated from request threads.
- That prevents a slow or hung browser session from freezing the Django process.
- The timeout in the view layer acts as the service-level protection against dead pages, long redirects, or stalled browser activity.
12. Deployment and infrastructure
- Dockerfile and docker-compose.yml package the app into one containerized service on port 8000.
- The Playwright base image matters architecturally because it bundles the browser and Linux dependencies required for headless Chromium.
- The repo is therefore optimized for reproducible local/CI execution rather than distributed scaling.
Main entry points: views.py API views · models.py Scan record · scanner.py Scanner · dashboard.html Dashboard
Components 14
Django app
HTTP routing, request handling, and app settings
- URL router · Top-level Django router that splits dashboard and API traffic
config/urls.py - API routes · Analyzer endpoint routing for audit and history
analyzer/urls.py - API views · Runs audit requests and serves history responses
analyzer/views.py - Settings · Project configuration including database and app wiring
config/settings.py
Scan pipeline
Browser automation and SEO grading
- Scanner · Launches Chromium and collects page-level audit signals
analyzer/scanner.py - SEO rules · Transforms raw scan data into scored checks and recommendations
analyzer/seo_rules.py - Worker pool · Module-level executor that isolates blocking browser work from request threads
analyzer/views.py
UI and storage
Browser dashboard, Django models, and local persistence
- Dashboard · Single-page HTML interface that submits audits and renders results
templates/dashboard.html - Scan record · Stores audit history and full JSON reports in SQLite
analyzer/models.py - Report JSON · Persisted scan report used to reload prior audits
analyzer/models.py - Migrations · Database schema evolution for the scan history store
analyzer/migrations/0001_initial.py
Deployment
Containerized runtime and service wiring
- App image · Playwright-ready container image for Django and Chromium
Dockerfile - Compose stack · Local container orchestration for the single service
docker-compose.yml - Admin CLI · Command-line entry for running the Django project
manage.py
Connections 16
- URL router → Dashboard: serves · Routes browser requests to the dashboard page
- URL router → API routes: mounts /api · Delegates API traffic to the analyzer app
- API routes → API views: dispatches · Maps audit and history endpoints to view handlers
- Dashboard → API views: fetches · Submits audits and loads history over HTTP
- API views → Worker pool: submits work · Offloads scan execution to a worker thread
- Worker pool → Scanner: runs · Executes the Playwright scan in a blocking worker
- Scanner → SEO rules: raw scan · Produces structured scan data consumed by the rule engine
- SEO rules → API views: report · Returns scored checks and recommendations to the response layer
- API views → Scan record: persists · Stores completed audits and history records
- Scan record → Report JSON: contains · Keeps the full report payload in a JSON field
- Scan record → Migrations: evolves via · Schema changes are managed through Django migrations
- Settings → Scan record: configures db · Defines the ORM-backed SQLite storage setup
- App image → Scanner: packages · Provides Playwright and browser dependencies
- Compose stack → App image: builds · Runs the application as one containerized service
- Admin CLI → Settings: bootstraps · Starts the Django project using the configured settings
- API views → Dashboard: returns JSON · Sends the audit payload consumed by the UI