# technology-ai-cogni/seo-backend architecture

> Architecture diagram of the GitHub repository technology-ai-cogni/seo-backend, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-08-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/technology-ai-cogni/seo-backend
- Repository: https://github.com/technology-ai-cogni/seo-backend

## Overview

#### Purpose and Entry Points
This repository is an SEO keyword-research, categorization, ranking, and dashboard system. It combines a Python backend that processes uploaded keyword CSVs and retrieves ranking/category data with a Vite/React dashboard for project and SEO-analysis views.

Primary runtime entry points are `backend/app.py` for the API service, `backend/scripts/run_pipeline.py` and `backend/scripts/run_pipeline_brightdata.py` for batch processing, and `frontend/seo-dashboard/src/main.jsx` for the browser application.

#### Backend API and Persistence Boundary
The backend exposes application behavior from `backend/app.py`, with database setup/access isolated in `backend/core/db.py`. Service-layer modules separate API-facing operations from ranking and category-checking logic:
- `backend/services/rank_checker.py`
- `backend/services/category_checker.py`
- `backend/core/db.py`

This is the main boundary between client/API requests, persisted project or analysis state, and external SEO-data retrieval.

#### Keyword Processing Pipeline
CSV datasets are the primary input/output contract. Source and generated files live under `backend/datasets/` and `backend/datasets/uploads/`, including pipeline-enriched CSVs. Pipeline orchestration is represented by:
- `backend/scripts/run_pipeline.py`
- `backend/scripts/run_pipeline_brightdata.py`
- `backend/scripts/category_assigner.py`

The flow is: ingest keyword sheet → retrieve/search-enrich data → classify intent, landing/blog suitability, and categories/clusters → write enriched pipeline artifacts for downstream API/dashboard consumption.

#### Search, Ranking, and Competitive Analysis
Ranking and SERP collection are implemented as dedicated scripts/services rather than frontend logic:
- `backend/scripts/serp_scraper.py`
- `backend/scripts/agentic_rank_checker.py`
- `backend/scripts/comp_analysis.py`

`backend/scripts/hosted_rank_check.py` and `backend/scripts/hosted_categorize.py` indicate hosted/external execution paths, while `backend/scripts/run_pipeline_brightdata.py` signals Bright Data-backed collection. These integrations are material external-service boundaries and should be treated as failure-prone, rate-limited stages with normalized outputs before persistence.

#### Experimental Category Intelligence
A more granular category pipeline exists independently under `backend/scripts/exp_category_pipeline/`. Its stages divide classification, SERP retrieval, metadata collection, clustering, grouping, naming, and recategorization:
- `backend/scripts/exp_category_pipeline/run_experiment.py`
- `backend/scripts/exp_category_pipeline/serp_fetch.py`
- `backend/scripts/exp_category_pipeline/metadata_fetch.py`
- `backend/scripts/exp_category_pipeline/classifiers.py`
- `backend/scripts/exp_category_pipeline/cluster_grouper.py`
- `backend/scripts/exp_category_pipeline/sequential_recategorize.py`

This is an analysis pipeline boundary: it transforms raw keyword/search evidence into structured category and cluster assignments, likely producing candidate logic for the main pipeline.

#### Frontend Dashboard
The frontend is a Vite React single-page application rooted at `frontend/seo-dashboard/src/main.jsx` and `frontend/seo-dashboard/src/App.jsx`. Shared navigation/layout is centralized in:
- `frontend/seo-dashboard/src/components/layout/Sidebar.jsx`
- `frontend/seo-dashboard/src/components/layout/Topbar.jsx`
- `frontend/seo-dashboard/src/data/navigation.js`

Feature pages cover dashboarding, keyword analysis, positions, competitors, top pages, AI visibility, and content planning. Core views include `frontend/seo-dashboard/src/components/pages/KeywordsPage.jsx`, `frontend/seo-dashboard/src/components/pages/PositionAnalysisPage.jsx`, and `frontend/seo-dashboard/src/components/pages/ContentEnginePage.jsx`.

#### Client Data Sources and Infrastructure
The dashboard has both local/demo and remote-data paths. CSV parsing and mock datasets are in `frontend/seo-dashboard/src/data/csvParser.js` and `frontend/seo-dashboard/src/data/mockData.js`; project access is abstracted by `frontend/seo-dashboard/src/lib/projectsApi.js`. `frontend/seo-dashboard/src/lib/supabaseClient.js` identifies Supabase as a client-side backend/service integration.

Deployment/runtime packaging is split between a Dockerized Python backend (`backend/Dockerfile`) and the Node/Vite frontend (`frontend/seo-dashboard/package.json`). The key architectural concern is keeping CSV/batch pipeline artifacts, backend database records, Supabase project data, and dashboard models consistent rather than allowing each path to become an independent source of truth.

Main entry points: [`backend/app.py`](https://github.com/technology-ai-cogni/seo-backend/blob/main/backend/app.py) (Python API), [`frontend/seo-dashboard/src/App.jsx`](https://github.com/technology-ai-cogni/seo-backend/blob/main/frontend/seo-dashboard/src/App.jsx) (Dashboard application), [`backend/scripts/run_pipeline.py`](https://github.com/technology-ai-cogni/seo-backend/blob/main/backend/scripts/run_pipeline.py) (Standard pipeline), [`frontend/seo-dashboard/src/lib/projectsApi.js`](https://github.com/technology-ai-cogni/seo-backend/blob/main/frontend/seo-dashboard/src/lib/projectsApi.js) (Projects API client)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_backend["Backend API"]
  node_api["Python API<br/>API service<br/>[app.py]"]
  node_db[("Application database<br/>persistence<br/>[db.py]")]
  node_rank_service["Rank checker<br/>backend service<br/>[rank_checker.py]"]
  node_category_service["Category checker<br/>backend service"]
end

subgraph group_pipeline["Keyword Pipelines"]
  node_csv_artifacts["Keyword CSV artifacts<br/>dataset contract"]
  node_main_pipeline["Standard pipeline<br/>batch orchestrator<br/>[run_pipeline.py]"]
  node_brightdata_pipeline["Bright Data pipeline<br/>batch orchestrator"]
  node_category_assigner["Category assignment<br/>pipeline stage"]
  node_serp_ranking["SERP and ranking collection<br/>collection stage<br/>[serp_scraper.py]"]
  node_hosted_execution{{"Hosted SEO execution<br/>external integration"}}
end

subgraph group_experimental["Experimental Categories"]
  node_experiment_runner["Category experiment<br/>analysis orchestrator<br/>[run_experiment.py]"]
  node_evidence_fetch["SERP and metadata fetch<br/>analysis stages<br/>[serp_fetch.py]"]
  node_classify_cluster["Classify and cluster<br/>analysis stages<br/>[classifiers.py]"]
end

subgraph group_frontend["React Dashboard"]
  node_dashboard_app["Dashboard application<br/>React SPA<br/>[App.jsx]"]
  node_browser_entry["Browser entry<br/>Vite entry<br/>[main.jsx]"]
  node_seo_views["SEO analysis views<br/>feature pages<br/>[KeywordsPage.jsx]"]
  node_projects_client["Projects API client<br/>client data access<br/>[projectsApi.js]"]
  node_local_data["CSV and mock data<br/>demo data path<br/>[csvParser.js]"]
  node_supabase[("Supabase<br/>client backend<br/>[supabaseClient.js]")]
end

node_external_search{{"Search data providers<br/>external service"}}

node_browser_entry -->|"mounts"| node_dashboard_app
node_dashboard_app -->|"renders"| node_seo_views
node_dashboard_app -->|"loads projects"| node_projects_client
node_dashboard_app -.->|"demo/import path"| node_local_data
node_projects_client -->|"reads and writes"| node_supabase
node_projects_client -->|"SEO operations"| node_api
node_api -->|"persists analysis state"| node_db
node_api -->|"checks rankings"| node_rank_service
node_api -->|"checks categories"| node_category_service
node_rank_service -->|"retrieves ranking data"| node_external_search
node_category_service -->|"retrieves category evidence"| node_external_search
node_csv_artifacts -->|"keyword input"| node_main_pipeline
node_main_pipeline -->|"enriches search data"| node_serp_ranking
node_serp_ranking -->|"collects SERPs"| node_external_search
node_main_pipeline -->|"assigns intent and categories"| node_category_assigner
node_category_assigner -->|"writes enriched CSV"| node_csv_artifacts
node_brightdata_pipeline -->|"uses hosted collection"| node_hosted_execution
node_hosted_execution -->|"calls providers"| node_external_search
node_brightdata_pipeline -->|"writes normalized output"| node_csv_artifacts
node_csv_artifacts -.->|"experimental input"| node_experiment_runner
node_experiment_runner -->|"collects evidence"| node_evidence_fetch
node_evidence_fetch -->|"supplies SERP metadata"| node_classify_cluster
node_classify_cluster -.->|"candidate assignments"| node_csv_artifacts
node_csv_artifacts -->|"downstream analysis data"| node_api

click node_api "https://github.com/technology-ai-cogni/seo-backend/blob/main/backend/app.py"
click node_db "https://github.com/technology-ai-cogni/seo-backend/blob/main/backend/core/db.py"
click node_rank_service "https://github.com/technology-ai-cogni/seo-backend/blob/main/backend/services/rank_checker.py"
click node_category_service "https://github.com/technology-ai-cogni/seo-backend/blob/main/backend/services/category_checker.py"
click node_main_pipeline "https://github.com/technology-ai-cogni/seo-backend/blob/main/backend/scripts/run_pipeline.py"
click node_brightdata_pipeline "https://github.com/technology-ai-cogni/seo-backend/blob/main/backend/scripts/run_pipeline_brightdata.py"
click node_category_assigner "https://github.com/technology-ai-cogni/seo-backend/blob/main/backend/scripts/category_assigner.py"
click node_serp_ranking "https://github.com/technology-ai-cogni/seo-backend/blob/main/backend/scripts/serp_scraper.py"
click node_hosted_execution "https://github.com/technology-ai-cogni/seo-backend/blob/main/backend/scripts/hosted_rank_check.py"
click node_experiment_runner "https://github.com/technology-ai-cogni/seo-backend/blob/main/backend/scripts/exp_category_pipeline/run_experiment.py"
click node_evidence_fetch "https://github.com/technology-ai-cogni/seo-backend/blob/main/backend/scripts/exp_category_pipeline/serp_fetch.py"
click node_classify_cluster "https://github.com/technology-ai-cogni/seo-backend/blob/main/backend/scripts/exp_category_pipeline/classifiers.py"
click node_dashboard_app "https://github.com/technology-ai-cogni/seo-backend/blob/main/frontend/seo-dashboard/src/App.jsx"
click node_browser_entry "https://github.com/technology-ai-cogni/seo-backend/blob/main/frontend/seo-dashboard/src/main.jsx"
click node_seo_views "https://github.com/technology-ai-cogni/seo-backend/blob/main/frontend/seo-dashboard/src/components/pages/KeywordsPage.jsx"
click node_projects_client "https://github.com/technology-ai-cogni/seo-backend/blob/main/frontend/seo-dashboard/src/lib/projectsApi.js"
click node_local_data "https://github.com/technology-ai-cogni/seo-backend/blob/main/frontend/seo-dashboard/src/data/csvParser.js"
click node_supabase "https://github.com/technology-ai-cogni/seo-backend/blob/main/frontend/seo-dashboard/src/lib/supabaseClient.js"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_api,node_db,node_rank_service,node_category_service toneBlue
class node_csv_artifacts,node_main_pipeline,node_brightdata_pipeline,node_category_assigner,node_serp_ranking,node_hosted_execution toneAmber
class node_experiment_runner,node_evidence_fetch,node_classify_cluster toneMint
class node_dashboard_app,node_browser_entry,node_seo_views,node_projects_client,node_local_data,node_supabase toneRose
class node_external_search toneNeutral
```

## Components

### Backend API

- [Python API](https://github.com/technology-ai-cogni/seo-backend/blob/main/backend/app.py): API service · Serves project and SEO-analysis operations. · `backend/app.py`
- [Application database](https://github.com/technology-ai-cogni/seo-backend/blob/main/backend/core/db.py): persistence · Database setup and access boundary. · `backend/core/db.py`
- [Rank checker](https://github.com/technology-ai-cogni/seo-backend/blob/main/backend/services/rank_checker.py): backend service · `backend/services/rank_checker.py`
- [Category checker](https://github.com/technology-ai-cogni/seo-backend/blob/main/backend/services/category_checker.py): backend service · `backend/services/category_checker.py`

### Keyword Pipelines

- **Keyword CSV artifacts**: dataset contract · Source uploads and enriched pipeline outputs.
- [Standard pipeline](https://github.com/technology-ai-cogni/seo-backend/blob/main/backend/scripts/run_pipeline.py): batch orchestrator · Ingests and enriches keyword sheets. · `backend/scripts/run_pipeline.py`
- [Bright Data pipeline](https://github.com/technology-ai-cogni/seo-backend/blob/main/backend/scripts/run_pipeline_brightdata.py): batch orchestrator · Runs collection through Bright Data. · `backend/scripts/run_pipeline_brightdata.py`
- [Category assignment](https://github.com/technology-ai-cogni/seo-backend/blob/main/backend/scripts/category_assigner.py): pipeline stage · `backend/scripts/category_assigner.py`
- [SERP and ranking collection](https://github.com/technology-ai-cogni/seo-backend/blob/main/backend/scripts/serp_scraper.py): collection stage · Normalizes search and competitive evidence. · `backend/scripts/serp_scraper.py`
- [Hosted SEO execution](https://github.com/technology-ai-cogni/seo-backend/blob/main/backend/scripts/hosted_rank_check.py): external integration · Rate-limited external rank and categorization path. · `backend/scripts/hosted_rank_check.py`

### Experimental Categories

- [Category experiment](https://github.com/technology-ai-cogni/seo-backend/blob/main/backend/scripts/exp_category_pipeline/run_experiment.py): analysis orchestrator · `backend/scripts/exp_category_pipeline/run_experiment.py`
- [SERP and metadata fetch](https://github.com/technology-ai-cogni/seo-backend/blob/main/backend/scripts/exp_category_pipeline/serp_fetch.py): analysis stages · Collects evidence for category intelligence. · `backend/scripts/exp_category_pipeline/serp_fetch.py`
- [Classify and cluster](https://github.com/technology-ai-cogni/seo-backend/blob/main/backend/scripts/exp_category_pipeline/classifiers.py): analysis stages · Produces category and cluster assignments. · `backend/scripts/exp_category_pipeline/classifiers.py`

### React Dashboard

- [Dashboard application](https://github.com/technology-ai-cogni/seo-backend/blob/main/frontend/seo-dashboard/src/App.jsx): React SPA · `frontend/seo-dashboard/src/App.jsx`
- [Browser entry](https://github.com/technology-ai-cogni/seo-backend/blob/main/frontend/seo-dashboard/src/main.jsx): Vite entry · `frontend/seo-dashboard/src/main.jsx`
- [SEO analysis views](https://github.com/technology-ai-cogni/seo-backend/blob/main/frontend/seo-dashboard/src/components/pages/KeywordsPage.jsx): feature pages · Keywords, positions, and content planning views. · `frontend/seo-dashboard/src/components/pages/KeywordsPage.jsx`
- [Projects API client](https://github.com/technology-ai-cogni/seo-backend/blob/main/frontend/seo-dashboard/src/lib/projectsApi.js): client data access · Abstracts dashboard project access. · `frontend/seo-dashboard/src/lib/projectsApi.js`
- [CSV and mock data](https://github.com/technology-ai-cogni/seo-backend/blob/main/frontend/seo-dashboard/src/data/csvParser.js): demo data path · Local parsing and fallback dashboard datasets. · `frontend/seo-dashboard/src/data/csvParser.js`
- [Supabase](https://github.com/technology-ai-cogni/seo-backend/blob/main/frontend/seo-dashboard/src/lib/supabaseClient.js): client backend · Client-side project-data integration. · `frontend/seo-dashboard/src/lib/supabaseClient.js`

### Other components

- **Search data providers**: external service · Bright Data and hosted SEO-data providers.

## Connections

- Browser entry → Dashboard application: mounts
- Dashboard application → SEO analysis views: renders
- Dashboard application → Projects API client: loads projects
- Dashboard application → CSV and mock data: demo/import path
- Projects API client → Supabase: reads and writes
- Projects API client → Python API: SEO operations
- Python API → Application database: persists analysis state
- Python API → Rank checker: checks rankings
- Python API → Category checker: checks categories
- Rank checker → Search data providers: retrieves ranking data
- Category checker → Search data providers: retrieves category evidence
- Keyword CSV artifacts → Standard pipeline: keyword input
- Standard pipeline → SERP and ranking collection: enriches search data
- SERP and ranking collection → Search data providers: collects SERPs
- Standard pipeline → Category assignment: assigns intent and categories
- Category assignment → Keyword CSV artifacts: writes enriched CSV
- Bright Data pipeline → Hosted SEO execution: uses hosted collection
- Hosted SEO execution → Search data providers: calls providers
- Bright Data pipeline → Keyword CSV artifacts: writes normalized output
- Keyword CSV artifacts → Category experiment: experimental input
- Category experiment → SERP and metadata fetch: collects evidence
- SERP and metadata fetch → Classify and cluster: supplies SERP metadata
- Classify and cluster → Keyword CSV artifacts: candidate assignments
- Keyword CSV artifacts → Python API: downstream analysis data