# luisvilriv/comprobador-de-paginas-web architecture

> Architecture diagram of the GitHub repository luisvilriv/comprobador-de-paginas-web, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/luisvilriv/comprobador-de-paginas-web
- Repository: https://github.com/luisvilriv/comprobador-de-paginas-web

## Overview

1. This repo is a containerized web-auditing platform split into three runtime pieces: a browser frontend, a FastAPI dashboard backend, and a separate Python scraper/auditor worker. The shared code under `shared/` is the main boundary that keeps the API and worker consistent.

2. The user-facing entry point is `docker/dashboard/frontend/`, served by `docker/dashboard/server.js`. The frontend is a modular JavaScript app (`api.js`, `websites.js`, `scheduler.js`, `audit.js`, `modals.js`, `i18n.js`) that talks only to the dashboard API, not directly to the database or scraper.

3. The dashboard backend lives in `docker/dashboard/api/` and is organized by domain routes: `clients`, `websites`, `runs`, `settings`, `summary`, plus `health`. This API is the orchestration layer for the UI: it exposes CRUD and reporting endpoints, validates payloads with Pydantic schemas, and delegates persistence to the shared database repositories.

4. The scraper side is a separate execution service under `scraper/` and `docker/scraper/`. It is structured as a pipeline: a context selects a scraping strategy, a service layer coordinates the run, and the result is stored through shared repository code. The Docker entrypoints (`entrypoint.py`, `scheduler.py`, `service.py`) indicate it can run either on demand or as a scheduled worker.

5. Scraping is strategy-based. `scraper/strategies/beautifulsoup_strategy.py` and `selenium_strategy.py` represent two crawl modes, with `base/` defining the abstraction and `context/` selecting the implementation. This separation is important because the architecture supports both lightweight HTML parsing and browser-driven execution.

6. The audit engine is centralized in `shared/auditor/`. It is not tied to the scraper implementation; instead, it consumes scraped page data and runs categorized checks in `checks/` such as browser, buttons, content, images, links, security, SEO, structure, and technical. Scoring and rule dictionaries live alongside the checks, so the audit logic is reusable across services.

7. Persistence is also shared. `shared/database/connection.py` and `shared/database/models.py` define the database layer, while repository packages are split by consumer: `repositories/dashboard/` for API-facing queries and `repositories/scraper/` for worker writes/reads. This keeps DB access behind a repository facade and prevents the dashboard and scraper from depending on raw SQLAlchemy internals.

8. The database is PostgreSQL, initialized by `docker/db-init/01_schema.sql` and optionally migrated by `02_migrate_optional_client.sql`. That means schema creation happens as part of the container startup flow, not through application bootstrapping.

9. There is an optional AI analysis service under `docker/ai-analyzer/`, with its own app, analyzer, and requirements. The presence of dedicated tests (`test_ai_analyzer_*`) suggests it enriches or post-processes audit results rather than replacing the core crawler/auditor pipeline.

10. Configuration is centralized in `config/settings.py` and logging in `config/logging_config.py`. The repo also relies heavily on environment-driven deployment via `docker/.env.example`, which matches the container-first design.

11. Report/export support exists but is secondary to the main flow. `shared/utils/pdf_generator.py` suggests the system can generate downloadable artifacts from audit data, likely consumed by the dashboard summary/report views.

12. The dominant data flow is: frontend request -> dashboard API -> shared repositories -> PostgreSQL for management data, and scheduler/worker run -> scrape pages -> audit checks -> persist results -> dashboard reads summaries/runs back from the same database. That shared database is the integration point between all major subsystems.

13. The test suite mirrors the architecture: endpoint tests for the API, unit tests for individual audit checks, Selenium/strategy tests for scraping behavior, and integration tests for the AI analyzer. This is a good sign that the repo is organized around boundaries that are meant to be independently verifiable.

Main entry points: [`docker/dashboard/api/routes`](https://github.com/luisvilriv/comprobador-de-paginas-web/tree/main/docker/dashboard/api/routes) (Domain Routes), [`docker/scraper/service.py`](https://github.com/luisvilriv/comprobador-de-paginas-web/blob/main/docker/scraper/service.py) (Run Service), [`shared/auditor`](https://github.com/luisvilriv/comprobador-de-paginas-web/tree/main/shared/auditor) (Auditor Core), [`config`](https://github.com/luisvilriv/comprobador-de-paginas-web/tree/main/config) (Config)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_frontend["Frontend"]
  node_frontend_app["Dashboard UI<br/>SPA"]
  node_dashboard_server["Server<br/>Node host<br/>[server.js]"]
end

subgraph group_dashboard["Dashboard API"]
  node_api_app["API App<br/>FastAPI<br/>[app.py]"]
  node_api_routes["Domain Routes<br/>HTTP routes"]
  node_api_schemas["Schemas<br/>Pydantic"]
end

subgraph group_scraper["Scraper Worker"]
  node_scraper_entry["Worker Entry<br/>Python service"]
  node_scraper_service["Run Service<br/>orchestrator<br/>[service.py]"]
  node_scraper_context["Scraper Context<br/>strategy selector<br/>[scraper_context.py]"]
  node_scraper_strategies["Strategies<br/>crawl modes"]
end

subgraph group_shared["Shared Core"]
  node_auditor_core["Auditor Core<br/>audit engine"]
  node_audit_checks["Checks<br/>rule packs"]
  node_db_connection["DB Layer<br/>SQLAlchemy"]
  node_repo_dashboard["Dashboard Repos<br/>read/write repos"]
  node_repo_scraper["Scraper Repos<br/>worker repos"]
  node_pdf_export["PDF Export<br/>reporting util<br/>[pdf_generator.py]"]
end

subgraph group_infra["Infrastructure"]
  node_postgres_init[("PostgreSQL<br/>schema db<br/>[01_schema.sql]")]
  node_ai_analyzer["AI Analyzer<br/>post-processor"]
  node_config["Config<br/>settings"]
end

node_dashboard_server -->|"serves"| node_frontend_app
node_frontend_app -->|"calls"| node_api_app
node_api_app -->|"mounts"| node_api_routes
node_api_routes -->|"validates"| node_api_schemas
node_api_routes -->|"reads/writes"| node_repo_dashboard
node_repo_dashboard -->|"uses"| node_db_connection
node_scraper_entry -->|"starts"| node_scraper_service
node_scraper_service -->|"selects"| node_scraper_context
node_scraper_context -->|"dispatches"| node_scraper_strategies
node_scraper_service -->|"audits"| node_auditor_core
node_auditor_core -->|"runs"| node_audit_checks
node_auditor_core -->|"persists"| node_repo_scraper
node_repo_scraper -->|"uses"| node_db_connection
node_db_connection -->|"backed by"| node_postgres_init
node_api_routes -->|"reports"| node_pdf_export
node_ai_analyzer -->|"consumes"| node_repo_scraper
node_config -->|"configures"| node_api_app
node_config -->|"configures"| node_scraper_service
node_config -->|"configures"| node_auditor_core

click node_frontend_app "https://github.com/luisvilriv/comprobador-de-paginas-web/tree/main/docker/dashboard/frontend"
click node_dashboard_server "https://github.com/luisvilriv/comprobador-de-paginas-web/blob/main/docker/dashboard/server.js"
click node_api_app "https://github.com/luisvilriv/comprobador-de-paginas-web/blob/main/docker/dashboard/api/app.py"
click node_api_routes "https://github.com/luisvilriv/comprobador-de-paginas-web/tree/main/docker/dashboard/api/routes"
click node_api_schemas "https://github.com/luisvilriv/comprobador-de-paginas-web/tree/main/docker/dashboard/api/schemas"
click node_scraper_entry "https://github.com/luisvilriv/comprobador-de-paginas-web/tree/main/docker/scraper"
click node_scraper_service "https://github.com/luisvilriv/comprobador-de-paginas-web/blob/main/docker/scraper/service.py"
click node_scraper_context "https://github.com/luisvilriv/comprobador-de-paginas-web/blob/main/scraper/context/scraper_context.py"
click node_scraper_strategies "https://github.com/luisvilriv/comprobador-de-paginas-web/tree/main/scraper/strategies"
click node_auditor_core "https://github.com/luisvilriv/comprobador-de-paginas-web/tree/main/shared/auditor"
click node_audit_checks "https://github.com/luisvilriv/comprobador-de-paginas-web/tree/main/shared/auditor/checks"
click node_db_connection "https://github.com/luisvilriv/comprobador-de-paginas-web/tree/main/shared/database"
click node_repo_dashboard "https://github.com/luisvilriv/comprobador-de-paginas-web/tree/main/shared/database/repositories/dashboard"
click node_repo_scraper "https://github.com/luisvilriv/comprobador-de-paginas-web/tree/main/shared/database/repositories/scraper"
click node_postgres_init "https://github.com/luisvilriv/comprobador-de-paginas-web/blob/main/docker/db-init/01_schema.sql"
click node_ai_analyzer "https://github.com/luisvilriv/comprobador-de-paginas-web/tree/main/docker/ai-analyzer"
click node_pdf_export "https://github.com/luisvilriv/comprobador-de-paginas-web/blob/main/shared/utils/pdf_generator.py"
click node_config "https://github.com/luisvilriv/comprobador-de-paginas-web/tree/main/config"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_frontend_app,node_dashboard_server toneBlue
class node_api_app,node_api_routes,node_api_schemas toneAmber
class node_scraper_entry,node_scraper_service,node_scraper_context,node_scraper_strategies toneMint
class node_auditor_core,node_audit_checks,node_db_connection,node_repo_dashboard,node_repo_scraper,node_pdf_export toneRose
class node_postgres_init,node_ai_analyzer,node_config toneIndigo
```

## Components

### Frontend

Browser UI for dashboard users.

- [Dashboard UI](https://github.com/luisvilriv/comprobador-de-paginas-web/tree/main/docker/dashboard/frontend): SPA · User-facing modular JavaScript frontend. · `docker/dashboard/frontend`
- [Server](https://github.com/luisvilriv/comprobador-de-paginas-web/blob/main/docker/dashboard/server.js): Node host · Serves the dashboard frontend and connects browser requests to the API. · `docker/dashboard/server.js`

### Dashboard API

FastAPI orchestration layer for client, website, run, and summary operations.

- [API App](https://github.com/luisvilriv/comprobador-de-paginas-web/blob/main/docker/dashboard/api/app.py): FastAPI · Dashboard backend application entrypoint. · `docker/dashboard/api/app.py`
- [Domain Routes](https://github.com/luisvilriv/comprobador-de-paginas-web/tree/main/docker/dashboard/api/routes): HTTP routes · Client, website, run, settings, summary, and health endpoints. · `docker/dashboard/api/routes`
- [Schemas](https://github.com/luisvilriv/comprobador-de-paginas-web/tree/main/docker/dashboard/api/schemas): Pydantic · Request and response validation models for dashboard endpoints. · `docker/dashboard/api/schemas`

### Scraper Worker

Separate scraping and auditing execution service.

- [Worker Entry](https://github.com/luisvilriv/comprobador-de-paginas-web/tree/main/docker/scraper): Python service · Scraper container entrypoints for scheduled or on-demand runs. · `docker/scraper`
- [Run Service](https://github.com/luisvilriv/comprobador-de-paginas-web/blob/main/docker/scraper/service.py): orchestrator · Coordinates scraping execution and persistence. · `docker/scraper/service.py`
- [Scraper Context](https://github.com/luisvilriv/comprobador-de-paginas-web/blob/main/scraper/context/scraper_context.py): strategy selector · Chooses the active scraping strategy. · `scraper/context/scraper_context.py`
- [Strategies](https://github.com/luisvilriv/comprobador-de-paginas-web/tree/main/scraper/strategies): crawl modes · BeautifulSoup and Selenium scraping implementations behind a common abstraction. · `scraper/strategies`

### Shared Core

Common audit, database, and utility code reused by services.

- [Auditor Core](https://github.com/luisvilriv/comprobador-de-paginas-web/tree/main/shared/auditor): audit engine · Runs reusable audit logic over scraped page data. · `shared/auditor`
- [Checks](https://github.com/luisvilriv/comprobador-de-paginas-web/tree/main/shared/auditor/checks): rule packs · Categorized audit checks for browser, content, links, security, SEO, structure, and technical issues. · `shared/auditor/checks`
- [DB Layer](https://github.com/luisvilriv/comprobador-de-paginas-web/tree/main/shared/database): SQLAlchemy · Shared database connection and models. · `shared/database`
- [Dashboard Repos](https://github.com/luisvilriv/comprobador-de-paginas-web/tree/main/shared/database/repositories/dashboard): read/write repos · Repository facade used by the API for dashboard-facing queries and mutations. · `shared/database/repositories/dashboard`
- [Scraper Repos](https://github.com/luisvilriv/comprobador-de-paginas-web/tree/main/shared/database/repositories/scraper): worker repos · Repository facade used by the worker to persist scrape and audit results. · `shared/database/repositories/scraper`
- [PDF Export](https://github.com/luisvilriv/comprobador-de-paginas-web/blob/main/shared/utils/pdf_generator.py): reporting util · Generates downloadable report artifacts from audit data. · `shared/utils/pdf_generator.py`

### Infrastructure

Container and database bootstrap pieces.

- [PostgreSQL](https://github.com/luisvilriv/comprobador-de-paginas-web/blob/main/docker/db-init/01_schema.sql): schema db · Database initialized by container SQL scripts. · `docker/db-init/01_schema.sql`
- [AI Analyzer](https://github.com/luisvilriv/comprobador-de-paginas-web/tree/main/docker/ai-analyzer): post-processor · Optional analysis service that enriches or post-processes audit results. · `docker/ai-analyzer`
- [Config](https://github.com/luisvilriv/comprobador-de-paginas-web/tree/main/config): settings · Centralized settings and logging configuration. · `config`

## Connections

- Server → Dashboard UI: serves · Node server delivers the browser UI.
- Dashboard UI → API App: calls · Frontend communicates only with the dashboard API.
- API App → Domain Routes: mounts · FastAPI app exposes domain route modules.
- Domain Routes → Schemas: validates · Endpoints use schemas for payload validation.
- Domain Routes → Dashboard Repos: reads/writes · API delegates persistence to dashboard repositories.
- Dashboard Repos → DB Layer: uses · Dashboard repositories use shared DB models and connection logic.
- Worker Entry → Run Service: starts · Container entrypoints launch the worker service.
- Run Service → Scraper Context: selects · Service chooses the active scraping strategy.
- Scraper Context → Strategies: dispatches · Context delegates to BeautifulSoup or Selenium implementations.
- Run Service → Auditor Core: audits · Scraped page data is passed into the shared auditor.
- Auditor Core → Checks: runs · Audit engine executes categorized rule sets.
- Auditor Core → Scraper Repos: persists · Audit and scrape results are stored through scraper repositories.
- Scraper Repos → DB Layer: uses · Worker repositories share the same database layer.
- DB Layer → PostgreSQL: backed by · Database layer targets the PostgreSQL schema initialized in containers.
- Domain Routes → PDF Export: reports · Summary endpoints can generate report artifacts.
- AI Analyzer → Scraper Repos: consumes · Optional analyzer works from stored scrape/audit data.
- Config → API App: configures · Settings influence API runtime behavior.
- Config → Run Service: configures · Settings influence worker runtime behavior.
- Config → Auditor Core: configures · Shared settings and logging shape audit execution.
