How swiftpackageindex/swiftpackageindex-server works
Project Type & Purpose
• Full-stack web application: A site for discovering and evaluating Swift Package Manager packages.
• Backend written in Swift using Vapor (MVC pattern, controllers, models, services).
• Frontend: static HTML/CSS/JS assets (built with esbuild and Sass) enhancing site interactivity.
• Integrates with external services (GitHub, GitLab, Plausible analytics), uses object storage (S3), a relational database (Postgres), and a cache (Redis).
File Structure Overview
• FrontEnd/: source assets (JavaScript controllers, SCSS styles, build script).
• Public/: compiled assets (images, robots.txt).
• Sources/App/: Vapor application (Commands, Controllers/API & Site, Core services, Models, Views, Migrations).
• S3Store/: wrapper for S3 interactions.
• Scripts/: helper scripts for ingestion loops, CI DB setup, emoji updates.
• infrastructure/: Prometheus/Loki/Promtail configs for monitoring/log aggregation.
• CI/CD configs: GitHub Actions (.github/workflows), GitLab CI, Dockerfile, devcontainer.
a. Frontend Layer
– Browser clients request pages and API endpoints.
– Static assets served via a CDN or directly by the Vapor app.
– JavaScript controllers manage dynamic UI (search, charts, modals).
b. API/Web Layer
– Vapor HTTP server (entrypoint → routes → controllers).
– Controllers orchestrate requests:
• API controllers (JSON endpoints for packages, search, collections).
• Site controllers (render HTML via Leaf-like Views).
c. Core Services Layer
– Authentication, Metrics, Logging, Error Handling Middleware.
– Dependency‐injected clients: HTTPClient (for GitHub/GitLab), RedisClient, S3Client, ShellClient.
– Background commands (ingestion, build triggers, reconciliation) executed via CLI commands or scheduled loops.
d. Data Layer
– PostgreSQL database (persistent storage for packages, builds, versions, migrations).
– Redis (caching, rate‐limiting, reference lookups).
– S3 Object Storage (readme HTML, doc archives).
e. External Integrations
– GitHub/GitLab APIs for repository metadata & build logs.
– Plausible analytics for event tracking.
– Package Collections sign/verify process.
f. Infrastructure & Observability
– Containerization (Docker, devcontainer).
– CI/CD pipelines (GitHub Actions, GitLab CI).
– Monitoring: Prometheus (metrics), Loki/Promtail (logs).
Architectural Patterns & Design Principles
• MVC: Controllers (entrypoints) → Models (data) → Views (HTML templates).
• Dependency Injection for pluggable clients (e.g., RedisClient).
• Command Pattern: CLI Commands encapsulate background tasks.
• Layered Architecture: clear separation between web/API, business logic, persistence.
• Eventual consistency: background ingestion and reconciliation ensure up‐to‐date data.