# kenkoooo/atcoderproblems architecture

> Architecture diagram of the GitHub repository kenkoooo/atcoderproblems, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kenkoooo/atcoderproblems
- Repository: https://github.com/kenkoooo/atcoderproblems

## Overview

##### Project type and purpose
- This is a full‑stack web application called “AtCoder Problems” composed of:
- A React/TypeScript front‑end (under atcoder‑problems‑frontend/)
- A Rust‑based back‑end server (under atcoder‑problems‑backend/)
- Serverless pieces (AWS Lambda functions under lambda‑functions/)
- Supporting assets (database schema under config/, documentation under doc/ and guide/)
- Its purpose is to let AtCoder users view problem lists, track submissions, see rankings, manage virtual contests, and more.

##### File structure
- Top‑level directories:
- atcoder‑problems‑frontend: static UI, API clients, local IndexedDB
- atcoder‑problems‑backend: crawler binaries, server endpoints, SQL‑client crate, atcoder‑client crate
- lambda‑functions: official‑api‑proxy and time‑estimator scripts
- config: database table definitions
- doc/guide: user and developer documentation
- Backend patterns:
- A set of command‑line binaries (in src/bin) for batch crawling and updates
- A layered server: endpoint → services → SQL‑client → database
- Use of AWS S3 (s3.rs) for storing JSON snapshots
- Middleware for GitHub OAuth
- Front‑end patterns:
- React components in src/components and src/pages
- Two API clients: APIClient (public) and InternalAPIClient (authenticated)
- Cypress for E2E tests
- Local cache in IndexedDB (SubmissionsDB.ts) and static_data for initial boot

##### README for technical stack
- Front‑end: React, TypeScript, Yarn, Cypress
- Back‑end: Rust (Cargo.toml), Dockerized (Dockerfile & docker-compose.yml), Postgres via SQL‑client crate
- Lambda: Python scripts, build.sh for packaging, invoked via API Gateway
- External dependencies: AtCoder official API (proxied), GitHub OAuth, AWS S3

##### Main components
a. Front‑end (Browser/React app)
b. API Gateway(s) or reverse‑proxy endpoints
c. Back‑end server (Rust)
d. PostgreSQL database
e. AWS S3 bucket for JSON snapshots
f. Batch crawler services (Docker containers or cron jobs)
g. AWS Lambda functions:
- official‑api‑proxy (Python)
- time‑estimator (Python)
h. External services: AtCoder official API, GitHub OAuth

##### Relationships and data flow
- Browser loads static HTML/CSS/JS from CDN or front‑end host
- Browser → public API endpoints → back‑end server
- Browser → internal API endpoints (authenticated via GitHub OAuth middleware) → back‑end
- Back‑end server → SQL‑client → PostgreSQL (queries for user/submission data)
- Back‑end server → S3 (read/write JSON snapshots)
- Front‑end calls Lambda functions via AWS API Gateway for:
- official API data (to bypass CORS)
- time estimation service
- Batch crawler binaries run on schedule in containers → use atcoder‑client library to scrape official AtCoder → update DB and push snapshots to S3

##### Architectural patterns and design principles
- Layered architecture in back‑end: CLI crawlers, HTTP server, service layer, data access layer
- Separation of concerns: static UI vs. dynamic APIs vs. crawler jobs vs. serverless functions
- Use of serverless for edge‑needed functionality (proxied API, compute‑on‑demand)
- Strong typing and modular crates in Rust (atcoder‑client, sql‑client)
- Client‑side state and cache (IndexedDB) to reduce server load

##### Technologies/Frameworks to annotate
- Front‑end: React, TypeScript, Yarn, Cypress
- Back‑end: Rust, tokio/Actix/Rocket (as chosen), Diesel or sql‑client
- Containerization: Docker, docker‑compose
- Database: PostgreSQL
- Cloud: AWS S3, Lambda, API Gateway
- Auth: GitHub OAuth

- Clear, descriptive labels on every component (e.g. “Rust HTTP Server”)
- Arrowed lines for request/response and data flows
- Front‑end vs. back‑end vs. serverless vs. storage vs. external
- Notes on batch jobs vs. real‑time services
- Authentication flow via GitHub OAuth middleware

##### Details
- Crawler binaries as a separate service converging into the same DB/S3
- Expose that atcoder‑client crate is used by crawlers and sql‑client crate by both services and tests
- Internal vs. public API endpoints in the server (e.g. internal_api/* vs. healthcheck)
- Static_data embed vs. dynamic fetch

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Client Layer
    subgraph "Client Layer"
        direction TB
        Browser["Browser / React App"]:::frontend
        APIClient["Public API Client"]:::frontend
        InternalAPIClient["Internal API Client"]:::frontend
        IndexedDB["Local IndexedDB Cache"]:::storage
        StaticData["Static Boot Data"]:::storage
        Cypress["E2E Tests (Cypress)"]:::frontend
    end

    %% Edge/API Layer
    subgraph "Edge/API Layer"
        direction TB
        APIGW["AWS API Gateway"]:::cloud
        StaticHost["Static Hosting (CDN)"]:::cloud
    end

    %% Application Layer
    subgraph "Application Layer"
        direction TB
        RustServer["Rust HTTP Server"]:::backend
        OAuthMW["GitHub OAuth Middleware"]:::backend
        Services["Service Layer"]:::backend
        SQLClientCrate["SQL-Client Crate"]:::backend
        AtCoderClientCrate["AtCoder API Client Crate"]:::backend
        S3Module["AWS S3 Integration Module"]:::backend
        RunServer["Server Binary Entrypoint"]:::backend
        CrawlerBinaries["Batch Crawler Binaries"]:::backend
        CrawlerModules["Crawler Modules"]:::backend
    end

    %% Serverless Layer
    subgraph "Serverless Layer"
        direction TB
        LambdaProxy["Lambda: Official API Proxy"]:::lambda
        LambdaEstimator["Lambda: Time Estimator"]:::lambda
    end

    %% Data Layer
    subgraph "Data Layer"
        direction TB
        Postgres["PostgreSQL Database"]:::storage
        S3Bucket["AWS S3 Bucket"]:::cloud
    end

    %% External Services
    subgraph "External Services"
        direction TB
        AtCoderAPI["AtCoder Official API"]:::external
        GitHubOAuthService["GitHub OAuth Service"]:::external
    end

    %% Connections: Client to Edge
    Browser -->|"loads UI"| StaticHost
    Browser -->|"HTTP GET/POST"| APIGW
    Browser --> APIClient
    Browser --> InternalAPIClient
    Browser --> IndexedDB
    Browser --> StaticData
    Browser --> Cypress

    %% Edge to Application
    APIGW -->|"public endpoints"| RustServer
    APIGW -->|"invoke"| LambdaProxy
    APIGW -->|"invoke"| LambdaEstimator

    StaticHost --- Browser

    %% Application internal
    RustServer --> OAuthMW
    OAuthMW --> Services
    Services --> SQLClientCrate
    Services --> S3Module
    Services --> AtCoderClientCrate
    RustServer --> RunServer
    RunServer --> Services

    CrawlerBinaries --> CrawlerModules
    CrawlerModules --> AtCoderClientCrate
    CrawlerBinaries --> SQLClientCrate
    CrawlerBinaries --> S3Module

    %% Serverless to external
    LambdaProxy --> AtCoderAPI
    LambdaEstimator --> Services

    %% App to Data
    SQLClientCrate -->|"SQL queries"| Postgres
    S3Module -->|"PUT/GET JSON"| S3Bucket

    CrawlerBinaries --> Postgres
    CrawlerBinaries --> S3Bucket

    %% External auth
    OAuthMW --> GitHubOAuthService

    %% Click Events
    click atcoder-problems-frontend/ "https://github.com/kenkoooo/atcoderproblems/tree/master/atcoder-problems-frontend/"
    click APIClient "https://github.com/kenkoooo/atcoderproblems/blob/master/atcoder-problems-frontend/src/api/APIClient.ts"
    click InternalAPIClient "https://github.com/kenkoooo/atcoderproblems/blob/master/atcoder-problems-frontend/src/api/InternalAPIClient.ts"
    click IndexedDB "https://github.com/kenkoooo/atcoderproblems/blob/master/atcoder-problems-frontend/src/database/SubmissionsDB.ts"
    click StaticData "https://github.com/kenkoooo/atcoderproblems/tree/master/atcoder-problems-frontend/public/static_data/"
    click Cypress "https://github.com/kenkoooo/atcoderproblems/tree/master/atcoder-problems-frontend/cypress/"
    click RustServer "https://github.com/kenkoooo/atcoderproblems/tree/master/atcoder-problems-backend/src/server/"
    click RunServer "https://github.com/kenkoooo/atcoderproblems/blob/master/atcoder-problems-backend/src/bin/run_server.rs"
    click OAuthMW "https://github.com/kenkoooo/atcoderproblems/blob/master/atcoder-problems-backend/src/server/middleware/github_auth.rs"
    click Services "https://github.com/kenkoooo/atcoderproblems/blob/master/atcoder-problems-backend/src/server/services.rs"
    click SQLClientCrate "https://github.com/kenkoooo/atcoderproblems/tree/master/atcoder-problems-backend/sql-client/"
    click AtCoderClientCrate "https://github.com/kenkoooo/atcoderproblems/tree/master/atcoder-problems-backend/atcoder-client/"
    click S3Module "https://github.com/kenkoooo/atcoderproblems/blob/master/atcoder-problems-backend/src/s3.rs"
    click CrawlerBinaries "https://github.com/kenkoooo/atcoderproblems/tree/master/atcoder-problems-backend/src/bin/"
    click CrawlerModules "https://github.com/kenkoooo/atcoderproblems/tree/master/atcoder-problems-backend/src/crawler/"
    click Postgres "https://github.com/kenkoooo/atcoderproblems/blob/master/config/database-definition.sql"
    click LambdaProxy "https://github.com/kenkoooo/atcoderproblems/tree/master/lambda-functions/official-api-proxy/"
    click LambdaEstimator "https://github.com/kenkoooo/atcoderproblems/tree/master/lambda-functions/time-estimator/"

    %% Styles
    classDef frontend fill:#D0E8FF,stroke:#007ACC;
    classDef backend fill:#DFF2BF,stroke:#4F8A10;
    classDef lambda fill:#FFF2CC,stroke:#B26A00;
    classDef storage fill:#F8D7DA,stroke:#721C24;
    classDef cloud fill:#E0E0E0,stroke:#888888,stroke-dasharray: 5 5;
    classDef external fill:#EEEEEE,stroke:#999999;
```
