# twangodev/uw-coursemap architecture

> Architecture diagram of the GitHub repository twangodev/uw-coursemap, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/twangodev/uw-coursemap
- Repository: https://github.com/twangodev/uw-coursemap

## Overview

##### Project Type and Purpose
- This is a full-stack, data-driven web application (“uw-coursemap”) for browsing UW–Madison courses, visualizing prerequisite graphs, and searching professor ratings.
- It consists of three logical subsystems:
– A Python-based data-generation pipeline that scrapes and aggregates raw data into static JSON.
– A Python “search” microservice (with FastAPI or Flask) backed by Elasticsearch for full-text and filtered course searches.
– A SvelteKit/Node.js frontend that renders pages, fetches static JSON, and calls the search API.

##### File Structure
- Top-level orchestration:
– docker-compose.yml + Dockerfile: Compose ties together the SvelteKit app, the search service, and Elasticsearch.
– .github/workflows: CI/CD pipelines build and deploy Docker images, regenerate docs, run tests.
- generation/ directory:
– Python modules (aggregate.py, webscrape.py, etc.) that load source data (UW API, OSM geojson), process it, and emit static JSON into data/.
– main.py coordinates the pipeline.
- search/ directory:
– A Python-based REST API (app.py) that indexes JSON into Elasticsearch (via es_util.py) and exposes search endpoints.
– Dockerfile and docker-compose.yml allow this service to run in isolation (with an ES container).
- src/ directory:
– SvelteKit frontend:
- /routes: page definitions and load functions (+page.ts) that read from data/ or call the search API.
- /lib: reusable UI components (many .svelte files), API wrappers, types, and utilities.
- data/: generated JSON files consumed by the frontend and indexed by the search service.

##### README for Additional Insights
- Confirms core features (graph visualization, course details, search).
- Highlights use of OpenAPI for public endpoints (static-openapi.yaml).

##### Main Components
a. Data Generation Pipeline (Python)
– Scraper & loaders → aggregation → JSON output (data/ directory)
b. Search Microservice (Python + Elasticsearch)
– Reads JSON → indexes into ES → exposes RESTful search endpoints
c. Frontend Web App (SvelteKit / Node.js)
– Static-page generation & SSR → serves UI components → reads static JSON and calls search API
d. Infrastructure & Deployment
– Docker Compose orchestrator
– GitHub Actions CI/CD

##### Relationships and Interactions
- Data Pipeline → produces JSON → placed into shared volume or artifact store
- Search Service → ingests JSON (at container startup or via manual trigger) → stores in Elasticsearch → API endpoints available
- Frontend →
– On build/SSR, reads static JSON files (prereqs, courses, instructors)
– At runtime, issues HTTP calls to Search Service’s REST API
- Docker Compose → wires together the three containers + Elasticsearch
- GitHub Actions → automates testing, building, versioning, and pushing images

##### Architectural Patterns and Design Principles
- Microservices: separate data-gen, search, and frontend concerns
- Component-based UI: Svelte components in /src/lib/components
- Static-site & SSR hybrid: pre-generated JSON for fast initial load + dynamic search API
- Containerization: Docker for consistent environments

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Generation Pipeline
    subgraph "Data Generation Pipeline"
        direction TB
        gen_dir["generation/"]:::python
        gen_main["main.py"]:::python
        gen_agg["aggregate.py"]:::python
        gen_ws["webscrape.py"]:::python
    end
    click gen_dir "https://github.com/twangodev/uw-coursemap/tree/main/generation/"
    click gen_main "https://github.com/twangodev/uw-coursemap/blob/main/generation/main.py"
    click gen_agg "https://github.com/twangodev/uw-coursemap/blob/main/generation/aggregate.py"
    click gen_ws "https://github.com/twangodev/uw-coursemap/blob/main/generation/webscrape.py"

    %% Data Store
    subgraph "Data Store"
        direction TB
        datastore["Static JSON<br/>(data/)"]:::data
    end
    click datastore "https://github.com/twangodev/uw-coursemap/tree/main/data/"

    %% Search Microservice
    subgraph "Search Microservice"
        direction TB
        search_dir["search/"]:::python
        app_py["app.py"]:::python
        es_util["es_util.py"]:::python
        data_py["data.py"]:::python
        search_dockerfile["Dockerfile"]:::infra
        search_comp["docker-compose.yml"]:::infra
    end
    click search_dir "https://github.com/twangodev/uw-coursemap/tree/main/search/"
    click app_py "https://github.com/twangodev/uw-coursemap/blob/main/search/app.py"
    click es_util "https://github.com/twangodev/uw-coursemap/blob/main/search/es_util.py"
    click data_py "https://github.com/twangodev/uw-coursemap/blob/main/search/data.py"
    click search_dockerfile "https://github.com/twangodev/uw-coursemap/tree/main/search/Dockerfile"
    click search_comp "https://github.com/twangodev/uw-coursemap/blob/main/search/docker-compose.yml"

    %% Elasticsearch
    elasticsearch["Elasticsearch"]:::elasticsearch

    %% Frontend Web App
    subgraph "Frontend Web App"
        direction TB
        src_dir["src/"]:::frontend
        routes["src/routes/"]:::frontend
        lib["src/lib/"]:::frontend
        pkgjson["package.json"]:::infra
        sveltecfg["svelte.config.js"]:::infra
        tailwind["tailwind.config.ts"]:::infra
        tsconfig["tsconfig.json"]:::infra
        vitecfg["vite.config.ts"]:::infra
    end
    click src_dir "https://github.com/twangodev/uw-coursemap/tree/main/src/"
    click routes "https://github.com/twangodev/uw-coursemap/tree/main/src/routes/"
    click lib "https://github.com/twangodev/uw-coursemap/tree/main/src/lib/"
    click pkgjson "https://github.com/twangodev/uw-coursemap/blob/main/package.json"
    click sveltecfg "https://github.com/twangodev/uw-coursemap/blob/main/svelte.config.js"
    click tailwind "https://github.com/twangodev/uw-coursemap/blob/main/tailwind.config.ts"
    click tsconfig "https://github.com/twangodev/uw-coursemap/blob/main/tsconfig.json"
    click vitecfg "https://github.com/twangodev/uw-coursemap/blob/main/vite.config.ts"

    %% Infrastructure & Orchestration
    subgraph "Infrastructure & Orchestration"
        direction TB
        root_dc["docker-compose.yml"]:::infra
        root_df["Dockerfile"]:::infra
    end
    click root_dc "https://github.com/twangodev/uw-coursemap/blob/main/docker-compose.yml"
    click root_df "https://github.com/twangodev/uw-coursemap/tree/main/Dockerfile"

    %% CI/CD
    subgraph "CI/CD (GitHub Actions)"
        direction TB
        workflows[".github/workflows/"]:::infra
        wf1["docker.yml"]:::infra
        wf2["generation.yml"]:::infra
        wf3["node.js.yml"]:::infra
        wf4["docs.yml"]:::infra
    end
    click workflows "https://github.com/twangodev/uw-coursemap/tree/main/.github/workflows/"
    click wf1 "https://github.com/twangodev/uw-coursemap/blob/main/.github/workflows/docker.yml"
    click wf2 "https://github.com/twangodev/uw-coursemap/blob/main/.github/workflows/generation.yml"
    click wf3 "https://github.com/twangodev/uw-coursemap/blob/main/.github/workflows/node.js.yml"
    click wf4 "https://github.com/twangodev/uw-coursemap/blob/main/.github/workflows/docs.yml"

    docs["static-openapi.yaml"]:::infra
    click docs "https://github.com/twangodev/uw-coursemap/blob/main/docs/public/static-openapi.yaml"

    %% Connections
    gen_dir -->|"writes JSON to"| datastore
    gen_dir -->|"triggers indexing on"| search_dir
    search_dir -->|"indexes & queries"| elasticsearch
    datastore -->|"read on build/SSR"| src_dir
    src_dir -->|"HTTP search requests"| search_dir
    root_dc --> gen_dir
    root_dc --> search_dir
    root_dc --> src_dir
    root_dc --> elasticsearch
    workflows --> root_dc

    %% Styles
    classDef python fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef data fill:#F0E68C,stroke:#333,stroke-width:1px
    classDef elasticsearch fill:#FFD700,stroke:#333,stroke-width:1px
    classDef frontend fill:#90EE90,stroke:#333,stroke-width:1px
    classDef infra fill:#D3D3D3,stroke:#333,stroke-width:1px
```
