# thefpiasta/study-csh-olympics architecture

> Architecture diagram of the GitHub repository thefpiasta/study-csh-olympics, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/thefpiasta/study-csh-olympics
- Repository: https://github.com/thefpiasta/study-csh-olympics

## Overview

This repository documents a research workflow for collecting Olympic venue data, extracting structured records from reports, combining venue datasets, and presenting the results in an interactive web application. The main user path is through maps and analytics backed by processed GeoJSON. The graph also preserves the documented PDF-extraction and venue-matching branches. Several workflow links come from the README rather than inspected calls; unsampled application internals are represented by their existing directories.

Main entry points: [`webapp/src/app`](https://github.com/thefpiasta/study-csh-olympics/tree/main/webapp/src/app) (Interactive web app), [`webapp/src/components/logger.js`](https://github.com/thefpiasta/study-csh-olympics/blob/main/webapp/src/components/logger.js) (Application logger), [`geojson_scraper/00_final_geojsons`](https://github.com/thefpiasta/study-csh-olympics/tree/main/geojson_scraper/00_final_geojsons) (Processed GeoJSON), [`geojson_scraper/01_scraper.py`](https://github.com/thefpiasta/study-csh-olympics/blob/main/geojson_scraper/01_scraper.py) (Venue scraper)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_collection["Data collection"]
  node_scraper["Venue scraper<br/>[01_scraper.py]"]
  node_pdfextract["PDF extraction"]
end

subgraph group_processing["Data processing"]
  node_geojson["GeoJSON templating"]
  node_dedupe["Duplicate removal"]
  node_rename["Venue renaming<br/>[04_renamer.py]"]
  node_combine["Venue combining"]
  node_harvard["Harvard data combine"]
  node_matching["Venue matching<br/>[main.py]"]
  node_matcher["Fuzzy matcher<br/>[venues_matcher.py]"]
  node_processed[("Processed GeoJSON")]
end

subgraph group_web["Web application"]
  node_webapp["Interactive web app"]
  node_maps["Venue maps"]
  node_analytics["Charts and analytics"]
  node_olympics_api["Olympic data API"]
  node_cities_api["City lookup API<br/>[route.js]"]
end

subgraph group_operations["Operations"]
  node_logger["Application logger<br/>[logger.js]"]
  node_log_api["Log collection API<br/>[route.js]"]
  node_monitor["Backend monitor API<br/>[route.js]"]
end

node_researcher(("Researcher"))
node_olympedia(("Olympedia"))
node_reports["Olympic reports"]
node_visitor(("Web visitor"))

node_researcher -->|"initiates scraping"| node_scraper
node_olympedia -->|"provides venue data"| node_scraper
node_reports -->|"provides PDFs"| node_pdfextract
node_researcher -->|"runs extraction"| node_pdfextract
node_scraper -->|"supplies scraped data"| node_geojson
node_geojson -->|"creates GeoJSON"| node_dedupe
node_dedupe -->|"removes duplicates"| node_rename
node_rename -->|"adds descriptive names"| node_combine
node_combine -->|"writes final data"| node_processed
node_harvard -->|"combines external data"| node_processed
node_matching -->|"calls matcher"| node_matcher
node_visitor -->|"views"| node_webapp
node_webapp -->|"offers maps"| node_maps
node_webapp -->|"offers analytics"| node_analytics
node_webapp -->|"loads GeoJSON"| node_processed
node_olympics_api -->|"reads GeoJSON"| node_processed
node_cities_api -->|"logs activity"| node_logger
node_olympics_api -->|"logs errors"| node_logger
node_logger -->|"sends client logs"| node_log_api
node_monitor -->|"logs warnings"| node_logger

click node_scraper "https://github.com/thefpiasta/study-csh-olympics/blob/main/geojson_scraper/01_scraper.py"
click node_pdfextract "https://github.com/thefpiasta/study-csh-olympics/tree/main/pdfToJson"
click node_geojson "https://github.com/thefpiasta/study-csh-olympics/blob/main/geojson_scraper/02_geojson_templater.py"
click node_dedupe "https://github.com/thefpiasta/study-csh-olympics/blob/main/geojson_scraper/03_duplicate_finder.py"
click node_rename "https://github.com/thefpiasta/study-csh-olympics/blob/main/geojson_scraper/04_renamer.py"
click node_combine "https://github.com/thefpiasta/study-csh-olympics/blob/main/geojson_scraper/05_venue_combiner.py"
click node_harvard "https://github.com/thefpiasta/study-csh-olympics/blob/main/geojson_scraper/06_harvard_combiner.py"
click node_matching "https://github.com/thefpiasta/study-csh-olympics/blob/main/archive/matching-poc/main.py"
click node_matcher "https://github.com/thefpiasta/study-csh-olympics/blob/main/archive/matching-poc/venues_matcher.py"
click node_processed "https://github.com/thefpiasta/study-csh-olympics/tree/main/geojson_scraper/00_final_geojsons"
click node_webapp "https://github.com/thefpiasta/study-csh-olympics/tree/main/webapp/src/app"
click node_maps "https://github.com/thefpiasta/study-csh-olympics/tree/main/webapp/src/app/map"
click node_analytics "https://github.com/thefpiasta/study-csh-olympics/tree/main/webapp/src/app/graphs"
click node_olympics_api "https://github.com/thefpiasta/study-csh-olympics/tree/main/webapp/src/app/api/olympics"
click node_cities_api "https://github.com/thefpiasta/study-csh-olympics/blob/main/webapp/src/app/api/cities/route.js"
click node_logger "https://github.com/thefpiasta/study-csh-olympics/blob/main/webapp/src/components/logger.js"
click node_log_api "https://github.com/thefpiasta/study-csh-olympics/blob/main/webapp/src/app/api/logs/route.js"
click node_monitor "https://github.com/thefpiasta/study-csh-olympics/blob/main/webapp/src/app/api/backend/monitor/route.js"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_scraper,node_pdfextract toneBlue
class node_geojson,node_dedupe,node_rename,node_combine,node_harvard,node_matching,node_matcher,node_processed toneAmber
class node_webapp,node_maps,node_analytics,node_olympics_api,node_cities_api toneMint
class node_logger,node_log_api,node_monitor toneRose
class node_researcher,node_olympedia,node_reports,node_visitor toneIndigo
```

## Components

### Data collection

- [Venue scraper](https://github.com/thefpiasta/study-csh-olympics/blob/main/geojson_scraper/01_scraper.py): `geojson_scraper/01_scraper.py`
- [PDF extraction](https://github.com/thefpiasta/study-csh-olympics/tree/main/pdfToJson): `pdfToJson`

### Data processing

- [GeoJSON templating](https://github.com/thefpiasta/study-csh-olympics/blob/main/geojson_scraper/02_geojson_templater.py): `geojson_scraper/02_geojson_templater.py`
- [Duplicate removal](https://github.com/thefpiasta/study-csh-olympics/blob/main/geojson_scraper/03_duplicate_finder.py): `geojson_scraper/03_duplicate_finder.py`
- [Venue renaming](https://github.com/thefpiasta/study-csh-olympics/blob/main/geojson_scraper/04_renamer.py): `geojson_scraper/04_renamer.py`
- [Venue combining](https://github.com/thefpiasta/study-csh-olympics/blob/main/geojson_scraper/05_venue_combiner.py): `geojson_scraper/05_venue_combiner.py`
- [Harvard data combine](https://github.com/thefpiasta/study-csh-olympics/blob/main/geojson_scraper/06_harvard_combiner.py): `geojson_scraper/06_harvard_combiner.py`
- [Venue matching](https://github.com/thefpiasta/study-csh-olympics/blob/main/archive/matching-poc/main.py): `archive/matching-poc/main.py`
- [Fuzzy matcher](https://github.com/thefpiasta/study-csh-olympics/blob/main/archive/matching-poc/venues_matcher.py): `archive/matching-poc/venues_matcher.py`
- [Processed GeoJSON](https://github.com/thefpiasta/study-csh-olympics/tree/main/geojson_scraper/00_final_geojsons): `geojson_scraper/00_final_geojsons`

### Web application

- [Interactive web app](https://github.com/thefpiasta/study-csh-olympics/tree/main/webapp/src/app): `webapp/src/app`
- [Venue maps](https://github.com/thefpiasta/study-csh-olympics/tree/main/webapp/src/app/map): `webapp/src/app/map`
- [Charts and analytics](https://github.com/thefpiasta/study-csh-olympics/tree/main/webapp/src/app/graphs): `webapp/src/app/graphs`
- [Olympic data API](https://github.com/thefpiasta/study-csh-olympics/tree/main/webapp/src/app/api/olympics): `webapp/src/app/api/olympics`
- [City lookup API](https://github.com/thefpiasta/study-csh-olympics/blob/main/webapp/src/app/api/cities/route.js): `webapp/src/app/api/cities/route.js`

### Operations

- [Application logger](https://github.com/thefpiasta/study-csh-olympics/blob/main/webapp/src/components/logger.js): `webapp/src/components/logger.js`
- [Log collection API](https://github.com/thefpiasta/study-csh-olympics/blob/main/webapp/src/app/api/logs/route.js): `webapp/src/app/api/logs/route.js`
- [Backend monitor API](https://github.com/thefpiasta/study-csh-olympics/blob/main/webapp/src/app/api/backend/monitor/route.js): `webapp/src/app/api/backend/monitor/route.js`

### Other components

- **Researcher**
- **Olympedia**
- **Olympic reports**
- **Web visitor**

## Connections

- Researcher → Venue scraper: initiates scraping (evidence: `README.md`)
- Olympedia → Venue scraper: provides venue data (evidence: `README.md`)
- Olympic reports → PDF extraction: provides PDFs (evidence: `README.md`)
- Researcher → PDF extraction: runs extraction (evidence: `README.md`)
- Venue scraper → GeoJSON templating: supplies scraped data (evidence: `README.md`)
- GeoJSON templating → Duplicate removal: creates GeoJSON (evidence: `README.md`)
- Duplicate removal → Venue renaming: removes duplicates (evidence: `README.md`)
- Venue renaming → Venue combining: adds descriptive names (evidence: `README.md`)
- Venue combining → Processed GeoJSON: writes final data (evidence: `README.md`)
- Harvard data combine → Processed GeoJSON: combines external data (evidence: `README.md`)
- Venue matching → Fuzzy matcher: calls matcher (evidence: `archive/matching-poc/main.py`)
- Web visitor → Interactive web app: views (evidence: `README.md`)
- Interactive web app → Venue maps: offers maps (evidence: `README.md`)
- Interactive web app → Charts and analytics: offers analytics (evidence: `README.md`)
- Interactive web app → Processed GeoJSON: loads GeoJSON (evidence: `README.md`)
- Olympic data API → Processed GeoJSON: reads GeoJSON (evidence: `webapp/src/app/api/olympics/all/route.js`)
- City lookup API → Application logger: logs activity (evidence: `webapp/src/app/api/cities/route.js`)
- Olympic data API → Application logger: logs errors (evidence: `webapp/src/app/api/olympics/[slug]/route.js`)
- Application logger → Log collection API: sends client logs (evidence: `webapp/src/components/logger.js`)
- Backend monitor API → Application logger: logs warnings (evidence: `webapp/src/app/api/backend/monitor/route.js`)
