# mrkocmocc/osu.eventaggregator architecture

> Architecture diagram of the GitHub repository mrkocmocc/osu.eventaggregator, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-08-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mrkocmocc/osu.eventaggregator
- Repository: https://github.com/mrkocmocc/osu.eventaggregator

## Overview

#### Purpose and Entry Points
OSU.EventAggregator collects Orenburg State University event data from news and open-day pages, persists normalized events, and exposes them through a Flask REST API plus a lightweight browser UI. Process startup is `run.py`; application construction and extension registration are centered in `app/__init__.py`.

#### Application and HTTP Boundary
The Flask application serves API operations under `/api` and the HTML interface. API routes provide event listing/detail/upcoming queries, statistics, parser triggering, and JSON import/export. `app/api/routes.py` is the orchestration boundary between requests, parsers, ORM queries, and serialization; `app/templates/index.html` consumes the service as a management/viewing interface.

#### Canonical Event Persistence Model
`Event` is the system’s canonical normalized record: source-specific identifiers, event metadata, source provenance, parsing timestamps, active state, and audit timestamps. Uniqueness of `external_id` supports repeatable parsing and update-or-create behavior rather than uncontrolled duplicates. The model and database integration reside in `app/models.py`, with database settings supplied via `app/config.py`.

#### Source Ingestion: News
The news ingestion path fetches OSU news pages, extracts candidate events from HTML and full article content, infers dates/types, and returns event-shaped records for persistence. It supports month/year scope, which the parse API can pass through. Its external dependency is the OSU news website, accessed over HTTP and parsed with Requests/BeautifulSoup. Primary implementation: `app/parsers/news_parser.py`.

#### Source Ingestion: Open Days
The open-day parser targets the university applicant/open-day source and interprets tabular HTML, including merged cells, to produce dates, times, locations, and event type. This is a separate source-adapter boundary from news parsing but converges on the same Event schema and persistence path. Primary implementation: `app/parsers/dod_parser.py`.

#### Data and Control Flow
A client invokes `POST /api/parse` in `app/api/routes.py`, optionally selecting `news`, `dod`, or both. Routes instantiate/dispatch source parsers; parsers retrieve and transform external HTML into normalized dictionaries; persistence uses the Event model and external IDs to create or update database rows. Read APIs query the same store with pagination, source/type/activity/date filters; export serializes filtered records, while import validates supplied event-shaped JSON and merges it into the canonical store.

#### Runtime and Infrastructure
The service is Python/Flask with SQLAlchemy and Flask-Migrate, backed by PostgreSQL 15. Docker Compose defines the application (`web`, port 5000) and PostgreSQL (`db`, port 5432) runtime topology with persistent database storage. Environment-based database credentials/URL and Flask secret configuration are represented by `.env` and `app/config.py`; container packaging is in `Dockerfile` and `docker-compose.yaml`.

Main entry points: [`app/api/routes.py`](https://github.com/mrkocmocc/osu.eventaggregator/blob/main/app/api/routes.py) (API routes), [`docker-compose.yaml`](https://github.com/mrkocmocc/osu.eventaggregator/blob/main/docker-compose.yaml) (Compose topology), [`app/models.py`](https://github.com/mrkocmocc/osu.eventaggregator/blob/main/app/models.py) (Event), [`app/__init__.py`](https://github.com/mrkocmocc/osu.eventaggregator/blob/main/app/__init__.py) (App factory)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_clients["Clients"]
  node_browser["Browser UI<br/>web client"]
  node_api_client["API client<br/>HTTP client"]
end

subgraph group_flask["Flask service"]
  node_run["Process startup<br/>Python entry point<br/>[run.py]"]
  node_app_factory["App factory<br/>Flask bootstrap<br/>[__init__.py]"]
  node_routes["API routes<br/>HTTP orchestration<br/>[routes.py]"]
  node_ui_template["HTML interface<br/>template<br/>[index.html]"]
  node_event_model[("Event<br/>SQLAlchemy model<br/>[models.py]")]
  node_config["Configuration<br/>settings<br/>[config.py]"]
end

subgraph group_ingestion["Source ingestion"]
  node_news_parser["News parser<br/>source adapter<br/>[news_parser.py]"]
  node_dod_parser["Open-day parser<br/>source adapter<br/>[dod_parser.py]"]
  node_osu_news["OSU news site<br/>external HTML source"]
  node_open_day_site["Applicant/open-day site<br/>external HTML source"]
end

subgraph group_runtime["Runtime"]
  node_docker_compose{{"Compose topology<br/>container orchestration"}}
  node_dockerfile["Web image<br/>container build"]
  node_postgres[("PostgreSQL 15<br/>database")]
  node_environment["Environment values<br/>environment file<br/>[.env]"]
end

node_run -->|"starts"| node_app_factory
node_app_factory -->|"registers"| node_routes
node_app_factory -->|"loads"| node_config
node_browser -->|"loads HTML"| node_ui_template
node_browser -->|"calls /api"| node_routes
node_api_client -->|"calls /api"| node_routes
node_routes -->|"dispatches parse"| node_news_parser
node_routes -->|"dispatches parse"| node_dod_parser
node_news_parser -->|"fetches HTML"| node_osu_news
node_dod_parser -->|"fetches HTML"| node_open_day_site
node_news_parser -->|"normalized records"| node_event_model
node_dod_parser -->|"normalized records"| node_event_model
node_routes -->|"queries and upserts"| node_event_model
node_event_model -->|"persists"| node_postgres
node_config -.->|"reads values"| node_environment
node_config -.->|"database URL"| node_postgres
node_docker_compose -->|"builds web"| node_dockerfile
node_docker_compose -->|"runs db"| node_postgres
node_docker_compose -->|"runs web"| node_run
node_docker_compose -.->|"loads"| node_environment

click node_run "https://github.com/mrkocmocc/osu.eventaggregator/blob/main/run.py"
click node_app_factory "https://github.com/mrkocmocc/osu.eventaggregator/blob/main/app/__init__.py"
click node_routes "https://github.com/mrkocmocc/osu.eventaggregator/blob/main/app/api/routes.py"
click node_ui_template "https://github.com/mrkocmocc/osu.eventaggregator/blob/main/app/templates/index.html"
click node_event_model "https://github.com/mrkocmocc/osu.eventaggregator/blob/main/app/models.py"
click node_config "https://github.com/mrkocmocc/osu.eventaggregator/blob/main/app/config.py"
click node_news_parser "https://github.com/mrkocmocc/osu.eventaggregator/blob/main/app/parsers/news_parser.py"
click node_dod_parser "https://github.com/mrkocmocc/osu.eventaggregator/blob/main/app/parsers/dod_parser.py"
click node_docker_compose "https://github.com/mrkocmocc/osu.eventaggregator/blob/main/docker-compose.yaml"
click node_dockerfile "https://github.com/mrkocmocc/osu.eventaggregator/blob/main/Dockerfile"
click node_environment "https://github.com/mrkocmocc/osu.eventaggregator/blob/main/.env"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_browser,node_api_client toneBlue
class node_run,node_app_factory,node_routes,node_ui_template,node_event_model,node_config toneAmber
class node_news_parser,node_dod_parser,node_osu_news,node_open_day_site toneMint
class node_docker_compose,node_dockerfile,node_postgres,node_environment toneRose
```

## Components

### Clients

- **Browser UI**: web client · Management and event viewing interface.
- **API client**: HTTP client

### Flask service

- [Process startup](https://github.com/mrkocmocc/osu.eventaggregator/blob/main/run.py): Python entry point · `run.py`
- [App factory](https://github.com/mrkocmocc/osu.eventaggregator/blob/main/app/__init__.py): Flask bootstrap · Constructs the application and registers extensions. · `app/__init__.py`
- [API routes](https://github.com/mrkocmocc/osu.eventaggregator/blob/main/app/api/routes.py): HTTP orchestration · Serves /api reads, parsing, import, and export. · `app/api/routes.py`
- [HTML interface](https://github.com/mrkocmocc/osu.eventaggregator/blob/main/app/templates/index.html): template · `app/templates/index.html`
- [Event](https://github.com/mrkocmocc/osu.eventaggregator/blob/main/app/models.py): SQLAlchemy model · Canonical normalized event record keyed by external_id. · `app/models.py`
- [Configuration](https://github.com/mrkocmocc/osu.eventaggregator/blob/main/app/config.py): settings · Database URL and Flask settings. · `app/config.py`

### Source ingestion

- [News parser](https://github.com/mrkocmocc/osu.eventaggregator/blob/main/app/parsers/news_parser.py): source adapter · Extracts and normalizes event candidates from news articles. · `app/parsers/news_parser.py`
- [Open-day parser](https://github.com/mrkocmocc/osu.eventaggregator/blob/main/app/parsers/dod_parser.py): source adapter · Parses tabular applicant/open-day data, including merged cells. · `app/parsers/dod_parser.py`
- **OSU news site**: external HTML source
- **Applicant/open-day site**: external HTML source

### Runtime

- [Compose topology](https://github.com/mrkocmocc/osu.eventaggregator/blob/main/docker-compose.yaml): container orchestration · Runs web on 5000 and PostgreSQL on 5432. · `docker-compose.yaml`
- [Web image](https://github.com/mrkocmocc/osu.eventaggregator/blob/main/Dockerfile): container build · `Dockerfile`
- **PostgreSQL 15**: database · Persistent canonical event store.
- [Environment values](https://github.com/mrkocmocc/osu.eventaggregator/blob/main/.env): environment file · Supplies credentials and secret configuration. · `.env`

## Connections

- Process startup → App factory: starts
- App factory → API routes: registers
- App factory → Configuration: loads
- Browser UI → HTML interface: loads HTML
- Browser UI → API routes: calls /api
- API client → API routes: calls /api
- API routes → News parser: dispatches parse
- API routes → Open-day parser: dispatches parse
- News parser → OSU news site: fetches HTML
- Open-day parser → Applicant/open-day site: fetches HTML
- News parser → Event: normalized records
- Open-day parser → Event: normalized records
- API routes → Event: queries and upserts
- Event → PostgreSQL 15: persists
- Configuration → Environment values: reads values
- Configuration → PostgreSQL 15: database URL
- Compose topology → Web image: builds web
- Compose topology → PostgreSQL 15: runs db
- Compose topology → Process startup: runs web
- Compose topology → Environment values: loads