mrkocmocc/osu.eventaggregator

Purpose and Entry Points

OSU.EventAggregator collects Orenburg State University event data from news and open-day pages, persists normalized events, and exposes them through a Flask REST API plus a lightweight browser UI. Process startup is run.py; application construction and extension registration are centered in app/__init__.py.

Application and HTTP Boundary

The Flask application serves API operations under /api and the HTML interface. API routes provide event listing/detail/upcoming queries, statistics, parser triggering, and JSON import/export. app/api/routes.py is the orchestration boundary between requests, parsers, ORM queries, and serialization; app/templates/index.html consumes the service as a management/viewing interface.

Canonical Event Persistence Model

Event is the system’s canonical normalized record: source-specific identifiers, event metadata, source provenance, parsing timestamps, active state, and audit timestamps. Uniqueness of external_id supports repeatable parsing and update-or-create behavior rather than uncontrolled duplicates. The model and database integration reside in app/models.py, with database settings supplied via app/config.py.

Source Ingestion: News

The news ingestion path fetches OSU news pages, extracts candidate events from HTML and full article content, infers dates/types, and returns event-shaped records for persistence. It supports month/year scope, which the parse API can pass through. Its external dependency is the OSU news website, accessed over HTTP and parsed with Requests/BeautifulSoup. Primary implementation: app/parsers/news_parser.py.

Source Ingestion: Open Days

The open-day parser targets the university applicant/open-day source and interprets tabular HTML, including merged cells, to produce dates, times, locations, and event type. This is a separate source-adapter boundary from news parsing but converges on the same Event schema and persistence path. Primary implementation: app/parsers/dod_parser.py.

Data and Control Flow

A client invokes POST /api/parse in app/api/routes.py, optionally selecting news, dod, or both. Routes instantiate/dispatch source parsers; parsers retrieve and transform external HTML into normalized dictionaries; persistence uses the Event model and external IDs to create or update database rows. Read APIs query the same store with pagination, source/type/activity/date filters; export serializes filtered records, while import validates supplied event-shaped JSON and merges it into the canonical store.

Runtime and Infrastructure

The service is Python/Flask with SQLAlchemy and Flask-Migrate, backed by PostgreSQL 15. Docker Compose defines the application (web, port 5000) and PostgreSQL (db, port 5432) runtime topology with persistent database storage. Environment-based database credentials/URL and Flask secret configuration are represented by .env and app/config.py; container packaging is in Dockerfile and docker-compose.yaml.