How rrshafikov/mts-data-enricher works
mts-data-enricher is a PostgreSQL-backed service that enriches pending item records from a configurable public REST API, plus a small web UI for viewing records and triggering work. The batch entry point runs the enrichment loop; it selects pending items, fetches each profile, extracts a subset of fields, then commits successful updates individually while continuing past failures. The web UI can also add pending items and invoke that same loop. The graph reflects the README and sampled modules; database migrations and deployment wiring are omitted as runtime architecture.
Main entry points: main.py Web routes · enricher.py Enrichment loop · config.py Settings · db.py DB sessions
Components 13
Batch enrichment
- Batch entry
app/__main__.py - Enrichment loop
app/enricher.py - API client
app/api_client.py
Web interface
- Web routes
app/web/main.py - Web assets
app/web/static/app.js - Page template
app/web/templates/index.html
Data and settings
- Pending item seeding
app/seed.py - DB sessions
app/db.py - Item model
app/models.py - Settings
app/config.py
Other components
- Operator
- Public REST API
- PostgreSQL
Connections 21
- Operator → Batch entry: starts batch
- Batch entry → Enrichment loop: runs
- Enrichment loop → API client: fetches profiles
- API client → Public REST API: HTTP requests
- Enrichment loop → DB sessions: queries and commits
- Enrichment loop → Item model: uses item schema
- DB sessions → PostgreSQL: connects
- Operator → Web routes: uses UI
- Web routes → Page template: renders page
- Web routes → Web assets: serves scripts
- Web routes → DB sessions: loads items
- Web routes → Item model: queries items
- Web routes → Pending item seeding: adds pending items
- Web routes → Enrichment loop: starts enrichment
- Pending item seeding → DB sessions: inserts items
- Pending item seeding → Item model: creates items
- Settings → DB sessions: configures connection
- Settings → API client: configures API
- Settings → Enrichment loop: sets batch size
- Settings → Batch entry: sets log level
- Settings → Web routes: sets log level