# rrshafikov/mts-data-enricher architecture

> Architecture diagram of the GitHub repository rrshafikov/mts-data-enricher, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rrshafikov/mts-data-enricher
- Repository: https://github.com/rrshafikov/mts-data-enricher

## Overview

mts-data-enricher is a PostgreSQL-backed service that enriches pending item records from a configurable public REST API, plus a small web UI for viewing records and triggering work. The batch entry point runs the enrichment loop; it selects pending items, fetches each profile, extracts a subset of fields, then commits successful updates individually while continuing past failures. The web UI can also add pending items and invoke that same loop. The graph reflects the README and sampled modules; database migrations and deployment wiring are omitted as runtime architecture.

Main entry points: [`app/web/main.py`](https://github.com/rrshafikov/mts-data-enricher/blob/main/app/web/main.py) (Web routes), [`app/enricher.py`](https://github.com/rrshafikov/mts-data-enricher/blob/main/app/enricher.py) (Enrichment loop), [`app/config.py`](https://github.com/rrshafikov/mts-data-enricher/blob/main/app/config.py) (Settings), [`app/db.py`](https://github.com/rrshafikov/mts-data-enricher/blob/main/app/db.py) (DB sessions)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_batch["Batch enrichment"]
  node_batch_entry["Batch entry<br/>[__main__.py]"]
  node_enricher["Enrichment loop<br/>[enricher.py]"]
  node_api_client["API client<br/>[api_client.py]"]
end

subgraph group_web["Web interface"]
  node_web_app["Web routes<br/>[main.py]"]
  node_web_assets["Web assets<br/>[app.js]"]
  node_web_template["Page template<br/>[index.html]"]
end

subgraph group_data["Data and settings"]
  node_seed["Pending item seeding<br/>[seed.py]"]
  node_db_access["DB sessions<br/>[db.py]"]
  node_item_model["Item model<br/>[models.py]"]
  node_settings["Settings<br/>[config.py]"]
end

node_operator(("Operator"))
node_public_api["Public REST API"]
node_postgres[("PostgreSQL")]

node_operator -->|"starts batch"| node_batch_entry
node_batch_entry -->|"runs"| node_enricher
node_enricher -->|"fetches profiles"| node_api_client
node_api_client -->|"HTTP requests"| node_public_api
node_enricher -->|"queries and commits"| node_db_access
node_enricher -->|"uses item schema"| node_item_model
node_db_access -->|"connects"| node_postgres
node_operator -->|"uses UI"| node_web_app
node_web_app -->|"renders page"| node_web_template
node_web_app -->|"serves scripts"| node_web_assets
node_web_app -->|"loads items"| node_db_access
node_web_app -->|"queries items"| node_item_model
node_web_app -->|"adds pending items"| node_seed
node_web_app -->|"starts enrichment"| node_enricher
node_seed -->|"inserts items"| node_db_access
node_seed -->|"creates items"| node_item_model
node_settings -->|"configures connection"| node_db_access
node_settings -->|"configures API"| node_api_client
node_settings -->|"sets batch size"| node_enricher
node_settings -->|"sets log level"| node_batch_entry
node_settings -->|"sets log level"| node_web_app

click node_batch_entry "https://github.com/rrshafikov/mts-data-enricher/blob/main/app/__main__.py"
click node_enricher "https://github.com/rrshafikov/mts-data-enricher/blob/main/app/enricher.py"
click node_api_client "https://github.com/rrshafikov/mts-data-enricher/blob/main/app/api_client.py"
click node_web_app "https://github.com/rrshafikov/mts-data-enricher/blob/main/app/web/main.py"
click node_web_assets "https://github.com/rrshafikov/mts-data-enricher/blob/main/app/web/static/app.js"
click node_web_template "https://github.com/rrshafikov/mts-data-enricher/blob/main/app/web/templates/index.html"
click node_seed "https://github.com/rrshafikov/mts-data-enricher/blob/main/app/seed.py"
click node_db_access "https://github.com/rrshafikov/mts-data-enricher/blob/main/app/db.py"
click node_item_model "https://github.com/rrshafikov/mts-data-enricher/blob/main/app/models.py"
click node_settings "https://github.com/rrshafikov/mts-data-enricher/blob/main/app/config.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_batch_entry,node_enricher,node_api_client toneBlue
class node_web_app,node_web_assets,node_web_template,node_postgres toneAmber
class node_seed,node_db_access,node_item_model,node_settings,node_public_api toneMint
class node_operator toneIndigo
```

## Components

### Batch enrichment

- [Batch entry](https://github.com/rrshafikov/mts-data-enricher/blob/main/app/__main__.py): `app/__main__.py`
- [Enrichment loop](https://github.com/rrshafikov/mts-data-enricher/blob/main/app/enricher.py): `app/enricher.py`
- [API client](https://github.com/rrshafikov/mts-data-enricher/blob/main/app/api_client.py): `app/api_client.py`

### Web interface

- [Web routes](https://github.com/rrshafikov/mts-data-enricher/blob/main/app/web/main.py): `app/web/main.py`
- [Web assets](https://github.com/rrshafikov/mts-data-enricher/blob/main/app/web/static/app.js): `app/web/static/app.js`
- [Page template](https://github.com/rrshafikov/mts-data-enricher/blob/main/app/web/templates/index.html): `app/web/templates/index.html`

### Data and settings

- [Pending item seeding](https://github.com/rrshafikov/mts-data-enricher/blob/main/app/seed.py): `app/seed.py`
- [DB sessions](https://github.com/rrshafikov/mts-data-enricher/blob/main/app/db.py): `app/db.py`
- [Item model](https://github.com/rrshafikov/mts-data-enricher/blob/main/app/models.py): `app/models.py`
- [Settings](https://github.com/rrshafikov/mts-data-enricher/blob/main/app/config.py): `app/config.py`

### Other components

- **Operator**
- **Public REST API**
- **PostgreSQL**

## Connections

- Operator → Batch entry: starts batch
- Batch entry → Enrichment loop: runs
- Enrichment loop → API client: fetches profiles
- API client → Public REST API: HTTP requests
- Enrichment loop → DB sessions: queries and commits
- Enrichment loop → Item model: uses item schema
- DB sessions → PostgreSQL: connects
- Operator → Web routes: uses UI
- Web routes → Page template: renders page
- Web routes → Web assets: serves scripts
- Web routes → DB sessions: loads items
- Web routes → Item model: queries items
- Web routes → Pending item seeding: adds pending items
- Web routes → Enrichment loop: starts enrichment
- Pending item seeding → DB sessions: inserts items
- Pending item seeding → Item model: creates items
- Settings → DB sessions: configures connection
- Settings → API client: configures API
- Settings → Enrichment loop: sets batch size
- Settings → Batch entry: sets log level
- Settings → Web routes: sets log level