# asselk1/ows-assignment architecture

> Architecture diagram of the GitHub repository asselk1/ows-assignment, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-03-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/asselk1/ows-assignment
- Repository: https://github.com/asselk1/ows-assignment

## Overview

This repository implements a domain-specific procurement AI system with two main faces:

1. an online serving layer exposed as an API and chat client, and
2. an offline data pipeline that ingests procurement data, validates it, stores it, and builds retrieval/analytics assets.

The architecture is centered on a relational database, a vector store for retrieval, and an LLM-backed agent that combines analytics, classification, and explanation generation.

Core subsystems

- API layer: `src/api`
  - `server.py` is the FastAPI application entrypoint.
  - `endpoints.py` defines the HTTP routes.
  - `models.py` defines request/response schemas.
  - `ows_client.py` is the outbound integration point to the external procurement data source or service.
  - This layer is the public boundary for both programmatic clients and the chat UI.

- Chat client: `chat.py`
  - A local CLI client that talks to the API.
  - It is a thin presentation layer; it does not contain business logic.

- Agent layer: `src/agent`
  - `classifier.py` likely routes or classifies the user query into intent/domain buckets.
  - `response_generator.py` produces the final user-facing answer.
  - `explainability.py` adds rationale, provenance, or why-an-answer framing.
  - This layer sits between the API and the underlying analytics/RAG/LLM components.

- Analytics layer: `src/analytics`
  - `statistics.py` and `queries.py` support database-driven analytics.
  - `anomaly_price.py`, `anomaly_volume.py`, and `anomaly_fairness.py` implement domain-specific anomaly detection.
  - This layer is where procurement-specific insights are computed from stored data.
  - It is the main structured-data reasoning subsystem.

- RAG layer: `src/rag`
  - `vector_store.py` manages the embedding-backed index.
  - `retriever.py` performs retrieval over indexed content.
  - This layer supports semantic lookup over contract-related material and is distinct from the SQL analytics path.

- LLM layer: `src/llm`
  - `client.py` wraps access to the language model provider.
  - The rest of the code likely treats this as an abstraction so the agent can remain model-agnostic.

- ETL layer: `src/etl`
  - `etl_plans.py`, `etl_subjects.py`, `etl_announcements.py`, `etl_lots.py`, `etl_contracts.py`, and `etl_contract_acts.py` correspond to domain entities in the procurement lifecycle.
  - `load_ref_data.py` handles reference/master data.
  - This layer is the batch ingestion boundary from the external procurement system into the local database.

- Data quality layer: `src/quality`
  - `data_quality.py` likely validates completeness, consistency, and other dataset integrity checks before data is used for analytics or retrieval.
  - This is a guardrail between ingestion and downstream consumption.

- Scheduler / refresh layer: `src/scheduler`
  - `refresh.py` suggests periodic rebuilds or synchronization jobs.
  - This is the automation boundary that keeps the database and vector store current.

- Database layer:
  - `src/database/create_tables.sql` defines the relational schema.
  - `initialize_db.py` bootstraps the database.
  - The repo structure indicates SQL-backed persistence, with the ETL and analytics code built around it.

- Maintenance / bootstrapping scripts:
  - `run_etl.py` likely orchestrates the full ingestion pipeline.
  - `index_all_contracts.py` likely rebuilds the retrieval index from stored contract data.
  - `initialize_db.py` creates or prepares the database schema.
  - These are offline operational entrypoints rather than user-facing features.

Main data flow

1. External procurement data source
   - The system pulls data from an external procurement API/service through `src/api/ows_client.py`.
   - The ETL code knows the domain entities and transforms raw source records into normalized local tables.

2. Ingestion and storage
   - `run_etl.py` or the ETL modules populate the relational database defined by `create_tables.sql`.
   - Reference data is loaded first, then transactional entities such as plans, announcements, lots, contracts, and contract acts.

3. Data validation
   - `src/quality/data_quality.py` checks the loaded data.
   - This is an important boundary because analytics and retrieval depend on the quality of the stored records.

4. Derived assets
   - Analytics modules read from the database to compute statistics and anomaly signals.
   - The RAG subsystem builds or refreshes a vector store from contract-related text/content using `index_all_contracts.py` and the retriever/vector store modules.
   - These are separate derived representations of the same domain data: one structured, one semantic.

5. Online query handling
   - A user query enters through `chat.py` or an HTTP client into the FastAPI server.
   - The API forwards the request into the agent layer.
   - The classifier decides whether the request should be answered using analytics, retrieval, general LLM generation, or a combination.
   - The response generator composes the final answer, possibly enriched with explainability details and retrieved evidence.

Important boundaries

- Online vs offline
  - Offline: ETL, DB initialization, index building, refresh jobs, data quality.
  - Online: API server, agent orchestration, chat client, query-time retrieval and analytics.
  - This separation is central to the design.

- Structured analytics vs semantic retrieval
  - Analytics modules use SQL/data statistics for precise procurement metrics and anomaly detection.
  - RAG uses embeddings/vector search for unstructured or semi-structured contract knowledge.
  - The agent can choose between them depending on the query.

- Domain integration vs presentation
  - `ows_client.py` and the ETL modules interface with external procurement data.
  - `chat.py` and `src/api/server.py` are presentation/serving interfaces.
  - The agent layer bridges these worlds.

Likely technologies and operational shape

- Python is the primary runtime.
- FastAPI/Uvicorn is used for the API server.
- SQL is used for persistence and analytics access.
- A vector store is used for retrieval-augmented generation.
- An external LLM is accessed through a wrapper client.
- Docker is supported through `docker-compose.yml` and `docker/Dockerfile.api`, indicating containerized deployment of the API stack.
- `config.yaml` likely centralizes environment configuration such as endpoints, credentials, database settings, and model/vector store parameters.

How to think about the architecture diagram

A good diagram for this repo would show:

- On the left, an external procurement data source feeding the ETL pipeline.
- In the middle, a relational database as the system of record.
- Nearby, a data-quality gate and a scheduler/refresh process that keep the database and retrieval index current.
- A separate vector store fed from contract data for semantic retrieval.
- On the right, the API server receiving requests from the chat client or other consumers.
- Inside the API, an agent layer that routes queries to:
  - analytics modules for statistics and anomaly detection,
  - retriever/vector store for semantic lookup,
  - the LLM client for generation and explanation.
- The final response returns through the API to the chat client.

In short, this is not just a chatbot repo. It is a procurement intelligence system with a clear split between ingestion, validation, structured analytics, retrieval, and LLM-driven response generation.

Main entry points: [`src/database/create_tables.sql`](https://github.com/asselk1/ows-assignment/blob/HEAD/src/database/create_tables.sql) (Relational DB), [`src/agent/classifier.py`](https://github.com/asselk1/ows-assignment/blob/HEAD/src/agent/classifier.py) (Classifier), [`run_etl.py`](https://github.com/asselk1/ows-assignment/blob/HEAD/run_etl.py) (ETL pipeline), [`src/api/endpoints.py`](https://github.com/asselk1/ows-assignment/blob/HEAD/src/api/endpoints.py) (API routes)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph "Online serving"
  chat_client["Chat client<br/>CLI frontend<br/>[chat.py]"]
  api_server["API server<br/>FastAPI app<br/>[server.py]"]
  api_routes["API routes<br/>HTTP boundary<br/>[endpoints.py]"]
  api_models["API models<br/>schemas<br/>[models.py]"]
  ows_client["OWS client<br/>source adapter<br/>[ows_client.py]"]
  agent_classifier["Classifier<br/>intent router<br/>[classifier.py]"]
  agent_generator["Response gen<br/>answer composer"]
  agent_explain["Explainability<br/>rationale layer<br/>[explainability.py]"]
end

subgraph "Offline pipeline"
  etl_pipeline["ETL pipeline<br/>batch ingestion<br/>[run_etl.py]"]
  etl_entities["Domain ETL<br/>entity loaders<br/>[etl_contracts.py]"]
  ref_loader["Ref data<br/>master data load<br/>[load_ref_data.py]"]
  data_quality["Data quality<br/>validation gate<br/>[data_quality.py]"]
  refresh_job["Refresh job<br/>sync scheduler<br/>[refresh.py]"]
  index_builder["Index build<br/>offline indexing"]
  db_bootstrap["DB bootstrap<br/>setup script<br/>[initialize_db.py]"]
end

subgraph "Core systems"
  llm_client["LLM client<br/>model wrapper<br/>[client.py]"]
  analytics_stats["Statistics<br/>SQL analytics<br/>[statistics.py]"]
  analytics_anomaly["Anomalies<br/>domain detection<br/>[anomaly_price.py]"]
  rag_retriever["Retriever<br/>semantic search<br/>[retriever.py]"]
  vector_store[("Vector store<br/>embedding index<br/>[vector_store.py]")]
  database[("Relational DB<br/>system of record<br/>[create_tables.sql]")]
end

external_source(("Procurement source<br/>external data source"))

chat_client -->|"queries"| api_server
api_server -->|"dispatches"| api_routes
api_routes -->|"validates"| api_models
api_routes -->|"hands off"| agent_classifier
agent_classifier -->|"routes to"| analytics_stats
agent_classifier -->|"routes to"| analytics_anomaly
agent_classifier -->|"routes to"| rag_retriever
agent_classifier -->|"routes to"| agent_generator
agent_generator -->|"enriches"| agent_explain
agent_generator -->|"generates with"| llm_client
rag_retriever -->|"reads"| vector_store
analytics_stats -->|"reads"| database
analytics_anomaly -->|"reads"| database
etl_pipeline -->|"starts with"| ref_loader
etl_pipeline -->|"loads"| etl_entities
etl_entities -->|"writes"| database
ref_loader -->|"writes"| database
data_quality -->|"checks"| database
refresh_job -->|"syncs"| database
refresh_job -->|"rebuilds"| vector_store
index_builder -->|"builds"| vector_store
index_builder -->|"reads"| database
db_bootstrap -->|"initializes"| database
ows_client -->|"connects to"| external_source
etl_pipeline -->|"ingests from"| ows_client

click chat_client "https://github.com/asselk1/ows-assignment/blob/master/chat.py"
click api_server "https://github.com/asselk1/ows-assignment/blob/master/src/api/server.py"
click api_routes "https://github.com/asselk1/ows-assignment/blob/master/src/api/endpoints.py"
click api_models "https://github.com/asselk1/ows-assignment/blob/master/src/api/models.py"
click ows_client "https://github.com/asselk1/ows-assignment/blob/master/src/api/ows_client.py"
click agent_classifier "https://github.com/asselk1/ows-assignment/blob/master/src/agent/classifier.py"
click agent_generator "https://github.com/asselk1/ows-assignment/blob/master/src/agent/response_generator.py"
click agent_explain "https://github.com/asselk1/ows-assignment/blob/master/src/agent/explainability.py"
click llm_client "https://github.com/asselk1/ows-assignment/blob/master/src/llm/client.py"
click analytics_stats "https://github.com/asselk1/ows-assignment/blob/master/src/analytics/statistics.py"
click analytics_anomaly "https://github.com/asselk1/ows-assignment/blob/master/src/analytics/anomaly_price.py"
click rag_retriever "https://github.com/asselk1/ows-assignment/blob/master/src/rag/retriever.py"
click vector_store "https://github.com/asselk1/ows-assignment/blob/master/src/rag/vector_store.py"
click database "https://github.com/asselk1/ows-assignment/blob/master/src/database/create_tables.sql"
click etl_pipeline "https://github.com/asselk1/ows-assignment/blob/master/run_etl.py"
click etl_entities "https://github.com/asselk1/ows-assignment/blob/master/src/etl/etl_contracts.py"
click ref_loader "https://github.com/asselk1/ows-assignment/blob/master/src/etl/load_ref_data.py"
click data_quality "https://github.com/asselk1/ows-assignment/blob/master/src/quality/data_quality.py"
click refresh_job "https://github.com/asselk1/ows-assignment/blob/master/src/scheduler/refresh.py"
click index_builder "https://github.com/asselk1/ows-assignment/blob/master/index_all_contracts.py"
click db_bootstrap "https://github.com/asselk1/ows-assignment/blob/master/initialize_db.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class chat_client,api_server,api_routes,api_models,ows_client,agent_classifier,agent_generator,agent_explain toneBlue
class etl_pipeline,etl_entities,ref_loader,data_quality,refresh_job,index_builder,db_bootstrap toneAmber
class llm_client,analytics_stats,analytics_anomaly,rag_retriever,vector_store,database toneMint
class external_source toneNeutral
```

## Components

### Online serving

API, chat, and query-time orchestration.

- [Chat client](https://github.com/asselk1/ows-assignment/blob/HEAD/chat.py): CLI frontend · Thin local client for sending user queries to the API. · `chat.py`
- [API server](https://github.com/asselk1/ows-assignment/blob/HEAD/src/api/server.py): FastAPI app · HTTP serving entrypoint for programmatic and chat-based queries. · `src/api/server.py`
- [API routes](https://github.com/asselk1/ows-assignment/blob/HEAD/src/api/endpoints.py): HTTP boundary · Defines request handling and public endpoints. · `src/api/endpoints.py`
- [API models](https://github.com/asselk1/ows-assignment/blob/HEAD/src/api/models.py): schemas · Request and response shapes for the public API. · `src/api/models.py`
- [OWS client](https://github.com/asselk1/ows-assignment/blob/HEAD/src/api/ows_client.py): source adapter · Outbound integration to the external procurement source. · `src/api/ows_client.py`
- [Classifier](https://github.com/asselk1/ows-assignment/blob/HEAD/src/agent/classifier.py): intent router · Routes queries to analytics, retrieval, or generation paths. · `src/agent/classifier.py`
- [Response gen](https://github.com/asselk1/ows-assignment/blob/HEAD/src/agent/response_generator.py): answer composer · Builds the final user-facing response. · `src/agent/response_generator.py`
- [Explainability](https://github.com/asselk1/ows-assignment/blob/HEAD/src/agent/explainability.py): rationale layer · Adds provenance and explanation framing to answers. · `src/agent/explainability.py`

### Offline pipeline

Ingestion, validation, indexing, and refresh jobs.

- [ETL pipeline](https://github.com/asselk1/ows-assignment/blob/HEAD/run_etl.py): batch ingestion · Transforms and loads procurement entities into the database. · `run_etl.py`
- [Domain ETL](https://github.com/asselk1/ows-assignment/blob/HEAD/src/etl/etl_contracts.py): entity loaders · Loads plans, subjects, announcements, lots, contracts, and acts. · `src/etl/etl_contracts.py`
- [Ref data](https://github.com/asselk1/ows-assignment/blob/HEAD/src/etl/load_ref_data.py): master data load · Loads reference and lookup data needed by downstream processing. · `src/etl/load_ref_data.py`
- [Data quality](https://github.com/asselk1/ows-assignment/blob/HEAD/src/quality/data_quality.py): validation gate · Checks completeness and consistency before analytics and retrieval. · `src/quality/data_quality.py`
- [Refresh job](https://github.com/asselk1/ows-assignment/blob/HEAD/src/scheduler/refresh.py): sync scheduler · Keeps database and derived assets current. · `src/scheduler/refresh.py`
- [Index build](https://github.com/asselk1/ows-assignment/blob/HEAD/index_all_contracts.py): offline indexing · Rebuilds the retrieval index from stored contract data. · `index_all_contracts.py`
- [DB bootstrap](https://github.com/asselk1/ows-assignment/blob/HEAD/initialize_db.py): setup script · Initializes or prepares the database schema. · `initialize_db.py`

### Core systems

Shared persistence, retrieval, and model access.

- [LLM client](https://github.com/asselk1/ows-assignment/blob/HEAD/src/llm/client.py): model wrapper · Abstracts access to the external language model provider. · `src/llm/client.py`
- [Statistics](https://github.com/asselk1/ows-assignment/blob/HEAD/src/analytics/statistics.py): SQL analytics · Database-driven procurement metrics and queries. · `src/analytics/statistics.py`
- [Anomalies](https://github.com/asselk1/ows-assignment/blob/HEAD/src/analytics/anomaly_price.py): domain detection · Price, volume, and fairness anomaly detection. · `src/analytics/anomaly_price.py`
- [Retriever](https://github.com/asselk1/ows-assignment/blob/HEAD/src/rag/retriever.py): semantic search · Fetches relevant contract content from the vector index. · `src/rag/retriever.py`
- [Vector store](https://github.com/asselk1/ows-assignment/blob/HEAD/src/rag/vector_store.py): embedding index · Backs semantic retrieval over contract-related text. · `src/rag/vector_store.py`
- [Relational DB](https://github.com/asselk1/ows-assignment/blob/HEAD/src/database/create_tables.sql): system of record · Primary structured store for normalized procurement data. · `src/database/create_tables.sql`

### Other components

- **Procurement source**: external data source · Upstream procurement API or service providing raw domain records.

## Connections

- Chat client → API server: queries · Chat UI sends user requests to the API.
- API server → API routes: dispatches · Server exposes the HTTP routes.
- API routes → API models: validates · Routes use request and response schemas.
- API routes → Classifier: hands off · Incoming requests are forwarded to the agent.
- Classifier → Statistics: routes to · Structured questions are answered with SQL analytics.
- Classifier → Anomalies: routes to · Anomaly-oriented questions use domain detectors.
- Classifier → Retriever: routes to · Semantic lookup requests go to retrieval.
- Classifier → Response gen: routes to · General or combined answers are composed downstream.
- Response gen → Explainability: enriches · Final answers can include rationale and provenance.
- Response gen → LLM client: generates with · The agent uses the language model for text generation.
- Retriever → Vector store: reads · Retriever queries the embedding-backed index.
- Statistics → Relational DB: reads · Analytics run against the relational store.
- Anomalies → Relational DB: reads · Anomaly detection uses structured stored data.
- ETL pipeline → Ref data: starts with · Reference data is loaded before transactional entities.
- ETL pipeline → Domain ETL: loads · Domain entity loaders populate procurement tables.
- Domain ETL → Relational DB: writes · Normalized records are persisted to the database.
- Ref data → Relational DB: writes · Master data is stored in the database.
- Data quality → Relational DB: checks · Validation runs over loaded data.
- Refresh job → Relational DB: syncs · Refresh jobs keep the database current.
- Refresh job → Vector store: rebuilds · Refresh jobs also update retrieval assets.
- Index build → Vector store: builds · Offline indexing populates the vector store.
- Index build → Relational DB: reads · Indexing pulls source content from stored contracts.
- DB bootstrap → Relational DB: initializes · Bootstrap creates or prepares the schema.
- OWS client → Procurement source: connects to · Outbound adapter reaches the upstream procurement source.
- ETL pipeline → OWS client: ingests from · ETL retrieves raw source data through the adapter.
