# utkristauwa/cits3200_57 architecture

> Architecture diagram of the GitHub repository utkristauwa/cits3200_57, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/utkristauwa/cits3200_57
- Repository: https://github.com/utkristauwa/cits3200_57

## Overview

TenderAI discovers tenders across procurement and funding portals, extracts tender and attachment content, and stores records for staff to browse. The main evidenced workflow is orchestrated by `manager.py`: it runs scrapers, processes documents and tender data, stores attachments, generates embeddings, and upserts records to BigQuery. The API supports filtered and semantic search, while the React app provides browsing and administration. README capabilities such as profile-based matching, review feedback, and Teams notifications are not evidenced in the sampled source, so their wiring is omitted.

Main entry points: [`manager.py`](https://github.com/utkristauwa/cits3200_57/blob/main/manager.py) (Pipeline manager), [`api/app/main.py`](https://github.com/utkristauwa/cits3200_57/blob/main/api/app/main.py) (FastAPI application), [`frontend/src/App.tsx`](https://github.com/utkristauwa/cits3200_57/blob/main/frontend/src/App.tsx) (TenderAI web app), [`api/app/bigquery.py`](https://github.com/utkristauwa/cits3200_57/blob/main/api/app/bigquery.py) (Tender search)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_collection["Collection and processing"]
  node_manager["Pipeline manager<br/>[manager.py]"]
  node_scrapers["Portal scrapers<br/>[run_scrapers.py]"]
  node_error_scrapers["Additional scrapers"]
  node_doc_extract["Attachment extraction<br/>[main.py]"]
  node_processor["Tender processing"]
  node_attachment_store["Attachment storage"]
end

subgraph group_data["Tender data services"]
  node_ingest["Tender ingestion"]
  node_bq_writer["BigQuery upserts<br/>[bigquery_client.py]"]
  node_bq[("Tender database")]
  node_cloud_storage[("Cloud Storage")]
end

subgraph group_api["API and administration"]
  node_api_app["FastAPI application<br/>[main.py]"]
  node_tender_api["Tender endpoints<br/>[tenders.py]"]
  node_search["Tender search<br/>[bigquery.py]"]
  node_auth_api["Authentication<br/>[auth.py]"]
  node_health_api["Health endpoints<br/>[health.py]"]
  node_functions["User administration<br/>[index.ts]"]
end

subgraph group_ui["Staff interface"]
  node_frontend["TenderAI web app<br/>[App.tsx]"]
  node_tender_page["Tender browsing<br/>[TendersPage.tsx]"]
  node_api_client["API client<br/>[api.ts]"]
  node_admin_pages["Admin pages<br/>[ConfigPage.tsx]"]
  node_health_page["System health view"]
  node_favorites["Saved tenders"]
end

node_staff(("SVA staff"))
node_portals(("Tender portals"))
node_firebase_auth[("Firebase Auth")]
node_firestore[("Firestore profiles")]

node_manager -->|"runs"| node_scrapers
node_manager -->|"runs"| node_error_scrapers
node_scrapers -->|"scrapes"| node_portals
node_error_scrapers -->|"scrapes"| node_portals
node_manager -->|"extracts text"| node_doc_extract
node_manager -->|"processes tenders"| node_processor
node_manager -->|"stores attachments"| node_attachment_store
node_attachment_store -->|"writes files"| node_cloud_storage
node_manager -->|"submits records"| node_ingest
node_ingest -->|"upserts"| node_bq_writer
node_bq_writer -->|"writes records"| node_bq
node_frontend -->|"routes to"| node_tender_page
node_frontend -->|"routes to"| node_admin_pages
node_frontend -->|"routes to"| node_health_page
node_staff -->|"uses"| node_frontend
node_tender_page -->|"requests tenders"| node_api_client
node_api_client -->|"calls"| node_api_app
node_api_app -->|"dispatches"| node_tender_api
node_api_app -->|"authenticates"| node_auth_api
node_api_app -->|"dispatches"| node_health_api
node_tender_api -->|"queries"| node_search
node_search -->|"searches"| node_bq
node_search -.->|"reads documents"| node_cloud_storage
node_health_page -.->|"reads status"| node_cloud_storage
node_api_client -->|"checks identity"| node_auth_api
node_auth_api -->|"verifies"| node_firebase_auth
node_admin_pages -->|"admin actions"| node_functions
node_functions -->|"reads/writes profiles"| node_firestore
node_functions -->|"manages accounts"| node_firebase_auth
node_tender_page -->|"saves selections"| node_favorites

click node_manager "https://github.com/utkristauwa/cits3200_57/blob/main/manager.py"
click node_scrapers "https://github.com/utkristauwa/cits3200_57/blob/main/web_scrapers/run_scrapers.py"
click node_error_scrapers "https://github.com/utkristauwa/cits3200_57/tree/main/error_scrapers"
click node_doc_extract "https://github.com/utkristauwa/cits3200_57/blob/main/document_scraper/main.py"
click node_processor "https://github.com/utkristauwa/cits3200_57/blob/main/processing/tender_processor.py"
click node_attachment_store "https://github.com/utkristauwa/cits3200_57/blob/main/attachment_store.py"
click node_ingest "https://github.com/utkristauwa/cits3200_57/blob/main/ingestion/validate_and_submit.py"
click node_bq_writer "https://github.com/utkristauwa/cits3200_57/blob/main/ingestion/bigquery_client.py"
click node_api_app "https://github.com/utkristauwa/cits3200_57/blob/main/api/app/main.py"
click node_tender_api "https://github.com/utkristauwa/cits3200_57/blob/main/api/app/routers/tenders.py"
click node_search "https://github.com/utkristauwa/cits3200_57/blob/main/api/app/bigquery.py"
click node_auth_api "https://github.com/utkristauwa/cits3200_57/blob/main/api/app/auth.py"
click node_health_api "https://github.com/utkristauwa/cits3200_57/blob/main/api/app/routers/health.py"
click node_functions "https://github.com/utkristauwa/cits3200_57/blob/main/functions/src/index.ts"
click node_frontend "https://github.com/utkristauwa/cits3200_57/blob/main/frontend/src/App.tsx"
click node_tender_page "https://github.com/utkristauwa/cits3200_57/blob/main/frontend/src/pages/TendersPage.tsx"
click node_api_client "https://github.com/utkristauwa/cits3200_57/blob/main/frontend/src/lib/api.ts"
click node_admin_pages "https://github.com/utkristauwa/cits3200_57/blob/main/frontend/src/pages/ConfigPage.tsx"
click node_health_page "https://github.com/utkristauwa/cits3200_57/blob/main/frontend/src/pages/SystemHealthPage.tsx"
click node_favorites "https://github.com/utkristauwa/cits3200_57/blob/main/frontend/src/lib/FavoritesContext.tsx"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_manager,node_scrapers,node_error_scrapers,node_doc_extract,node_processor,node_attachment_store toneBlue
class node_ingest,node_bq_writer,node_bq,node_cloud_storage,node_firebase_auth,node_firestore toneAmber
class node_api_app,node_tender_api,node_search,node_auth_api,node_health_api,node_functions toneMint
class node_frontend,node_tender_page,node_api_client,node_admin_pages,node_health_page,node_favorites toneRose
class node_staff,node_portals toneIndigo
```

## Components

### Collection and processing

- [Pipeline manager](https://github.com/utkristauwa/cits3200_57/blob/main/manager.py): `manager.py`
- [Portal scrapers](https://github.com/utkristauwa/cits3200_57/blob/main/web_scrapers/run_scrapers.py): `web_scrapers/run_scrapers.py`
- [Additional scrapers](https://github.com/utkristauwa/cits3200_57/tree/main/error_scrapers): `error_scrapers`
- [Attachment extraction](https://github.com/utkristauwa/cits3200_57/blob/main/document_scraper/main.py): `document_scraper/main.py`
- [Tender processing](https://github.com/utkristauwa/cits3200_57/blob/main/processing/tender_processor.py): `processing/tender_processor.py`
- [Attachment storage](https://github.com/utkristauwa/cits3200_57/blob/main/attachment_store.py): `attachment_store.py`

### Tender data services

- [Tender ingestion](https://github.com/utkristauwa/cits3200_57/blob/main/ingestion/validate_and_submit.py): `ingestion/validate_and_submit.py`
- [BigQuery upserts](https://github.com/utkristauwa/cits3200_57/blob/main/ingestion/bigquery_client.py): `ingestion/bigquery_client.py`
- **Tender database**
- **Cloud Storage**

### API and administration

- [FastAPI application](https://github.com/utkristauwa/cits3200_57/blob/main/api/app/main.py): `api/app/main.py`
- [Tender endpoints](https://github.com/utkristauwa/cits3200_57/blob/main/api/app/routers/tenders.py): `api/app/routers/tenders.py`
- [Tender search](https://github.com/utkristauwa/cits3200_57/blob/main/api/app/bigquery.py): `api/app/bigquery.py`
- [Authentication](https://github.com/utkristauwa/cits3200_57/blob/main/api/app/auth.py): `api/app/auth.py`
- [Health endpoints](https://github.com/utkristauwa/cits3200_57/blob/main/api/app/routers/health.py): `api/app/routers/health.py`
- [User administration](https://github.com/utkristauwa/cits3200_57/blob/main/functions/src/index.ts): `functions/src/index.ts`

### Staff interface

- [TenderAI web app](https://github.com/utkristauwa/cits3200_57/blob/main/frontend/src/App.tsx): `frontend/src/App.tsx`
- [Tender browsing](https://github.com/utkristauwa/cits3200_57/blob/main/frontend/src/pages/TendersPage.tsx): `frontend/src/pages/TendersPage.tsx`
- [API client](https://github.com/utkristauwa/cits3200_57/blob/main/frontend/src/lib/api.ts): `frontend/src/lib/api.ts`
- [Admin pages](https://github.com/utkristauwa/cits3200_57/blob/main/frontend/src/pages/ConfigPage.tsx): `frontend/src/pages/ConfigPage.tsx`
- [System health view](https://github.com/utkristauwa/cits3200_57/blob/main/frontend/src/pages/SystemHealthPage.tsx): `frontend/src/pages/SystemHealthPage.tsx`
- [Saved tenders](https://github.com/utkristauwa/cits3200_57/blob/main/frontend/src/lib/FavoritesContext.tsx): `frontend/src/lib/FavoritesContext.tsx`

### Other components

- **SVA staff**
- **Tender portals**
- **Firebase Auth**
- **Firestore profiles**

## Connections

- Pipeline manager → Portal scrapers: runs
- Pipeline manager → Additional scrapers: runs
- Portal scrapers → Tender portals: scrapes
- Additional scrapers → Tender portals: scrapes
- Pipeline manager → Attachment extraction: extracts text
- Pipeline manager → Tender processing: processes tenders
- Pipeline manager → Attachment storage: stores attachments
- Attachment storage → Cloud Storage: writes files
- Pipeline manager → Tender ingestion: submits records
- Tender ingestion → BigQuery upserts: upserts
- BigQuery upserts → Tender database: writes records
- TenderAI web app → Tender browsing: routes to
- TenderAI web app → Admin pages: routes to
- TenderAI web app → System health view: routes to
- SVA staff → TenderAI web app: uses
- Tender browsing → API client: requests tenders
- API client → FastAPI application: calls
- FastAPI application → Tender endpoints: dispatches
- FastAPI application → Authentication: authenticates
- FastAPI application → Health endpoints: dispatches
- Tender endpoints → Tender search: queries
- Tender search → Tender database: searches
- Tender search → Cloud Storage: reads documents
- System health view → Cloud Storage: reads status
- API client → Authentication: checks identity
- Authentication → Firebase Auth: verifies
- Admin pages → User administration: admin actions
- User administration → Firestore profiles: reads/writes profiles
- User administration → Firebase Auth: manages accounts
- Tender browsing → Saved tenders: saves selections