# aodn/data-access-service architecture

> Architecture diagram of the GitHub repository aodn/data-access-service, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/aodn/data-access-service
- Repository: https://github.com/aodn/data-access-service

## Overview

Data Access Service exposes HTTP APIs for dataset metadata, data access, estimation, site observations and map tiles. The principal request path enters the FastAPI server and API routers; data access includes subsetting and format delivery, while separate batch jobs prepare tiler products and estimation indexes. Site feature responses have a documented service/repository split. Startup wiring and the site data path are supported by sampled source; several route-to-service and batch-job internals are represented by their tree paths and names, so uncertain call links are omitted.

Main entry points: [`data_access_service/server.py`](https://github.com/aodn/data-access-service/blob/main/data_access_service/server.py) (FastAPI server), [`data_access_service/core/routes`](https://github.com/aodn/data-access-service/tree/main/data_access_service/core/routes) (Data API routes), [`data_access_service/sites/sites_repository.py`](https://github.com/aodn/data-access-service/blob/main/data_access_service/sites/sites_repository.py) (Site repositories), [`data_access_service/core/tiler_routes`](https://github.com/aodn/data-access-service/tree/main/data_access_service/core/tiler_routes) (Tiler routes)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_http["HTTP API"]
  node_server["FastAPI server<br/>[server.py]"]
  node_api_router["Data API routes"]
  node_tiler_router["Tiler routes"]
  node_metadata["Metadata<br/>[metadata.py]"]
  node_data_routes["Data endpoints<br/>[data.py]"]
  node_estimate_route["Estimate endpoint"]
  node_api["API service<br/>[api.py]"]
end

subgraph group_sites["Site Observations"]
  node_site_service["Site feature service"]
  node_site_repo["Site repositories"]
  node_sites_db[("Sites DuckDB<br/>[duckdbclient.py]")]
end

subgraph group_tiler["Tile Services"]
  node_tile_products["Tile products"]
  node_tile_render["Tile rendering"]
  node_tile_store["Tile data store"]
  node_tile_db[("Tiler DuckDB<br/>[duckdbclient.py]")]
end

subgraph group_batch["Batch Processing"]
  node_pmtiles["PMTiles generation<br/>[generator.py]"]
  node_tile_batch["Raster tiling<br/>[generator.py]"]
  node_sites_refresh["Sites refresh<br/>[refresher.py]"]
  node_scheduler["Task scheduler<br/>[scheduler.py]"]
end

subgraph group_data["Data Access"]
  node_subset["Subset processing<br/>[subsetting_main.py]"]
  node_subset_tasks["Subset workers"]
  node_estimation["Estimate index"]
end

node_consumer(("API consumer"))
node_s3[("S3 storage")]

node_consumer -->|"sends requests"| node_server
node_server -->|"mounts routes"| node_api_router
node_server -->|"mounts routes"| node_tiler_router
node_server -->|"initializes"| node_api
node_server -->|"builds repositories"| node_site_repo
node_server -->|"starts"| node_scheduler
node_api_router -->|"serves metadata"| node_metadata
node_api_router -->|"serves data"| node_data_routes
node_api_router -->|"serves estimates"| node_estimate_route
node_site_service -->|"queries"| node_site_repo
node_site_repo -->|"executes queries"| node_sites_db
node_server -->|"initializes"| node_tile_db
node_server -->|"initializes"| node_tile_store
node_tiler_router -->|"routes product requests"| node_tile_products
node_tile_store -->|"queries"| node_tile_db
node_data_routes -.->|"supports subsetting"| node_subset
node_subset -->|"dispatches work"| node_subset_tasks
node_estimate_route -.->|"uses index"| node_estimation
node_tile_batch -.->|"writes tile data"| node_s3
node_pmtiles -.->|"writes tile archives"| node_s3
node_sites_refresh -.->|"refreshes datasets"| node_s3

click node_server "https://github.com/aodn/data-access-service/blob/main/data_access_service/server.py"
click node_api_router "https://github.com/aodn/data-access-service/tree/main/data_access_service/core/routes"
click node_tiler_router "https://github.com/aodn/data-access-service/tree/main/data_access_service/core/tiler_routes"
click node_metadata "https://github.com/aodn/data-access-service/blob/main/data_access_service/core/routes/metadata.py"
click node_data_routes "https://github.com/aodn/data-access-service/blob/main/data_access_service/core/routes/data.py"
click node_estimate_route "https://github.com/aodn/data-access-service/blob/main/data_access_service/core/routes/estimation_index.py"
click node_api "https://github.com/aodn/data-access-service/blob/main/data_access_service/core/api.py"
click node_site_service "https://github.com/aodn/data-access-service/blob/main/data_access_service/sites/site_feature_service.py"
click node_site_repo "https://github.com/aodn/data-access-service/blob/main/data_access_service/sites/sites_repository.py"
click node_sites_db "https://github.com/aodn/data-access-service/blob/main/data_access_service/core/duckdbclient.py"
click node_tile_products "https://github.com/aodn/data-access-service/tree/main/data_access_service/tiler/services/product"
click node_tile_render "https://github.com/aodn/data-access-service/tree/main/data_access_service/tiler/services/rendering"
click node_tile_store "https://github.com/aodn/data-access-service/tree/main/data_access_service/tiler/services/store"
click node_tile_db "https://github.com/aodn/data-access-service/blob/main/data_access_service/core/duckdbclient.py"
click node_subset "https://github.com/aodn/data-access-service/blob/main/data_access_service/batch/subsetting/subsetting_main.py"
click node_subset_tasks "https://github.com/aodn/data-access-service/tree/main/data_access_service/batch/subsetting/tasks"
click node_estimation "https://github.com/aodn/data-access-service/tree/main/data_access_service/batch/estimation"
click node_pmtiles "https://github.com/aodn/data-access-service/blob/main/data_access_service/batch/pmtiles/generator.py"
click node_tile_batch "https://github.com/aodn/data-access-service/blob/main/data_access_service/batch/tiler/generator.py"
click node_sites_refresh "https://github.com/aodn/data-access-service/blob/main/data_access_service/batch/sites_parquet/refresher.py"
click node_scheduler "https://github.com/aodn/data-access-service/blob/main/data_access_service/core/scheduler.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_server,node_api_router,node_tiler_router,node_metadata,node_data_routes,node_estimate_route,node_api toneBlue
class node_site_service,node_site_repo,node_sites_db,node_s3 toneAmber
class node_tile_products,node_tile_render,node_tile_store,node_tile_db,node_consumer toneMint
class node_pmtiles,node_tile_batch,node_sites_refresh,node_scheduler toneRose
class node_subset,node_subset_tasks,node_estimation toneIndigo
```

## Components

### HTTP API

- [FastAPI server](https://github.com/aodn/data-access-service/blob/main/data_access_service/server.py): `data_access_service/server.py`
- [Data API routes](https://github.com/aodn/data-access-service/tree/main/data_access_service/core/routes): `data_access_service/core/routes`
- [Tiler routes](https://github.com/aodn/data-access-service/tree/main/data_access_service/core/tiler_routes): `data_access_service/core/tiler_routes`
- [Metadata](https://github.com/aodn/data-access-service/blob/main/data_access_service/core/routes/metadata.py): `data_access_service/core/routes/metadata.py`
- [Data endpoints](https://github.com/aodn/data-access-service/blob/main/data_access_service/core/routes/data.py): `data_access_service/core/routes/data.py`
- [Estimate endpoint](https://github.com/aodn/data-access-service/blob/main/data_access_service/core/routes/estimation_index.py): `data_access_service/core/routes/estimation_index.py`
- [API service](https://github.com/aodn/data-access-service/blob/main/data_access_service/core/api.py): `data_access_service/core/api.py`

### Site Observations

- [Site feature service](https://github.com/aodn/data-access-service/blob/main/data_access_service/sites/site_feature_service.py): `data_access_service/sites/site_feature_service.py`
- [Site repositories](https://github.com/aodn/data-access-service/blob/main/data_access_service/sites/sites_repository.py): `data_access_service/sites/sites_repository.py`
- [Sites DuckDB](https://github.com/aodn/data-access-service/blob/main/data_access_service/core/duckdbclient.py): `data_access_service/core/duckdbclient.py`

### Tile Services

- [Tile products](https://github.com/aodn/data-access-service/tree/main/data_access_service/tiler/services/product): `data_access_service/tiler/services/product`
- [Tile rendering](https://github.com/aodn/data-access-service/tree/main/data_access_service/tiler/services/rendering): `data_access_service/tiler/services/rendering`
- [Tile data store](https://github.com/aodn/data-access-service/tree/main/data_access_service/tiler/services/store): `data_access_service/tiler/services/store`
- [Tiler DuckDB](https://github.com/aodn/data-access-service/blob/main/data_access_service/core/duckdbclient.py): `data_access_service/core/duckdbclient.py`

### Batch Processing

- [PMTiles generation](https://github.com/aodn/data-access-service/blob/main/data_access_service/batch/pmtiles/generator.py): `data_access_service/batch/pmtiles/generator.py`
- [Raster tiling](https://github.com/aodn/data-access-service/blob/main/data_access_service/batch/tiler/generator.py): `data_access_service/batch/tiler/generator.py`
- [Sites refresh](https://github.com/aodn/data-access-service/blob/main/data_access_service/batch/sites_parquet/refresher.py): `data_access_service/batch/sites_parquet/refresher.py`
- [Task scheduler](https://github.com/aodn/data-access-service/blob/main/data_access_service/core/scheduler.py): `data_access_service/core/scheduler.py`

### Data Access

- [Subset processing](https://github.com/aodn/data-access-service/blob/main/data_access_service/batch/subsetting/subsetting_main.py): `data_access_service/batch/subsetting/subsetting_main.py`
- [Subset workers](https://github.com/aodn/data-access-service/tree/main/data_access_service/batch/subsetting/tasks): `data_access_service/batch/subsetting/tasks`
- [Estimate index](https://github.com/aodn/data-access-service/tree/main/data_access_service/batch/estimation): `data_access_service/batch/estimation`

### Other components

- **API consumer**
- **S3 storage**

## Connections

- API consumer → FastAPI server: sends requests
- FastAPI server → Data API routes: mounts routes
- FastAPI server → Tiler routes: mounts routes
- FastAPI server → API service: initializes
- FastAPI server → Site repositories: builds repositories
- FastAPI server → Task scheduler: starts
- Data API routes → Metadata: serves metadata
- Data API routes → Data endpoints: serves data
- Data API routes → Estimate endpoint: serves estimates
- Site feature service → Site repositories: queries
- Site repositories → Sites DuckDB: executes queries
- FastAPI server → Tiler DuckDB: initializes
- FastAPI server → Tile data store: initializes
- Tiler routes → Tile products: routes product requests
- Tile data store → Tiler DuckDB: queries
- Data endpoints → Subset processing: supports subsetting
- Subset processing → Subset workers: dispatches work
- Estimate endpoint → Estimate index: uses index
- Raster tiling → S3 storage: writes tile data
- PMTiles generation → S3 storage: writes tile archives
- Sites refresh → S3 storage: refreshes datasets