# nhsdigital/national-document-repository architecture

> Architecture diagram of the GitHub repository nhsdigital/national-document-repository, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-11-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nhsdigital/national-document-repository
- Repository: https://github.com/nhsdigital/national-document-repository

## Overview

##### Project Type & Purpose
- This is a full-stack, AWS-centric document repository service (“national-document-repository”) consisting of:
- A React-based web UI (in app/) that end-users interact with.
- A set of Python AWS Lambda functions (in lambdas/) providing REST and FHIR endpoints, background processing (bulk-upload pipelines, PDF stitching, virus scan handling), and integrations.
- Automated CI/CD pipelines (.github/workflows) for testing, scanning, and deploying UI and Lambdas across sandbox, pre-prod, and prod.
- End-to-end tests and performance scripts.

##### Key Architectural Layers & Directories
- Frontend (app/): React SPA, Cypress E2E tests, Dockerfile, env configuration, public assets.
- Backend/Serverless (lambdas/):
- handlers/ – Lambda entry points for each API route and background task.
- services/ – Business logic, integrations (S3, DynamoDB, SQS, SNS, OAuth, PDS, ODS, FHIR, NRL).
- repositories/ – Data access to DynamoDB, S3, SQS.
- models/, utils/, enums/ – Shared types, constants, helpers.
- tests/ – Unit & E2E tests.
- Infrastructure & Integration:
- apim/specification.yaml – API definitions.
- scripts/ – helper scripts (e.g., deploy certs, run tests).
- performance/ – tooling for load data generation.
- CI/CD (.github/workflows): Linting, unit tests, Vitest, Cypress E2E, SBOM scans, deploy UI + Lambdas.

##### Main Components
a. User Layer
- Browser (React SPA)
- Auth via NHS OAuth (external)
b. Presentation/API Layer
- Amazon CloudFront (optional, serving SPA)
- Amazon API Gateway (REST & FHIR endpoints)
c. Compute Layer (AWS Lambda functions)
- Authoriser & Auth callbacks
- DocumentReference CRUD, Search, Presign Upload
- Bulk Upload pipeline (handlers + SQS queues + processors)
- Virus scan notification handler
- PDF stitching handler
- FHIR handlers (FHIR DocumentReference search/upload)
- Background ECS task for data_collection
d. Data Layer
- Amazon S3 (raw uploads, stitched PDFs)
- Amazon DynamoDB (metadata, access audit, job statuses)
- Amazon SQS (bulk-upload queue, PDF-stitching queue, MNS queue)
- Amazon SNS (notifications, alerts)
- Amazon SSM Parameter Store (config, OAuth secrets)
e. Observability
- Amazon CloudWatch Logs & Alarms
- SNS topics for alarm notifications
f. External Integrations
- NHS OAuth token endpoint
- PDS (Patient Demographic Service)
- ODS (Organisation Data Service)
- NRL (National Record Locator)
- Teams/Slack for feedback notifications

##### Relationships & Data Flows
- User → React SPA → API Gateway → Lambda
- Lambda → DynamoDB / S3 / SQS / SNS / SSM
- S3 upload triggers SQS → Lambda (virus scan result) → next processing
- Bulk upload: user uploads CSV → Lambda places messages on SQS → ECS Data-collection task consumes SQS → Dynamo/S3 writes
- API Gateway FHIR endpoints → FHIR Lambdas → DynamoDB, S3
- CloudWatch metrics generate Alarm → SNS → Email/Teams

##### Architectural Patterns & Principles
- Serverless microservices: Each logical operation in its own Lambda handler.
- Event-driven processing: S3 events → SQS → Lambdas; SQS between bulk-upload steps.
- Separation of concerns: handlers vs services vs repositories vs utils.
- Infrastructure as code / declarative API spec (APIM YAML).
- CI/CD gating with unit, Vitest, Cypress, CodeQL, SBOM.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User Layer
    subgraph "User Layer"
        Browser["React SPA (Browser)"]:::ui
    end

    %% Presentation/API Layer
    subgraph "Presentation/API Layer"
        CloudFront["Amazon CloudFront"]:::compute
        APISpec["API Specification"]:::artifact
        APIGWRest["API Gateway (REST)"]:::api
        APIGWFhir["API Gateway (FHIR)"]:::api
    end

    %% Compute Layer
    subgraph "Compute Layer"
        subgraph "Lambda Handlers"
            AuthLambda["Auth & Callbacks Lambdas"]:::compute
            DRLambda["DocumentReference CRUD/Search Lambdas"]:::compute
            PresignLambda["Presign Upload Lambda"]:::compute
            BulkLambda["Bulk Upload Pipeline Lambdas"]:::compute
            VirusLambda["Virus Scan Handlers"]:::compute
            StitchLambda["PDF Stitching Handlers"]:::compute
            FhirLambda["FHIR DocumentReference Lambdas"]:::compute
            FeedbackLambda["Feedback & Alerting Lambdas"]:::compute
        end
        ECS["ECS Data Collection Task"]:::compute
    end

    %% Data Layer
    subgraph "Data Layer"
        S3["Amazon S3"]:::storage
        DynamoDB["Amazon DynamoDB"]:::storage
        SQS1["Bulk Upload SQS Queue"]:::queue
        SQS2["PDF Stitch SQS Queue"]:::queue
        SNS["Amazon SNS Topics"]:::queue
        SSM["SSM Parameter Store"]:::store
    end

    %% Observability
    subgraph "Observability"
        CW["CloudWatch Logs & Alarms"]:::monitor
        AlarmSNS["Alarm SNS Topic"]:::queue
    end

    %% External Integrations
    subgraph "External Integrations"
        OAuth["NHS OAuth"]:::external
        PDS["PDS"]:::external
        ODS["ODS"]:::external
        NRL["NRL"]:::external
        Teams["Teams/Slack"]:::external
    end

    %% Connections - User to API
    Browser --> CloudFront --> APIGWRest
    Browser --> CloudFront --> APIGWFhir
    APISpec --> APIGWRest

    %% API to Compute
    APIGWRest --> AuthLambda
    APIGWRest --> DRLambda
    APIGWRest --> PresignLambda
    APIGWRest --> BulkLambda
    APIGWRest --> FhirLambda
    APIGWRest --> FeedbackLambda

    %% Lambda to Data
    AuthLambda -->|config| SSM
    DRLambda --> DynamoDB
    DRLambda --> S3
    PresignLambda --> S3
    BulkLambda --> SQS1
    VirusLambda --> DynamoDB
    VirusLambda --> S3
    StitchLambda --> S3
    FhirLambda --> DynamoDB
    FhirLambda --> S3
    FeedbackLambda --> SNS

    %% Event-driven flows
    S3 -->|upload event| SQS2
    SQS2 --> VirusLambda
    VirusLambda --> StitchLambda

    SQS1 --> ECS
    ECS --> DynamoDB
    ECS --> S3

    %% Observability flows
    AuthLambda --> CW
    DRLambda --> CW
    BulkLambda --> CW
    VirusLambda --> CW
    StitchLambda --> CW
    FhirLambda --> CW
    ECS --> CW
    CW --> AlarmSNS --> Teams

    %% External integrations from services (implicit via Lambdas)
    AuthLambda --> OAuth
    DRLambda --> PDS
    DRLambda --> ODS
    DRLambda --> NRL

    %% Click Events
    click Browser "https://github.com/nhsdigital/national-document-repository/tree/main/app/"
    click APISpec "https://github.com/nhsdigital/national-document-repository/blob/main/apim/specification.yaml"
    click AuthLambda "https://github.com/nhsdigital/national-document-repository/tree/main/lambdas/handlers/"
    click DRLambda "https://github.com/nhsdigital/national-document-repository/tree/main/lambdas/handlers/"
    click PresignLambda "https://github.com/nhsdigital/national-document-repository/blob/main/lambdas/handlers/edge_presign_handler.py"
    click BulkLambda "https://github.com/nhsdigital/national-document-repository/tree/main/lambdas/handlers/"
    click VirusLambda "https://github.com/nhsdigital/national-document-repository/tree/main/lambdas/handlers/"
    click StitchLambda "https://github.com/nhsdigital/national-document-repository/tree/main/lambdas/handlers/"
    click FhirLambda "https://github.com/nhsdigital/national-document-repository/tree/main/lambdas/handlers/"
    click FeedbackLambda "https://github.com/nhsdigital/national-document-repository/tree/main/lambdas/handlers/"
    click ECS "https://github.com/nhsdigital/national-document-repository/blob/main/lambdas/ecs/data_collection/main.py"
    click .github "https://github.com/nhsdigital/national-document-repository/tree/main/ .github/workflows/"

    %% Styles
    classDef ui fill:#a2d9ce,stroke:#0b5345,color:#0b5345
    classDef compute fill:#85c1e9,stroke:#1b4f72,color:#1b4f72
    classDef storage fill:#1abc9c,stroke:#145a32,color:#145a32
    classDef queue fill:#f5b041,stroke:#b9770e,color:#b9770e
    classDef api fill:#af7ac5,stroke:#512e5f,color:#512e5f
    classDef artifact fill:#d5dbdb,stroke:#566573,color:#566573
    classDef external fill:#d6dbdf,stroke:#5d6d7e,color:#5d6d7e
    classDef monitor fill:#f7dc6f,stroke:#7d6608,color:#7d6608
    classDef store fill:#aed6f1,stroke:#1f618d,color:#1f618d
```
