How nhsdigital/national-document-repository works
Project Type & Purpose
• This is a full-stack, AWS-centric document repository service (“national-document-repository”) consisting of:
• A React-based web UI (in app/) that end-users interact with.
• A set of Python AWS Lambda functions (in lambdas/) providing REST and FHIR endpoints, background processing (bulk-upload pipelines, PDF stitching, virus scan handling), and integrations.
• Automated CI/CD pipelines (.github/workflows) for testing, scanning, and deploying UI and Lambdas across sandbox, pre-prod, and prod.
• End-to-end tests and performance scripts.
Key Architectural Layers & Directories
• Frontend (app/): React SPA, Cypress E2E tests, Dockerfile, env configuration, public assets.
• Backend/Serverless (lambdas/):
• handlers/ – Lambda entry points for each API route and background task.
• services/ – Business logic, integrations (S3, DynamoDB, SQS, SNS, OAuth, PDS, ODS, FHIR, NRL).
• repositories/ – Data access to DynamoDB, S3, SQS.
• models/, utils/, enums/ – Shared types, constants, helpers.
• tests/ – Unit & E2E tests.
• Infrastructure & Integration:
• apim/specification.yaml – API definitions.
• scripts/ – helper scripts (e.g., deploy certs, run tests).
• performance/ – tooling for load data generation.
• CI/CD (.github/workflows): Linting, unit tests, Vitest, Cypress E2E, SBOM scans, deploy UI + Lambdas.
Main Components
a. User Layer
• Browser (React SPA)
• Auth via NHS OAuth (external)
b. Presentation/API Layer
• Amazon CloudFront (optional, serving SPA)
• Amazon API Gateway (REST & FHIR endpoints)
c. Compute Layer (AWS Lambda functions)
• Authoriser & Auth callbacks
• DocumentReference CRUD, Search, Presign Upload
• Bulk Upload pipeline (handlers + SQS queues + processors)
• Virus scan notification handler
• PDF stitching handler
• FHIR handlers (FHIR DocumentReference search/upload)
• Background ECS task for data_collection
d. Data Layer
• Amazon S3 (raw uploads, stitched PDFs)
• Amazon DynamoDB (metadata, access audit, job statuses)
• Amazon SQS (bulk-upload queue, PDF-stitching queue, MNS queue)
• Amazon SNS (notifications, alerts)
• Amazon SSM Parameter Store (config, OAuth secrets)
e. Observability
• Amazon CloudWatch Logs & Alarms
• SNS topics for alarm notifications
f. External Integrations
• NHS OAuth token endpoint
• PDS (Patient Demographic Service)
• ODS (Organisation Data Service)
• NRL (National Record Locator)
• Teams/Slack for feedback notifications
Relationships & Data Flows
• User → React SPA → API Gateway → Lambda
• Lambda → DynamoDB / S3 / SQS / SNS / SSM
• S3 upload triggers SQS → Lambda (virus scan result) → next processing
• Bulk upload: user uploads CSV → Lambda places messages on SQS → ECS Data-collection task consumes SQS → Dynamo/S3 writes
• API Gateway FHIR endpoints → FHIR Lambdas → DynamoDB, S3
• CloudWatch metrics generate Alarm → SNS → Email/Teams
Architectural Patterns & Principles
• Serverless microservices: Each logical operation in its own Lambda handler.
• Event-driven processing: S3 events → SQS → Lambdas; SQS between bulk-upload steps.
• Separation of concerns: handlers vs services vs repositories vs utils.
• Infrastructure as code / declarative API spec (APIM YAML).
• CI/CD gating with unit, Vitest, Cypress, CodeQL, SBOM.