# ankit5641/vanco-backend architecture

> Architecture diagram of the GitHub repository ankit5641/vanco-backend, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ankit5641/vanco-backend
- Repository: https://github.com/ankit5641/vanco-backend

## Overview

1. This repo is an asynchronous document-processing backend, split into two runtime processes: an HTTP API and a background worker. The API accepts uploads and creates jobs; the worker consumes queued jobs and performs text extraction.

2. The main request path starts in `src/server.js` and `src/app.js`, then routes under `src/routes/`. Uploads go through `upload.middleware.js` before reaching `upload.controller.js`. Job lookup and deletion go through `job.controller.js`, with UUID validation in `validate.middleware.js`.

3. The API is intentionally thin. Controllers orchestrate higher-level flow, while persistence and cloud interactions are pushed into `src/models/` and `src/services/`. That separation is important for the diagram: HTTP layer, application logic, data access, and AWS integrations are distinct boundaries.

4. Job state is stored in PostgreSQL via Prisma. `src/prisma/schema.prisma` and the migrations define the jobs table, while `src/config/database.js` provides the Prisma client and `src/models/job.model.js` centralizes database queries. This database is the source of truth for job status, extracted text, confidence score, and retry metadata.

5. File storage lives in S3. `src/services/s3.service.js` handles upload, download, and delete operations, so the file binary is not kept in the API or database. The job record stores metadata and the S3 object is the durable document copy.

6. Asynchronous decoupling is handled by SQS. `src/jobs/jobQueue.js` and `src/services/sqs.service.js` enqueue work after a successful upload. The API returns `202 Accepted` with a `jobId` immediately, and the worker later polls the queue using long polling.

7. The worker is a separate process under `src/worker/worker.js` and `src/worker/processor.js`. It receives a queue message, downloads the file from S3, runs text extraction, updates PostgreSQL, and then acknowledges the SQS message. This is the core “background processing” boundary in the architecture.

8. Text extraction is local, not AWS Textract. Despite the service name `src/services/textract.service.js`, the README indicates it uses `pdf-parse` for text-based PDFs and Tesseract.js for images, with `sharp` preprocessing for better OCR quality. That means extraction happens inside the worker process, not through an external paid API.

9. Notifications and observability are handled by SNS and CloudWatch. `src/services/sns.service.js` publishes job-completed or failed events, and `src/services/cloudwatch.service.js` plus `src/utils/logger.js` provide structured logging. These are side effects of processing, not part of the primary user-facing flow.

10. Configuration is centralized. `src/config/env.js` validates environment variables, `src/config/aws.js` builds AWS clients, and `src/services/ssm.service.js` fetches runtime parameters from AWS SSM Parameter Store. This keeps secrets and deployment-specific values outside application code.

11. Cross-cutting safeguards live in `src/utils/`. `circuitBreaker.js` prevents repeated calls into failing dependencies, while `helpers.js` contains shared utilities like UUID generation and backoff logic. These are supporting concerns around reliability rather than standalone subsystems.

12. Deletion and retrieval are separate from ingestion. `GET /api/result/:jobId` reads job status and text from PostgreSQL, `GET /api/jobs` returns lightweight job listings, and `DELETE /api/jobs/:jobId` removes the S3 object before deleting the database record. That order matters because the file is treated as the durable asset and the database as the index of record.

13. Deployment is containerized with Docker and `docker-compose.yml`. The compose setup runs PostgreSQL, the API, migrations, and the worker as distinct services, which matches the architecture in code: a web-facing app process, a background consumer, and shared infrastructure services.

14. For an architecture diagram, the essential data flow is: Client → API upload → S3 + PostgreSQL job row → SQS queue → worker → S3 download → OCR/PDF parsing → PostgreSQL update → optional SNS notification → client polls result endpoint. The key boundary is that upload and processing are fully decoupled across two processes and a message queue.

Main entry points: [`src/worker/processor.js`](https://github.com/ankit5641/vanco-backend/blob/main/src/worker/processor.js) (Processor), [`src/config/aws.js`](https://github.com/ankit5641/vanco-backend/blob/main/src/config/aws.js) (AWS Config), [`src/controllers/upload.controller.js`](https://github.com/ankit5641/vanco-backend/blob/main/src/controllers/upload.controller.js) (Upload Ctrl), [`src/models/job.model.js`](https://github.com/ankit5641/vanco-backend/blob/main/src/models/job.model.js) (Job Model)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_grp_api["HTTP API"]
  node_n_server["Server<br/>entrypoint<br/>[server.js]"]
  node_n_app["App<br/>[app.js]"]
  node_n_routes["Routes<br/>route map<br/>[index.js]"]
  node_n_upload_mw["Upload MW<br/>middleware"]
  node_n_validate_mw["Validate MW<br/>middleware"]
  node_n_upload_ctrl["Upload Ctrl<br/>controller"]
  node_n_job_ctrl["Job Ctrl<br/>controller<br/>[job.controller.js]"]
end

subgraph group_grp_worker["Worker"]
  node_n_worker["Worker<br/>entrypoint<br/>[worker.js]"]
  node_n_processor["Processor<br/>pipeline<br/>[processor.js]"]
  node_n_textract["Extractor<br/>ocr/pdf"]
end

subgraph group_grp_data["Data & Infra"]
  node_n_job_model["Job Model<br/>db access<br/>[job.model.js]"]
  node_n_prisma["Prisma Schema<br/>[schema.prisma]"]
  node_n_db[("Database Client<br/>persistence<br/>[database.js]")]
  node_n_s3[("S3 Service<br/>storage<br/>[s3.service.js]")]
  node_n_sqs["SQS Service<br/>queue<br/>[sqs.service.js]"]
  node_n_jobqueue["Job Queue<br/>queue adapter<br/>[jobQueue.js]"]
  node_n_sns["SNS Service<br/>events<br/>[sns.service.js]"]
end

subgraph group_grp_shared["Shared Core"]
  node_n_cloudwatch["CloudWatch<br/>observability"]
  node_n_logger["Logger<br/>logging<br/>[logger.js]"]
  node_n_env["Env Config<br/>[env.js]"]
  node_n_aws["AWS Config<br/>[aws.js]"]
  node_n_ssm["SSM Service<br/>config fetch<br/>[ssm.service.js]"]
  node_n_utils["Helpers<br/>[helpers.js]"]
end

node_n_server -->|"starts"| node_n_app
node_n_app -->|"mounts"| node_n_routes
node_n_routes -->|"upload path"| node_n_upload_mw
node_n_routes -->|"validation path"| node_n_validate_mw
node_n_upload_mw -->|"forwards"| node_n_upload_ctrl
node_n_validate_mw -->|"guards"| node_n_job_ctrl
node_n_upload_ctrl -->|"stores file"| node_n_s3
node_n_upload_ctrl -->|"creates job"| node_n_job_model
node_n_job_ctrl -->|"reads/writes"| node_n_job_model
node_n_job_model -->|"uses"| node_n_db
node_n_db -->|"maps schema"| node_n_prisma
node_n_upload_ctrl -->|"enqueues"| node_n_jobqueue
node_n_jobqueue -->|"sends"| node_n_sqs
node_n_worker -->|"runs"| node_n_processor
node_n_processor -->|"polls"| node_n_sqs
node_n_processor -->|"downloads"| node_n_s3
node_n_processor -->|"extracts text"| node_n_textract
node_n_processor -->|"updates status"| node_n_job_model
node_n_processor -->|"publishes event"| node_n_sns
node_n_sns -->|"observes"| node_n_cloudwatch
node_n_cloudwatch -->|"uses"| node_n_logger
node_n_env -->|"feeds"| node_n_aws
node_n_aws -->|"builds clients"| node_n_s3
node_n_aws -->|"builds clients"| node_n_sqs
node_n_aws -->|"builds clients"| node_n_sns
node_n_aws -->|"supports"| node_n_ssm
node_n_utils -.->|"helpers"| node_n_jobqueue
node_n_utils -.->|"helpers"| node_n_processor

click node_n_server "https://github.com/ankit5641/vanco-backend/blob/main/src/server.js"
click node_n_app "https://github.com/ankit5641/vanco-backend/blob/main/src/app.js"
click node_n_routes "https://github.com/ankit5641/vanco-backend/blob/main/src/routes/index.js"
click node_n_upload_mw "https://github.com/ankit5641/vanco-backend/blob/main/src/middleware/upload.middleware.js"
click node_n_validate_mw "https://github.com/ankit5641/vanco-backend/blob/main/src/middleware/validate.middleware.js"
click node_n_upload_ctrl "https://github.com/ankit5641/vanco-backend/blob/main/src/controllers/upload.controller.js"
click node_n_job_ctrl "https://github.com/ankit5641/vanco-backend/blob/main/src/controllers/job.controller.js"
click node_n_job_model "https://github.com/ankit5641/vanco-backend/blob/main/src/models/job.model.js"
click node_n_prisma "https://github.com/ankit5641/vanco-backend/blob/main/src/prisma/schema.prisma"
click node_n_db "https://github.com/ankit5641/vanco-backend/blob/main/src/config/database.js"
click node_n_s3 "https://github.com/ankit5641/vanco-backend/blob/main/src/services/s3.service.js"
click node_n_sqs "https://github.com/ankit5641/vanco-backend/blob/main/src/services/sqs.service.js"
click node_n_jobqueue "https://github.com/ankit5641/vanco-backend/blob/main/src/jobs/jobQueue.js"
click node_n_worker "https://github.com/ankit5641/vanco-backend/blob/main/src/worker/worker.js"
click node_n_processor "https://github.com/ankit5641/vanco-backend/blob/main/src/worker/processor.js"
click node_n_textract "https://github.com/ankit5641/vanco-backend/blob/main/src/services/textract.service.js"
click node_n_sns "https://github.com/ankit5641/vanco-backend/blob/main/src/services/sns.service.js"
click node_n_cloudwatch "https://github.com/ankit5641/vanco-backend/blob/main/src/services/cloudwatch.service.js"
click node_n_logger "https://github.com/ankit5641/vanco-backend/blob/main/src/utils/logger.js"
click node_n_env "https://github.com/ankit5641/vanco-backend/blob/main/src/config/env.js"
click node_n_aws "https://github.com/ankit5641/vanco-backend/blob/main/src/config/aws.js"
click node_n_ssm "https://github.com/ankit5641/vanco-backend/blob/main/src/services/ssm.service.js"
click node_n_utils "https://github.com/ankit5641/vanco-backend/blob/main/src/utils/helpers.js"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_n_server,node_n_app,node_n_routes,node_n_upload_mw,node_n_validate_mw,node_n_upload_ctrl,node_n_job_ctrl toneBlue
class node_n_worker,node_n_processor,node_n_textract toneAmber
class node_n_job_model,node_n_prisma,node_n_db,node_n_s3,node_n_sqs,node_n_jobqueue,node_n_sns toneMint
class node_n_cloudwatch,node_n_logger,node_n_env,node_n_aws,node_n_ssm,node_n_utils toneRose
```

## Components

### HTTP API

Request handling and job management

- [Server](https://github.com/ankit5641/vanco-backend/blob/main/src/server.js): entrypoint · Process bootstrap for the API · `src/server.js`
- [App](https://github.com/ankit5641/vanco-backend/blob/main/src/app.js): http app · Wires middleware, routes, and handlers · `src/app.js`
- [Routes](https://github.com/ankit5641/vanco-backend/blob/main/src/routes/index.js): route map · Top-level HTTP routing · `src/routes/index.js`
- [Upload MW](https://github.com/ankit5641/vanco-backend/blob/main/src/middleware/upload.middleware.js): middleware · Handles request upload parsing · `src/middleware/upload.middleware.js`
- [Validate MW](https://github.com/ankit5641/vanco-backend/blob/main/src/middleware/validate.middleware.js): middleware · Validates job identifiers and request inputs · `src/middleware/validate.middleware.js`
- [Upload Ctrl](https://github.com/ankit5641/vanco-backend/blob/main/src/controllers/upload.controller.js): controller · Creates jobs from uploaded documents · `src/controllers/upload.controller.js`
- [Job Ctrl](https://github.com/ankit5641/vanco-backend/blob/main/src/controllers/job.controller.js): controller · Reads and deletes job records · `src/controllers/job.controller.js`

### Worker

Background queue consumer and document processing

- [Worker](https://github.com/ankit5641/vanco-backend/blob/main/src/worker/worker.js): entrypoint · Background consumer process · `src/worker/worker.js`
- [Processor](https://github.com/ankit5641/vanco-backend/blob/main/src/worker/processor.js): pipeline · Consumes messages and runs document flow · `src/worker/processor.js`
- [Extractor](https://github.com/ankit5641/vanco-backend/blob/main/src/services/textract.service.js): ocr/pdf · Local PDF parsing and OCR preprocessing · `src/services/textract.service.js`

### Data & Infra

Persistence, messaging, and cloud integrations

- [Job Model](https://github.com/ankit5641/vanco-backend/blob/main/src/models/job.model.js): db access · Centralized job queries and updates · `src/models/job.model.js`
- [Prisma Schema](https://github.com/ankit5641/vanco-backend/blob/main/src/prisma/schema.prisma): schema · Job table and relational model · `src/prisma/schema.prisma`
- [Database Client](https://github.com/ankit5641/vanco-backend/blob/main/src/config/database.js): persistence · Prisma-backed PostgreSQL connection · `src/config/database.js`
- [S3 Service](https://github.com/ankit5641/vanco-backend/blob/main/src/services/s3.service.js): storage · Uploads, downloads, and deletes document objects · `src/services/s3.service.js`
- [SQS Service](https://github.com/ankit5641/vanco-backend/blob/main/src/services/sqs.service.js): queue · Enqueues and receives background work · `src/services/sqs.service.js`
- [Job Queue](https://github.com/ankit5641/vanco-backend/blob/main/src/jobs/jobQueue.js): queue adapter · Application-level enqueue boundary · `src/jobs/jobQueue.js`
- [SNS Service](https://github.com/ankit5641/vanco-backend/blob/main/src/services/sns.service.js): events · Publishes job completion and failure events · `src/services/sns.service.js`

### Shared Core

Configuration, logging, and cross-cutting utilities

- [CloudWatch](https://github.com/ankit5641/vanco-backend/blob/main/src/services/cloudwatch.service.js): observability · Operational logging and metrics sink · `src/services/cloudwatch.service.js`
- [Logger](https://github.com/ankit5641/vanco-backend/blob/main/src/utils/logger.js): logging · Structured application logging · `src/utils/logger.js`
- [Env Config](https://github.com/ankit5641/vanco-backend/blob/main/src/config/env.js): config · Validates runtime environment variables · `src/config/env.js`
- [AWS Config](https://github.com/ankit5641/vanco-backend/blob/main/src/config/aws.js): config · Creates AWS clients and shared settings · `src/config/aws.js`
- [SSM Service](https://github.com/ankit5641/vanco-backend/blob/main/src/services/ssm.service.js): config fetch · Loads runtime parameters from Parameter Store · `src/services/ssm.service.js`
- [Helpers](https://github.com/ankit5641/vanco-backend/blob/main/src/utils/helpers.js): utility · Shared utilities such as IDs and backoff · `src/utils/helpers.js`

## Connections

- Server → App: starts · Bootstraps the HTTP application
- App → Routes: mounts · Registers route groups
- Routes → Upload MW: upload path · Prepares upload requests
- Routes → Validate MW: validation path · Checks job IDs and request data
- Upload MW → Upload Ctrl: forwards · Passes parsed upload data to controller
- Validate MW → Job Ctrl: guards · Protects job lookup and deletion endpoints
- Upload Ctrl → S3 Service: stores file · Writes document binary to durable object storage
- Upload Ctrl → Job Model: creates job · Persists initial job record
- Job Ctrl → Job Model: reads/writes · Loads status and removes job rows
- Job Model → Database Client: uses · Executes Prisma database operations
- Database Client → Prisma Schema: maps schema · Backed by the Prisma job model
- Upload Ctrl → Job Queue: enqueues · Schedules asynchronous processing
- Job Queue → SQS Service: sends · Pushes work to the queue service
- Worker → Processor: runs · Main background processing loop
- Processor → SQS Service: polls · Receives queued job messages
- Processor → S3 Service: downloads · Fetches source documents for processing
- Processor → Extractor: extracts text · Performs PDF parsing and OCR
- Processor → Job Model: updates status · Writes extracted text and retry metadata
- Processor → SNS Service: publishes event · Emits completion or failure notifications
- SNS Service → CloudWatch: observes · Side-effect telemetry and logging
- CloudWatch → Logger: uses · Structured log emission
- Env Config → AWS Config: feeds · Provides validated configuration
- AWS Config → S3 Service: builds clients · Configures AWS storage access
- AWS Config → SQS Service: builds clients · Configures queue access
- AWS Config → SNS Service: builds clients · Configures notification access
- AWS Config → SSM Service: supports · Shared AWS runtime access
- Helpers → Job Queue: helpers · Supports retry and ID generation
- Helpers → Processor: helpers · Supports backoff and shared logic