ankit5641/vanco-backend

1. This repo is an asynchronous document-processing backend, split into two runtime processes: an HTTP API and a background worker. The API accepts uploads and creates jobs; the worker consumes queued jobs and performs text extraction.

2. The main request path starts in src/server.js and src/app.js, then routes under src/routes/. Uploads go through upload.middleware.js before reaching upload.controller.js. Job lookup and deletion go through job.controller.js, with UUID validation in validate.middleware.js.

3. The API is intentionally thin. Controllers orchestrate higher-level flow, while persistence and cloud interactions are pushed into src/models/ and src/services/. That separation is important for the diagram: HTTP layer, application logic, data access, and AWS integrations are distinct boundaries.

4. Job state is stored in PostgreSQL via Prisma. src/prisma/schema.prisma and the migrations define the jobs table, while src/config/database.js provides the Prisma client and src/models/job.model.js centralizes database queries. This database is the source of truth for job status, extracted text, confidence score, and retry metadata.

5. File storage lives in S3. src/services/s3.service.js handles upload, download, and delete operations, so the file binary is not kept in the API or database. The job record stores metadata and the S3 object is the durable document copy.

6. Asynchronous decoupling is handled by SQS. src/jobs/jobQueue.js and src/services/sqs.service.js enqueue work after a successful upload. The API returns 202 Accepted with a jobId immediately, and the worker later polls the queue using long polling.

7. The worker is a separate process under src/worker/worker.js and src/worker/processor.js. It receives a queue message, downloads the file from S3, runs text extraction, updates PostgreSQL, and then acknowledges the SQS message. This is the core “background processing” boundary in the architecture.

8. Text extraction is local, not AWS Textract. Despite the service name src/services/textract.service.js, the README indicates it uses pdf-parse for text-based PDFs and Tesseract.js for images, with sharp preprocessing for better OCR quality. That means extraction happens inside the worker process, not through an external paid API.

9. Notifications and observability are handled by SNS and CloudWatch. src/services/sns.service.js publishes job-completed or failed events, and src/services/cloudwatch.service.js plus src/utils/logger.js provide structured logging. These are side effects of processing, not part of the primary user-facing flow.

10. Configuration is centralized. src/config/env.js validates environment variables, src/config/aws.js builds AWS clients, and src/services/ssm.service.js fetches runtime parameters from AWS SSM Parameter Store. This keeps secrets and deployment-specific values outside application code.

11. Cross-cutting safeguards live in src/utils/. circuitBreaker.js prevents repeated calls into failing dependencies, while helpers.js contains shared utilities like UUID generation and backoff logic. These are supporting concerns around reliability rather than standalone subsystems.

12. Deletion and retrieval are separate from ingestion. GET /api/result/:jobId reads job status and text from PostgreSQL, GET /api/jobs returns lightweight job listings, and DELETE /api/jobs/:jobId removes the S3 object before deleting the database record. That order matters because the file is treated as the durable asset and the database as the index of record.

13. Deployment is containerized with Docker and docker-compose.yml. The compose setup runs PostgreSQL, the API, migrations, and the worker as distinct services, which matches the architecture in code: a web-facing app process, a background consumer, and shared infrastructure services.

14. For an architecture diagram, the essential data flow is: Client → API upload → S3 + PostgreSQL job row → SQS queue → worker → S3 download → OCR/PDF parsing → PostgreSQL update → optional SNS notification → client polls result endpoint. The key boundary is that upload and processing are fully decoupled across two processes and a message queue.