# srajasimman/aws-cdk-sftp-to-s3-sync architecture

> Architecture diagram of the GitHub repository srajasimman/aws-cdk-sftp-to-s3-sync, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/srajasimman/aws-cdk-sftp-to-s3-sync
- Repository: https://github.com/srajasimman/aws-cdk-sftp-to-s3-sync

## Overview

##### Project Type & Purpose
- This is an AWS infrastructure-as-code project built with AWS CDK (TypeScript) plus a small serverless application (Lambda) and EC2-based SFTP server.
- Its goal is to automate file ingestion from an SFTP endpoint (hosted on an EC2 instance) into an S3 bucket, with idempotency, error handling, scheduling, and secret management.

##### File Structure Analysis
- Top‐level CDK app:
- bin/app.ts – entry point for CDK.
- lib/sftp-ingest-stack.ts – defines the main CDK stack.
- lib/constructs/vpc.ts & s3-bucket.ts – reusable constructs for VPC and S3.
- cdk.json, tsconfig.json, package.json – CDK config & build.
- Lambda function code lives under src/:
- src/index.ts – handler.
- src/lib/{idempotency,logger,s3Writer,secrets}.ts – core logic (DynamoDB idempotency, S3 write, secret retrieval).
- Deployment toolchain files: .envrc, devbox.json, bun.lock, etc.

##### README Insights
- Lists all AWS components: VPC (public/private subnets), EC2 (SFTP), S3 bucket, Lambda, EventBridge (cron), optional DynamoDB, Secrets Manager, SQS DLQ.
- Highlights security best practices (private Lambda, IAM least privilege, encryption).
- Mentions outputs (public IP, ARNs) and post-deployment steps (upload credentials).

##### Diagramming the Architecture
a. Main Components:
- VPC: public & private subnets
- EC2 Instance (SFTP server) in public subnet
- EventBridge Rule (15-minute cron)
- Lambda Function in private subnet
- DynamoDB Table (idempotency) – optional
- S3 Bucket (file sink)
- Secrets Manager (SFTP credentials)
- SQS Dead‐Letter Queue
- IAM Roles & VPC Endpoints (for S3, Secrets Manager, DynamoDB)
b. Relationships & Interactions:
- EventBridge invokes Lambda on schedule.
- Lambda retrieves credentials from Secrets Manager.
- Lambda SSH-connects to EC2 SFTP server in public subnet.
- Lambda downloads new files, checks/updates DynamoDB for idempotency.
- Lambda writes processed files to S3.
- On Lambda failure, messages go to SQS DLQ.
- EC2 stores logs to CloudWatch via IAM/SSM.
c. Architectural Patterns & Principles:
- Infrastructure as Code (CDK) – reusable constructs, single Stack.
- Serverless pattern for ingestion logic (Lambda + EventBridge).
- Idempotent handling via DynamoDB.
- Secure secret storage (Secrets Manager).
- Private subnets for compute, public for remote access.
- Least-privilege IAM roles.
d. Key Technologies & Libraries:
- AWS CDK v2 (TypeScript)
- AWS Lambda, EC2, VPC, S3, DynamoDB, Secrets Manager, EventBridge, SQS.
- TypeScript helper modules (ssh via Node, S3 SDK, DynamoDB SDK).

##### Capturing Essential Elements
- Don’t omit IAM roles & VPC endpoints – they enforce least privilege and private networking.
- Stack’s outputs (public IP, ARNs) are annotated as CDK “Outputs.”

## Diagram (Mermaid)

```mermaid
flowchart TD
  subgraph "Control Plane (CDK Deployment)"
    CDK_CLI["CDK CLI"]:::tool
    App["bin/app.ts"]:::cdk
    Stack["lib/sftp-ingest-stack.ts"]:::cdk
    VPC_Construct["lib/constructs/vpc.ts"]:::cdk
    S3_Construct["lib/constructs/s3-bucket.ts"]:::cdk
    CDK_CLI --> App
    App --> Stack
    Stack --> VPC_Construct
    Stack --> S3_Construct
  end

  subgraph "AWS Environment"
    subgraph "VPC"
      direction TB
      subgraph "Public Subnet"
        EC2["EC2 (SFTP Server)"]:::compute
      end
      subgraph "Private Subnet"
        Lambda["Lambda Function"]:::compute
      end
      subgraph "VPC Endpoints"
        VPCE_S3["S3 Endpoint"]:::storage
        VPCE_SM["SecretsManager Endpoint"]:::secret
        VPCE_DDB["DynamoDB Endpoint"]:::storage
      end
    end

    EB["EventBridge Rule\n(15-min)"]:::orchestration
    DDB["DynamoDB Table"]:::storage
    S3["S3 Bucket"]:::storage
    SM["Secrets Manager"]:::secret
    DLQ["SQS Dead-Letter Queue"]:::orchestration
    CW["CloudWatch Logs"]:::monitoring

    EB -->|invoke| Lambda
    Lambda -->|fetch credentials| SM
    Lambda -->|SSH connect| EC2
    Lambda -->|read/write| DDB
    Lambda -->|put object| S3
    Lambda -->|on failure| DLQ
    EC2 -->|"log to"| CW

    subgraph "Lambda Function Code"
      direction TB
      Handler["src/index.ts"]:::code
      Idemp["src/lib/idempotency.ts"]:::code
      S3W["src/lib/s3Writer.ts"]:::code
      SecLib["src/lib/secrets.ts"]:::code
      Logger["src/lib/logger.ts"]:::code
      Handler --> Idemp
      Handler --> S3W
      Handler --> SecLib
      Handler --> Logger
      Lambda --> Handler
    end
  end

  classDef compute fill:#D0E8FF,stroke:#0366D6,color:#0366D6,stroke-width:1px;
  classDef storage fill:#E6FFEA,stroke:#027A1E,color:#027A1E,stroke-width:1px;
  classDef orchestration fill:#FFF4E5,stroke:#C87000,color:#C87000,stroke-width:1px;
  classDef secret fill:#F2F2F2,stroke:#999999,color:#999999,stroke-width:1px;
  classDef monitoring fill:#F0F0F0,stroke:#666666,color:#666666,stroke-width:1px;
  classDef cdk fill:#F3E5F5,stroke:#6A1B9A,color:#6A1B9A,stroke-width:1px;
  classDef tool fill:#E0F7FA,stroke:#006064,color:#006064,stroke-width:1px;
  classDef code fill:#FFF0F5,stroke:#C2185B,color:#C2185B,stroke-width:1px;

  click App "https://github.com/srajasimman/aws-cdk-sftp-to-s3-sync/blob/main/bin/app.ts"
  click Stack "https://github.com/srajasimman/aws-cdk-sftp-to-s3-sync/blob/main/lib/sftp-ingest-stack.ts"
  click VPC_Construct "https://github.com/srajasimman/aws-cdk-sftp-to-s3-sync/blob/main/lib/constructs/vpc.ts"
  click S3_Construct "https://github.com/srajasimman/aws-cdk-sftp-to-s3-sync/blob/main/lib/constructs/s3-bucket.ts"
  click Handler "https://github.com/srajasimman/aws-cdk-sftp-to-s3-sync/blob/main/src/index.ts"
  click Idemp "https://github.com/srajasimman/aws-cdk-sftp-to-s3-sync/blob/main/src/lib/idempotency.ts"
  click S3W "https://github.com/srajasimman/aws-cdk-sftp-to-s3-sync/blob/main/src/lib/s3Writer.ts"
  click SecLib "https://github.com/srajasimman/aws-cdk-sftp-to-s3-sync/blob/main/src/lib/secrets.ts"
  click Logger "https://github.com/srajasimman/aws-cdk-sftp-to-s3-sync/blob/main/src/lib/logger.ts"
```
