How aws-samples/esg-compliance-report-processor works
Project Type and Purpose
• This is a full-stack, serverless AWS application with an ECS-hosted Streamlit frontend.
• It ingests compliance reports, orchestrates processing via Step Functions and Lambdas, stores data in DynamoDB/S3, and provides a secured UI behind Cognito.
File Structure
• Top-level “cdk/” directory: all CDK stacks and Lambda function definitions, layer configs, tests.
• “streamlit/” directory: the Python Streamlit web app, Dockerfile, setup scripts, and Cognito + ECS configuration.
• “assets/architecture” holds the reference SVG. “examples/” contains sample inputs.
Main Components to Model
a. S3 Report Upload Bucket
b. EventBridge Rule & Step Functions State Machine
c. AWS Lambda Functions (twelve-plus small, single-purpose Lambdas)
d. Lambda Layers for third-party libs (PyMuPDF, Textract clients, LangChain, etc.)
e. Amazon Textract & Amazon Bedrock (external AI/ML services)
f. Amazon DynamoDB Tables (supplier_table, compliance_grading_table)
g. Amazon SNS Topic (optional notifications)
h. API Gateway (approve/reject endpoints)
i. Amazon Cognito User Pool & Identity Pool (OIDC integration)
j. VPC, ECS Fargate Cluster & Task Definition (Streamlit UI)
k. Application Load Balancer with ACM certificate
l. Route 53 alias record (DNS entry)
Relationships & Data Flow
• User uploads report via Streamlit UI → S3 “report_bucket.”
• S3 “PutObject” event → EventBridge → Step Functions.
• Step Functions invokes Lambdas in sequence:
• report_split → extract_nc → bedrock_supplier_extraction → supplier_details → get_nc → get_status → generate_email → email_approved/email_rejected → send_emails → upload_grading → validate_unrated_issues.
• Lambdas read/write DynamoDB tables and may publish to SNS.
• Textract & Bedrock invoked within specific Lambdas.
• Frontend polls API Gateway endpoints (secured by Cognito) for status and email summary.
• UI served by ECS Fargate behind ALB. ALB enforces HTTPS with ACM cert.
• Cognito sits between ALB and API Gateway to protect endpoints.
Architectural Patterns & Principles
• Event-driven, serverless pipeline (S3 → EventBridge → Step Functions → Lambdas).
• Single-responsibility Lambdas (micro-functions).
• Layer reuse for heavy dependencies.
• IAM-backed security and least-privilege roles.
• Decoupling of components via queues/events (EventBridge, SNS).
• Frontend/backend separation (ECS UI vs. serverless backend).
Technologies & Frameworks to Annotate
• AWS CDK (Python)
• AWS Lambda & Lambda Layers
• AWS Step Functions
• Amazon S3, DynamoDB, SNS, API Gateway
• Amazon Textract & Amazon Bedrock
• Amazon Cognito (OIDC)
• ECS Fargate, ALB, ACM, Route 53
• Streamlit, PyMuPDF, LangChain, Pandas
Must-Include Elements
• Callouts for configuration: ACM cert, Secrets Manager for OIDC credentials.