How code-dot-org/aiproxy works
Project Type & Purpose
• This is a Python-based backend microservice (“AI Proxy”) that exposes an HTTP API (Flask) to wrap LLM calls (OpenAI or AWS Bedrock).
• It runs in Docker containers on AWS ECS Fargate, with infra defined via CloudFormation and delivered via a CodePipeline–CodeBuild CI/CD process.
File-Structure Highlights
• /src Flask app entry point (__init__.py) + route handlers (assessment.py, openai.py, test.py)
• /lib/assessment Core business logic: feature extraction, decision trees, labeling, reporting, confidence logic
• /cicd Three-phase infra templates & buildspecs:
• phase 1: CICD dependency stack
• phase 2: CICD pipeline stack (CodePipeline, CodeBuild, ECR)
• phase 3: application stack (ECS service, ALB, Secrets Manager, S3 IAM roles)
• Dockerfile & docker-compose.yml container build & local‐dev orchestration
• bin/ & ci-*.sh test-and-validation scripts
• tests/ unit, integration, accuracy, and route tests
README Insights
• Config via config.txt env vars & AWS Secrets Manager
• Local dev: Docker Compose, AWS credentials profile (aiproxy_development)
• Rubric Tester CLI (lib/assessment/rubric_tester.py): copies S3 data locally (datasets, experiments, releases) and runs accuracy measurements
• Logging → Python logging → CloudWatch via ECS log driver
• API docs in API.md; testing docs in TESTING.md; CICD details in cicd/README.md
1. Client → ALB → ECS task → Flask route
2. Flask route → business logic → OpenAI/AWS Bedrock → business logic → Flask response → Client
3. Rubric CLI → local container/lib → S3 (datasets & config) → LLM → local report
4. Git push/PR → CodePipeline → CodeBuild (build, lint, test) → ECR → CloudFormation → ECS update
5. ECS task at startup reads env vars & fetches secrets from Secrets Manager
Architectural Patterns & Design Principles
• Layered service pattern (API → service → external)
• Containerized immutability (Docker)
• Infrastructure as code (CloudFormation)
• Automated CI/CD (CodePipeline, CodeBuild)
• Separation of concerns: routing vs. business logic vs. infra
Relevant Technologies & Frameworks
• Python 3.11, Flask, Docker, Docker Compose
• AWS ECS Fargate, ECR, CodePipeline, CodeBuild, CloudFormation, S3, Secrets Manager, CloudWatch
• OpenAI Python SDK / AWS Bedrock access
• Pytest for unit/integration tests