# awslabs/sagemaker-corporate-credit-rating architecture

> Architecture diagram of the GitHub repository awslabs/sagemaker-corporate-credit-rating, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/awslabs/sagemaker-corporate-credit-rating
- Repository: https://github.com/awslabs/sagemaker-corporate-credit-rating

## Overview

##### Project Type and Purpose
- This is an AWS SageMaker JumpStart Industry solution for a _multimodal_ corporate credit rating model. It combines tabular financial data (Altman’s ratios) with text (SEC filings) to train an AutoGluon model, deploy it to a SageMaker endpoint, and serve real-time predictions.
- It also includes CloudFormation templates to provision all AWS infrastructure and a “solution-assistant” Lambda for orchestration or helper workflows.

##### File Structure Analysis
Top-level directories and files:
- .github & workflows – CI/CD tests, CodeQL scanning, issue templates
- cloudformation – YAML templates for:
– core stack (corporate-credit-rating.yaml)
– nested stacks: endpoint, permissions, notebook instance, SageMaker Studio/solution-assistant Lambda
– Python helper (lambda_fn.py) for solution-assistant
- scripts – packaging and kernel setup for notebooks
- src – two Jupyter notebooks (train & endpoint demo), and Python modules:
– model-training/train.py
– model-inference/inference.py
– prepare_model_code.sh
- README.md – detailed solution overview, architectures, cost/cleanup instructions
- manifest.json, LICENSE, CONTRIBUTING, etc.

##### Key Architectural Insights from the README
- Data lives in Amazon S3 (synthetic dataset and notebooks).
- Training uses the SageMaker MXNet AutoGluon estimator.
- Deployment produces a real-time HTTPS SageMaker endpoint.
- CloudWatch monitors training jobs and endpoints.
- CloudFormation drives end-to-end provisioning.
- A helper Lambda (“solution-assistant”) can be optionally invoked via AWS events or API Gateway.

##### Main Components
A. Developer & CI/CD
- GitHub repo & GitHub Actions → triggers tests and validates CloudFormation
B. Infrastructure Provisioning
- AWS CloudFormation (parent stack)
- Nested stacks: IAM Roles, S3 buckets, SageMaker Notebook Instance, Solution Assistant Lambda, SageMaker Endpoint configuration
C. Data Storage
- Amazon S3 buckets: raw synthetic dataset, model artifacts, inference code
D. Compute & Processing
- SageMaker Notebook Instance: runs corporate-credit-rating.ipynb & endpoint-demo.ipynb
- SageMaker Training Job (AutoGluon MXNet) using train.py
- SageMaker Model & Endpoint using inference.py
- (Optional) Solution-Assistant Lambda
E. Monitoring & Logging
- Amazon CloudWatch for logs and metrics of training jobs, endpoint invocations
F. External Clients
- Jupyter notebooks or external applications invoking the SageMaker HTTPS endpoint for predictions

##### Relationships and Data Flow
1. Developer pushes code → GitHub Actions runs CI → validates infrastructure templates
2. CloudFormation provisions all AWS resources (S3, IAM, Notebook, Lambda, Endpoint config)
3. Data and code land in S3
4. SageMaker Notebook downloads data via SageMaker Python SDK
5. Notebook invokes SageMaker Training Job
– Input: data from S3
– Compute: train.py performs feature engineering & AutoGluon training
– Output: model artifacts to S3
6. Notebook invokes deployment → SageMaker creates Model & Endpoint
– Inference code (inference.py) packaged in container
7. External clients or notebooks send requests to the HTTPS endpoint → receive predictions
8. CloudWatch captures metrics/logs at each stage
9. (Optional) Solution-Assistant Lambda responds to API Gateway or events for helper actions

##### Architectural Patterns & Principles
- MLOps pipeline: clear separation between data, training, and serving
- Infrastructure as Code: CloudFormation for reproducibility
- Modular design: discrete scripts for training vs inference; nested stacks for isolation
- Serverless compute: SageMaker-managed services & Lambda for orchestration
- Managed storage: S3 for data and artifacts, IAM for least-privilege access
- Monitoring: centralized logging and metrics with CloudWatch

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Layer 1: Source & CI/CD
    subgraph "1. Source & CI/CD"
        GitHub["GitHub Repo"]:::external
        CI["GitHub Actions"]:::external
        CodeQL["CodeQL Scanning"]:::external
    end

    %% Layer 2: Infrastructure Provisioning
    subgraph "2. Infrastructure Provisioning"
        CF["CloudFormation Parent Stack"]:::infra
        subgraph "Nested Stacks"
            Permissions["IAM Permissions Stack"]:::infra
            EndpointStack["Endpoint Config Stack"]:::infra
            NotebookStack["SageMaker Notebook Instance Stack"]:::infra
            LambdaStack["Solution-Assistant Lambda Stack"]:::infra
        end
    end

    %% Layer 3: Data Storage
    subgraph "3. Data Storage"
        S3["Amazon S3\n(synthetic-data, artifacts)"]:::aws
        Manifest["manifest.json"]:::code
    end

    %% Layer 4: Training & Serving
    subgraph "4. Training & Serving"
        Notebook["SageMaker Notebook Instance"]:::aws
        TrainingNb["Training Notebook"]:::code
        EndpointNb["Endpoint Demo Notebook"]:::code
        TrainScript["train.py"]:::code
        InferScript["inference.py"]:::code
        PackHelper["prepare_model_code.sh"]:::code
        TrainJob["SageMaker Training Job"]:::aws
        Model["SageMaker Model"]:::aws
        Endpoint["SageMaker HTTPS Endpoint"]:::aws
        SolutionLambda["Solution-Assistant Lambda"]:::infra
    end

    %% Layer 5: Monitoring & Clients
    subgraph "5. Monitoring & Clients"
        CW["CloudWatch"]:::aws
        Client["External Client\nor App"]:::external
    end

    %% Connections
    GitHub --> CI
    CI --> CodeQL
    CI --> CF
    CF --> Permissions
    CF --> EndpointStack
    CF --> NotebookStack
    CF --> LambdaStack
    CF --> S3
    S3 --> Notebook
    Notebook --> TrainingNb
    Notebook --> EndpointNb
    Notebook --> TrainScript
    Notebook --> InferScript
    TrainingNb --> TrainJob
    TrainJob --> S3
    TrainingNb --> PackHelper
    PackHelper --> Model
    EndpointNb --> Model
    Model --> Endpoint
    Client --> Endpoint
    Endpoint --> Client
    TrainJob --> CW
    Endpoint --> CW
    LambdaStack --> SolutionLambda
    SolutionLambda --> CW

    %% Click Events
    click CI "https://github.com/awslabs/sagemaker-corporate-credit-rating/blob/main/.github/workflows/ci.yaml"
    click CodeQL "https://github.com/awslabs/sagemaker-corporate-credit-rating/blob/main/.github/workflows/codeql.yaml"
    click CF "https://github.com/awslabs/sagemaker-corporate-credit-rating/blob/main/cloudformation/corporate-credit-rating.yaml"
    click EndpointStack "https://github.com/awslabs/sagemaker-corporate-credit-rating/blob/main/cloudformation/corporate-credit-rating-endpoint.yaml"
    click Permissions "https://github.com/awslabs/sagemaker-corporate-credit-rating/blob/main/cloudformation/corporate-credit-rating-permissions.yaml"
    click NotebookStack "https://github.com/awslabs/sagemaker-corporate-credit-rating/blob/main/cloudformation/corporate-credit-rating-sagemaker-notebook-instance.yaml"
    click LambdaStack "https://github.com/awslabs/sagemaker-corporate-credit-rating/blob/main/cloudformation/solution-assistant/solution-assistant.yaml"
    click Manifest "https://github.com/awslabs/sagemaker-corporate-credit-rating/blob/main/manifest.json"
    click TrainingNb "https://github.com/awslabs/sagemaker-corporate-credit-rating/blob/main/src/corporate-credit-rating.ipynb"
    click EndpointNb "https://github.com/awslabs/sagemaker-corporate-credit-rating/blob/main/src/endpoint-demo.ipynb"
    click TrainScript "https://github.com/awslabs/sagemaker-corporate-credit-rating/blob/main/src/model-training/train.py"
    click InferScript "https://github.com/awslabs/sagemaker-corporate-credit-rating/blob/main/src/model-inference/inference.py"
    click PackHelper "https://github.com/awslabs/sagemaker-corporate-credit-rating/blob/main/src/prepare_model_code.sh"
    click SolutionLambda "https://github.com/awslabs/sagemaker-corporate-credit-rating/blob/main/cloudformation/solution-assistant/src/lambda_fn.py"

    %% Styles
    classDef aws fill:#bae1ff,stroke:#333,stroke-width:1px
    classDef code fill:#bffcc6,stroke:#333,stroke-width:1px
    classDef infra fill:#ffd8b1,stroke:#333,stroke-width:1px
    classDef external fill:#e0e0e0,stroke:#333,stroke-width:1px
```
