# a-vijai/cryptocurrency_project architecture

> Architecture diagram of the GitHub repository a-vijai/cryptocurrency_project, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/a-vijai/cryptocurrency_project
- Repository: https://github.com/a-vijai/cryptocurrency_project

## Overview

##### Project type and purpose
- This is a serverless ETL pipeline for collecting, transforming, and loading cryptocurrency data.
- It leverages AWS Step Functions to orchestrate three AWS Lambda functions and uses EventBridge to schedule daily runs.
- Data lands in Amazon S3 (raw & transformed), is loaded into Amazon Redshift Serverless, and surfaced via Looker Studio.

##### File structure
- Top‐level files:
- aws_lambda_extract.py – Lambda handler to call the crypto API and write raw JSON to S3
- aws_lambda_transform.py – Lambda handler to read raw data, normalize/clean, write transformed data back to S3
- aws_lambda_load.py – Lambda handler to load transformed data from S3 into Redshift Serverless
- state_machine.json – Defines the AWS Step Functions state machine with three sequential tasks
- README.md – Overview of architecture, scheduling, and BI integration
- No separate “infra” folder: state_machine.json is the only orchestration definition; Lambdas are defined inline or via an external deployment tool (not shown).

##### README
- Confirms:
- AWS Step Functions for ETL orchestration
- EventBridge cron trigger for daily execution
- S3 for raw and transformed storage
- Amazon Redshift Serverless for warehousing
- Looker Studio for reporting/visualization

- Clear labels on every component
1. EventBridge → Step Functions
2. Step Functions → Extract → S3 raw
3. Step Functions → Transform → S3 transformed
4. Step Functions → Load → Redshift
5. Looker → Redshift

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Subgraphs
    subgraph "Orchestration"
        direction TB
        EventBridge["EventBridge (Scheduler)"]:::orchestration
        SF["Step Functions State Machine"]:::orchestration
    end

    subgraph "Compute"
        direction TB
        Extract["Lambda: Extract"]:::compute
        Transform["Lambda: Transform"]:::compute
        Load["Lambda: Load"]:::compute
    end

    subgraph "Storage"
        direction TB
        RawS3["S3 Bucket: raw-data"]:::storage
        TransS3["S3 Bucket: transformed-data"]:::storage
        Redshift["Amazon Redshift Serverless"]:::storage
    end

    subgraph "BI/Visualization"
        direction TB
        Looker["Looker Studio"]:::bi
    end

    Docs["README.md (Architecture Docs)"]:::doc

    %% Connections
    EventBridge -->|"triggers"| SF
    SF -->|"Extract Task"| Extract
    Extract -->|"writes raw JSON"| RawS3
    SF -->|"Transform Task"| Transform
    Transform -->|"reads raw & writes transformed"| TransS3
    SF -->|"Load Task"| Load
    Load -->|"loads to Redshift"| Redshift
    Looker -->|"queries"| Redshift
    SF --> Docs

    %% Click Events
    click SF "https://github.com/a-vijai/cryptocurrency_project/blob/main/state_machine.json"
    click Extract "https://github.com/a-vijai/cryptocurrency_project/blob/main/aws_lambda_extract.py"
    click Transform "https://github.com/a-vijai/cryptocurrency_project/blob/main/aws_lambda_transform.py"
    click Load "https://github.com/a-vijai/cryptocurrency_project/blob/main/aws_lambda_load.py"
    click Docs "https://github.com/a-vijai/cryptocurrency_project/blob/main/README.md"

    %% Styles
    classDef orchestration fill:#B2EBF2,stroke:#0288D1,stroke-width:2px
    classDef compute fill:#FFF9C4,stroke:#FBC02D,stroke-width:2px
    classDef storage fill:#C8E6C9,stroke:#388E3C,stroke-width:2px
    classDef bi fill:#FFCCBC,stroke:#E64A19,stroke-width:2px
    classDef doc fill:#E0E0E0,stroke:#757575,stroke-width:1px,stroke-dasharray: 5 5
```
