How miroapp/cloud-data-import works
Main System Components
a) Client Interface Layer:
• CLI Interface (src/cli.ts)
• GUI Configuration Interface (src/aws-app/config/getConfigViaGui.ts)
b) Core Processing Layer:
• Configuration Management (src/aws-app/config/)
• Scanner System (src/scanners/)
• Processing Engine (src/aws-app/process/)
c) AWS Integration Layer:
• Resource Scanners (src/scanners/scan-functions/aws/)
• Rate Limiting System (src/aws-app/utils/AWSRateLimitExhaustionRetryStrategy.ts)
Input Flow
User Input -> CLI/GUI -> Configuration Manager -> Scanner System
Processing Flow
Scanner System -> AWS Services -> Resource Collection -> Processing Engine -> Output Generation
Specific Component Details
Key elements:
a) Scanner Components:
• Global Scanner
• Regional Scanner
• Resource-specific scanners (show major categories)
b) Processing Components:
• Container Management
• Resource Processing
• Placement Data Generation
c) Rate Limiting and Error Handling:
• Rate Limiter Factory
• Retry Strategy
• Error Management
Detailed Component Breakdown
Create sections for:
a) Configuration Management:
CLI Arguments -> Environment Config -> GUI Input -> Combined Config
b) Scanner Architecture:
Scanner Factory -> Regional/Global Scanners -> Resource-specific Scanners
c) Processing Pipeline:
Raw Data -> Container Assignment -> Resource Processing -> Placement Calculation -> JSON Output
Important Notes
Special considerations:
• Rate limiting mechanism between AWS API calls
• Error handling and retry logic
• Regional vs. Global service scanning distinction
• Resource tagging and metadata collection
• Output file generation and compression
Technical Details to Highlight
Key technical aspects:
• Node.js runtime environment
• AWS SDK integration
• JSON schema validation
• Rate limiting configurations
• Regional service scanning capabilities
This system design should effectively communicate how the tool scans AWS infrastructure, processes the data, and prepares it for visualization in Miro's Cloud View feature.