How azure/azurestor works
Main Components
• Client Interface: Includes S3 method implementations for managing storage objects (files, blobs, etc.) and for performing file transfers.
• Admin Interface: Built on top of AzureRMR with R6 classes; this part manages storage account creation, deletion, key management, and SAS token generation.
• Storage Endpoints: The central abstraction provided by the “storage_endpoint” function which determines the type of storage (blob, file, or ADLS) and handles authentication (access key, SAS, AAD token).
• Transfer Utilities: Components for uploading and downloading files, including support for parallel transfers, file format helpers (CSV, RDS, etc.), connection-based transfers, and integration with AzCopy.
• Authentication & Authorization: Modules that support multiple authentication mechanisms (access keys, SAS, OAuth tokens/AAD tokens). This includes integration with AzureRMR and AzureAuth in case of AAD tokens.
• External Dependencies: Integration with external components like the AzCopy utility (for high-performance file transfers) and the AzureRMR package.
Relationships and Interactions
• The admin side interacts with the Azure Resource Manager via AzureRMR. This module is responsible for creating, listing, and deleting storage accounts along with retrieving authentication keys and SAS tokens.
• The endpoints for blob, file, and ADLS storage should be displayed as separate components (or as subcomponents under a “Storage Endpoints” node) that share similar interfaces but differ in authentication method and supported operations.
Architectural Patterns / Design Principles
• Modular Design: Different R scripts (found in the “R” directory) encapsulate different functionalities (e.g., azcopy support, lease management, generic transfer utilities). Represent these modules as distinct nodes or layers.
Technologies, Frameworks, and Libraries
• Specify that this is an R package using S3/S4 methods and R6 classes.
• AzureRMR for resource management and AzureAuth for authentication.
• AzCopy as an external command-line utility that is invoked from within the package’s functions.
• Use of parallel processing (via background process pools) for file transfers.
Design
• Descriptive notes where necessary (for example: “Handles both SAS and AAD token-based authentication” or “Utilizes parallel transfers for efficiency”).
Additional Elements
• Don’t forget to indicate test components (the “tests” directory) and documentation (the “man” and “vignettes” folders), which reflect a well-documented, testable system design. While these might not be central to runtime architecture, they emphasize the project’s organization.