How alcionai/corso works
Project Type and Purpose
• Corso is an open‑source, command‑line data‑protection tool for Microsoft 365 (Exchange, OneDrive, SharePoint, Teams).
• It provides an intuitive CLI for administrators to back up, export, restore, and purge M365 data into storage of their choice.
High‑Level File Structure Analysis
• Top‑level folders:
– src/cli – CLI commands and flags
– src/internal – core engine: M365 connectors, Kopia integration, orchestrators, utilities
– src/pkg – shared libraries: account management, config, storage adapters, API clients
– cmd/… – specialized binaries and test drivers (sanity, longevity, graph PowerShell helper)
– build/, docker/, .github/ – build scripts and CI/CD workflows
– website/ – Docusaurus‑based documentation site
• Key patterns:
– Adapter and factory patterns for M365 services and storage backends
– “operations” orchestration layer managing backup, export, restore flows
– Wrapper layer around Kopia (the backup engine) for snapshot management
– Streamstore abstraction for writing incremental data streams
– Internal “converters” to normalize M365 payloads (EML, ICS, VCF) to a unified model
Core Components
A. CLI Frontend
– Entry point (corso.go) → dispatches to subcommands: backup, export, restore, purge, debug
B. Controller / Orchestrator Layer
– internal/controller.go
– internal/operations (backup.go, export.go, restore.go, retention_config.go)
– Factories and dependency injection (cmd/factory)
C. M365 Service Adapters
– pkg/services/m365/api/graph and pkg/services/m365/service/* for Exchange, OneDrive, SharePoint, Teams
– internal/m365/collection and internal/m365/service layers
D. Data Conversion
– internal/converters (EML, ICS, VCF)
– internal/model for unified schema
E. Backup Engine Integration
– internal/kopia wrapper, inject, merge, retention
– pkg/store/model_store and pkg/store/wrapper for storing Kopia snapshots
F. Storage Backends
– pkg/storage/filesystem and pkg/storage/s3 adapters
– cli/repo for filesystem or S3 repositories
G. StreamStore & Metrics
– internal/streamstore for streaming data to Kopia
– internal/stats and pkg/metrics for telemetry
H. Utilities & Common
– internal/common (config, limiters, retry, crash handling, logging)
Component Interactions & Data Flows
1) User invokes corso backup → CLI layer parses flags and options.
2) CLI creates an operations.Backup instance via factory.
3) Backup operation calls M365 connectors → paginates via Graph API → collects items.
4) Collected items pass through converters → normalized model.
5) Model fed into streamstore → streamed into Kopia wrapper.
6) Kopia wrapper commits snapshot into a repository (S3 or FS) via storage adapter.
7) Operations record metadata and metrics; exit.
Architectural Patterns & Principles
• Modular layering: CLI → orchestrator → domain services → engine
• Adapter pattern for external dependencies (Graph API, Kopia, S3/FS)
• Factory/DI to allow testing with mocks (internal/*/mock packages)
• Separation of concerns: converters, models, storage kept distinct
• Extensibility points: new M365 services or storage backends plug into existing abstractions
Relevant Technologies & Frameworks
• Go modules with well‑structured pkg/ and internal/ layout
• Kopia backup engine via Go wrapper
• GitHub Actions for CI, release, website linting
• Docusaurus for documentation site
• Docker for self‑contained builds and tooling (build/, docker/)