# snyk/snyk-api-import architecture

> Architecture diagram of the GitHub repository snyk/snyk-api-import, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/snyk/snyk-api-import
- Repository: https://github.com/snyk/snyk-api-import

## Overview

##### Main Components

a. CLI Interface Layer:
- Main command handlers (`src/cmds/`)
- Primary commands: import, orgs:create, orgs:data, import:data, list:imported

b. Core Service Layer:
- API Integration Services (`src/lib/api/`)
- Source Control Handlers (`src/lib/source-handlers/`)
- Project Management (`src/lib/project/`)

c. External Systems Integration:
- SCM Systems (GitHub, GitLab, Bitbucket Cloud, Bitbucket Server, Azure DevOps)
- Snyk API
- File System for logging and data storage

##### Component Relationships

**[CLI Commands] → [Core Services] → [Source Handlers] → [External SCMs]**
- ↓
- [Snyk API Client]
- ↓
- [Snyk API]

##### Specific Implementation Details

Key architectural elements:
- Rate Limiting System (`src/lib/limiters.ts`)
- Queue Management for API requests
- Retry Logic
- Logging System (`src/loggers/`)

##### Component Details

**┌─────────────────────────────────────────┐**
- CLI Commands               │
- ┌─────────┐ ┌────────┐ ┌────────────┐  │
- Import  │ │  Orgs  │ │    List    │  │
- ┘ └────────┘ └────────────┘  │
- ┬───────────────────┘
- ↓
**┌─────────────────────────────────────────┐**
- Core Services                │
- ┌─────────┐ ┌────────┐ ┌────────────┐  │
- API    │ │Project │ │   Queue    │  │
- Services │ │Handler │ │  Manager   │  │
- ┘ └────────┘ └────────────┘  │
- ┬───────────────────┘
- ↓
**┌─────────────────────────────────────────┐**
- Source Handlers                 │
- ┌─────────┐ ┌────────┐ ┌────────────┐  │
- GitHub  │ │GitLab  │ │ Bitbucket  │  │
- ┘ └────────┘ └────────────┘  │
- ┘

##### Key Flows to Highlight

a. Import Flow:
**User Command → Rate Limiter → Source Handler → SCM API**
- ↓
- Project Analysis
- ↓
- Snyk Import

b. Organization Management Flow:
**Org Command → Data Generation → Snyk API → Org Creation**

##### Important Notes

- Error handling paths
- Retry mechanisms
- Rate-limiting checkpoints
- Logging points
- Configuration input points

##### Technical Details

- Node.js runtime environment
- External API dependencies
- File system interactions
- Configuration management

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph CLI["CLI Interface Layer"]
        Import["🔄 Import Command"]
        OrgsCreate["🏢 Orgs Create Command"]
        ListImported["📋 List Imported Command"]
    end

    subgraph CoreServices["Core Services Layer"]
        subgraph APIServices["API Services"]
            ImportAPI["📥 Import API"]
            OrgAPI["🏛️ Organization API"]
            ProjectAPI["📁 Project API"]
            FeatureFlags["🚩 Feature Flags"]
        end
        
        subgraph ProjectMgmt["Project Management"]
            BranchUpdates["🌿 Branch Updates"]
            ProjectTypes["📦 Project Types"]
            ManifestSupport["📄 Manifest Support"]
        end
        
        RateLimiter["⏱️ Rate Limiting System"]
        QueueManager["📊 Queue Manager"]
    end

    subgraph SourceControl["Source Control Layer"]
        GitHub["🐱 GitHub Handler"]
        GitLab["🦊 GitLab Handler"]
        BitbucketCloud["🐟 Bitbucket Cloud Handler"]
        AzureDevOps["☁️ Azure DevOps Handler"]
    end

    subgraph Logging["Logging System"]
        ImportLog["📝 Import Logger"]
        ErrorLog["⚠️ Error Logger"]
        OrgLog["📔 Organization Logger"]
    end

    subgraph External["External Systems"]
        SnykAPI["🛡️ Snyk API"]
        SCMSystems["💻 SCM Systems"]
    end

    Import --> APIServices
    OrgsCreate --> OrgAPI
    ListImported --> ProjectAPI

    APIServices --> RateLimiter
    RateLimiter --> QueueManager
    QueueManager --> SourceControl
    
    SourceControl --> SCMSystems
    APIServices --> SnykAPI
    
    Import --> ImportLog
    OrgsCreate --> OrgLog
    APIServices --> ErrorLog

    ProjectMgmt --> APIServices
    FeatureFlags --> APIServices

    classDef default fill:#f9f9f9,stroke:#333,stroke-width:1px
    classDef cli fill:#dae8fc,stroke:#6c8ebf
    classDef core fill:#d5e8d4,stroke:#82b366
    classDef source fill:#fff2cc,stroke:#d6b656
    classDef logging fill:#f8cecc,stroke:#b85450
    classDef external fill:#e1d5e7,stroke:#9673a6

    class CLI,Import,OrgsCreate,ListImported cli
    class CoreServices,APIServices,ProjectMgmt,RateLimiter,QueueManager core
    class SourceControl,GitHub,GitLab,BitbucketCloud,AzureDevOps source
    class Logging,ImportLog,ErrorLog,OrgLog logging
    class External,SnykAPI,SCMSystems external

    click Import "https://github.com/snyk/snyk-api-import/blob/master/src/cmds/import.ts"
    click OrgsCreate "https://github.com/snyk/snyk-api-import/blob/master/src/cmds/orgs:create.ts"
    click ListImported "https://github.com/snyk/snyk-api-import/blob/master/src/cmds/list:imported.ts"
    click ImportAPI "https://github.com/snyk/snyk-api-import/tree/master/src/lib/api/import/"
    click OrgAPI "https://github.com/snyk/snyk-api-import/tree/master/src/lib/api/org/"
    click ProjectAPI "https://github.com/snyk/snyk-api-import/tree/master/src/lib/api/project/"
    click FeatureFlags "https://github.com/snyk/snyk-api-import/tree/master/src/lib/api/feature-flags/"
    click GitHub "https://github.com/snyk/snyk-api-import/tree/master/src/lib/source-handlers/github/"
    click GitLab "https://github.com/snyk/snyk-api-import/tree/master/src/lib/source-handlers/gitlab/"
    click BitbucketCloud "https://github.com/snyk/snyk-api-import/tree/master/src/lib/source-handlers/bitbucket-cloud/"
    click AzureDevOps "https://github.com/snyk/snyk-api-import/tree/master/src/lib/source-handlers/azure/"
    click BranchUpdates "https://github.com/snyk/snyk-api-import/blob/master/src/lib/project/update-branch.ts"
    click RateLimiter "https://github.com/snyk/snyk-api-import/blob/master/src/lib/limiters.ts"
    click ImportLog "https://github.com/snyk/snyk-api-import/blob/master/src/loggers/log-imported-projects.ts"
    click ErrorLog "https://github.com/snyk/snyk-api-import/blob/master/src/loggers/log-failed-imports.ts"
    click OrgLog "https://github.com/snyk/snyk-api-import/blob/master/src/loggers/log-created-org.ts"
    click QueueManager "https://github.com/snyk/snyk-api-import/blob/master/src/lib/api/import/request-with-rate-limit.ts"
    click ManifestSupport "https://github.com/snyk/snyk-api-import/blob/master/src/lib/supported-project-types/supported-manifests.ts"
    click ProjectTypes "https://github.com/snyk/snyk-api-import/tree/master/src/lib/supported-project-types/"
```
