# forcedotcom/dataloader architecture

> Architecture diagram of the GitHub repository forcedotcom/dataloader, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/forcedotcom/dataloader
- Repository: https://github.com/forcedotcom/dataloader

## Overview

Based on our analysis of the file tree and README, this project is Salesforce Data Loader – a Java-based application with both a GUI (using SWT) and command-line/batch modes. It allows users to extract, insert, update, delete, and upsert Salesforce data. The project is organized into several logical layers including user interface components, business logic and process controllers, various client modules for communicating with Salesforce (using SOAP, REST, and Bulk APIs), and data access objects for handling CSV, database, and file interactions. In addition, supporting utilities such as mapping, encryption, error handling, and configuration provide critical shared functionality.

##### Main Components
- User Interface Layer: Represented in the “ui” package (dialogs, wizards, and SWT-related components) and the scripts/launchers (dataloader.sh, rundl.sh) that trigger GUI or command-line versions.
- Controller and Process Runner: The “controller” package (Controller.java) and process management in the “process” package (DataLoaderRunner, ProcessRunner, IProcess) coordinate user actions and backend processes.
- Salesforce Client & Integration Layer: Include the “client” package (e.g., BulkV1Client, BulkV2Client, RESTClient, PartnerClient) and “client/transport” implementations, which establish connectivity and protocol-specific communication with Salesforce.
- Data Access and Mapping: Identify the “dao” packages for CSV/database access and the “mapping” packages (LoadMapper, SOQLMapper, etc.) that transform data between CSV, Salesforce objects, and other formats.
- OAuth and Security Modules: Outline the “oauth” and “security” packages, which include components for managing authentication (OAuth flows) and encryption utilities (EncryptionAesUtil, EncryptionUtil).
- Utility and Configuration: Represent utilities (in “util”) and configuration management (AppConfig, config files in resources) that support logging (log4j, etc.) and system-wide settings.

##### Relationships and Interactions
- The UI layer (GUI dialogs, wizards) sends user commands to the Controller.
- The Controller orchestrates operations by invoking the Process Runner (often a command-line entry point in DataLoaderRunner) to perform actions.
- Process modules then call into the integration layer (various client implementations) to communicate with Salesforce using different APIs.
- Data is read from and written to CSV, databases, or flat files via the DAO implementations and then converted via Mapping functions.
- Utility modules (logger, encryption, OAuth flows) are used by all layers for cross-cutting concerns.

##### Technologies and Frameworks
- Java (with Maven as the build tool, as indicated in pom.xml)
- SWT for the graphical user interface
- salesforce API integrations via SOAP, REST, and Bulk interfaces
- Apache log4j for logging
- Maven for dependency management and build processes

## Diagram (Mermaid)

```mermaid
flowchart TD

    %% User Interface Layer
    subgraph "User Interface Layer"
        UI_Dialogs["User UI (SWT-based dialogs)"]:::ui
        UI_Launchers["GUI Launchers (rundl.sh, dlbuilder.sh)"]:::ui
    end

    %% Controller and Process Runner
    subgraph "Controller and Process Runner"
        Controller["Data Loader Controller"]:::controller
        Process_Runner["Process Runner (DataLoaderRunner, ProcessRunner)"]:::controller
    end

    %% Salesforce Client & Integration Layer
    subgraph "Salesforce Client & Integration Layer"
        Salesforce_API["Salesforce API Clients"]:::client
        Transport["Transport Implementations"]:::client
    end

    %% Data Access and Mapping
    subgraph "Data Access and Mapping"
        DAO["DAO (CSV/Database Access)"]:::dao
        Mapping["Mapping Functions"]:::dao
    end

    %% OAuth and Security Modules
    subgraph "OAuth and Security Modules"
        OAuth["OAuth Flows"]:::oauth
        Security["Security & Encryption"]:::oauth
    end

    %% Utility and Configuration
    subgraph "Utility and Configuration"
        Config["Application Configuration"]:::util
        Utilities["Common Utility Functions"]:::util
    end

    %% External Salesforce System
    Salesforce_System["Salesforce"]:::external

    %% Connections between components
    UI_Dialogs -->|"sends command"| Controller
    UI_Launchers -->|"triggers"| Controller

    Controller -->|"orchestrates"| Process_Runner

    Process_Runner -->|"invokes"| Salesforce_API
    Process_Runner -->|"invokes"| Transport

    Salesforce_API -->|"calls "| Salesforce_System
    Transport -->|"uses "| Salesforce_System

    %% Interactions with Data access layer
    Process_Runner -->|"reads/writes data"| DAO
    DAO -->|"converts data"| Mapping
    Mapping -->|"formats data for"| Salesforce_API

    %% Cross-cutting concerns via Utilities & Config
    UI_Dialogs ---|"uses"| Config
    Controller ---|"logs/gets config"| Config
    Process_Runner ---|"logs/gets config"| Utilities
    Salesforce_API ---|"logs/encrypts"| Security
    DAO ---|"maps config"| Config
    OAuth ---|"authenticates via"| Config
    Security ---|"uses"| Utilities

    %% Additional dependency: All layers use utility functions
    Controller ---|"uses"| Utilities
    Salesforce_API ---|"uses"| Utilities
    DAO ---|"uses"| Utilities
    OAuth ---|"uses"| Utilities
    Security ---|"uses"| Utilities

    %% Click Events
    click UI_Dialogs "https://github.com/forcedotcom/dataloader/tree/master/src/main/java/com/salesforce/dataloader/ui"
    click UI_Launchers "https://github.com/forcedotcom/dataloader/blob/master/rundl.sh"
    click Controller "https://github.com/forcedotcom/dataloader/blob/master/src/main/java/com/salesforce/dataloader/controller/Controller.java"
    click Process_Runner "https://github.com/forcedotcom/dataloader/blob/master/src/main/java/com/salesforce/dataloader/process/DataLoaderRunner.java"
    click Process_Runner "https://github.com/forcedotcom/dataloader/blob/master/src/main/java/com/salesforce/dataloader/process/ProcessRunner.java"
    click Salesforce_API "https://github.com/forcedotcom/dataloader/tree/master/src/main/java/com/salesforce/dataloader/client"
    click Transport "https://github.com/forcedotcom/dataloader/tree/master/src/main/java/com/salesforce/dataloader/client/transport"
    click DAO "https://github.com/forcedotcom/dataloader/tree/master/src/main/java/com/salesforce/dataloader/dao"
    click Mapping "https://github.com/forcedotcom/dataloader/tree/master/src/main/java/com/salesforce/dataloader/mapping"
    click OAuth "https://github.com/forcedotcom/dataloader/tree/master/src/main/java/com/salesforce/dataloader/oauth"
    click Security "https://github.com/forcedotcom/dataloader/tree/master/src/main/java/com/salesforce/dataloader/security"
    click Config "https://github.com/forcedotcom/dataloader/tree/master/src/main/java/com/salesforce/dataloader/config"
    click Config "https://github.com/forcedotcom/dataloader/blob/master/src/main/resources/config.properties"
    click Utilities "https://github.com/forcedotcom/dataloader/tree/master/src/main/java/com/salesforce/dataloader/util"

    %% Styles
    classDef ui fill:#F9E79F,stroke:#E67E22,stroke-width:2px;
    classDef controller fill:#AED6F1,stroke:#3498DB,stroke-width:2px;
    classDef client fill:#D5F5E3,stroke:#28B463,stroke-width:2px;
    classDef dao fill:#F5B7B1,stroke:#C0392B,stroke-width:2px;
    classDef oauth fill:#D2B4DE,stroke:#8E44AD,stroke-width:2px;
    classDef util fill:#FADBD8,stroke:#D35400,stroke-width:2px;
    classDef external fill:#F0F3F4,stroke:#7B7D7D,stroke-width:2px,stroke-dasharray: 5 5;
```
