# htcondor/htcondor-ce architecture

> Architecture diagram of the GitHub repository htcondor/htcondor-ce, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/htcondor/htcondor-ce
- Repository: https://github.com/htcondor/htcondor-ce

## Overview

##### Main Architectural Components
- HTCondor-CE Core: Representing the wrapper layer, as seen in the “src” directory. This includes multiple command wrappers (e.g., condor_ce_q, condor_ce_submit, condor_ce_router_defaults, etc.) that interface with the native HTCondor components.
- Configuration Layer: The “config” directory holds a vast array of configuration files for different components (e.g., collector, router, and batch system defaults). This layer ensures that the system can adapt to different environments.
- External Dependencies and Integration Scripts: The “contrib” directory (with subdirectories for Puppet configs, BDII, and Bosco scripts) suggests support for automated deployment and integration. This demonstrates how the system can be extended or deployed in grid environments.
- Testing and Containerization: The “tests” directory (with containers setups and Dockerfiles) is included to validate configurations and simulate different host setups (local and remote submission).
- Web Interface Components: The “templates” directory along with the “src/htcondorce” package suggests that there is a web-based interface or monitoring capability (including static assets).

##### Relationships and Interactions
- Configuration and Logging: Include elements representing where configuration files (from the config directory and contrib scripts) feed into the system’s behavior. Also show the “Disk” element as a repository for original and routed job advertisements.

##### Architectural Patterns and Design Principles
- Layered Architecture: The system separates concerns between the HTCondor-CE wrapper layer, the native HTCondor components, and external configuration/deployment layers.
- Extensibility: With core functionalities abstracted as thin wrappers, additional functionality can be added without changing the underlying batch system.
- Integration Focus: The use of standardized configuration files and interfacing (via scripts and puppet templates) shows a design built for integration and flexibility.

##### Relevant Technologies and Frameworks
- HTCondor components are central, so clearly show SchedD, Collector, and related daemons.
- Scripting Languages: The source files and test scripts are mostly Python and shell scripts.
- Container Tools: Docker and possibly other container orchestration systems (as suggested in the “tests/containers” directory).
- Continuous Integration: GitHub workflows and linters show a modern development lifecycle integration.

##### Key Elements
- Clearly Labeled Nodes: Each key component (e.g., SchedD, Job Router, Grid Manager, Disk, Central Collector, SSH components for remote submission) should have a clear, descriptive label.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Frontend Layer
    subgraph "Frontend Layer"
        A1["HTCondor-CE Core"]:::internal
        A2["condor_ce_q"]:::internal
        A3["condor_ce_submit"]:::internal
        A4["local-wrapper"]:::internal
        A1 --> A2
        A1 --> A3
        A1 --> A4

        subgraph "Job Router"
            JR1["condor_ce_job_router_info"]:::internal
            JR2["condor_ce_router_defaults"]:::internal
            JR3["condor_ce_router_q"]:::internal
            JR1 --> JR2
            JR2 --> JR3
        end
        A1 --> JR1

        %% Web Interface split into two components
        WI1["Templates"]:::web
        WI2["Interface Logic"]:::web
        A1 -->|"UI_update"| WI1
        A1 -->|"UI_update"| WI2
    end

    %% Middleware Layer
    subgraph "Middleware Layer"
        GM["Grid Manager"]:::internal
        LBS["Local Batch Submission"]:::internal
        RS["SSH Remote Submission"]:::external
        GM --> LBS
        GM --> RS
    end

    %% Backend Layer
    subgraph "Backend Layer"
        SchedD["HTCondor SchedD"]:::internal
        CC["Central Collector"]:::internal
        Disk["Job Disk Repository"]:::internal
        SchedD --> CC
    end

    %% Supporting Components
    subgraph "Supporting Components"
        CF["Configuration Files"]:::config
        ED["External Dependencies"]:::external
        Test["Tests & Containers"]:::test
    end

    %% Inter-layer Connections
    A1 -->|"submits"| SchedD
    JR3 -->|"routes"| GM
    GM -->|"logs"| Disk

    CF -->|"configures"| A1
    CF -->|"configures"| GM
    ED -->|"integrates"| CF
    Test -->|"tests"| A1

    %% Class Definitions
    classDef internal fill:#AEE,stroke:#000,stroke-width:2px;
    classDef config fill:#CFC,stroke:#070,stroke-width:2px;
    classDef external fill:#FBB,stroke:#900,stroke-width:2px;
    classDef test fill:#FFB,stroke:#990,stroke-width:2px;
    classDef web fill:#BEE,stroke:#039,stroke-width:2px;

    %% Click Events
    click A1 "https://github.com/htcondor/htcondor-ce/tree/main/src"
    click JR1 "https://github.com/htcondor/htcondor-ce/tree/main/src/condor_ce_job_router_info"
    click JR2 "https://github.com/htcondor/htcondor-ce/tree/main/src/condor_ce_router_defaults"
    click JR3 "https://github.com/htcondor/htcondor-ce/tree/main/src/condor_ce_router_q"
    click GM "https://github.com/htcondor/htcondor-ce/tree/main/config"
    click CF "https://github.com/htcondor/htcondor-ce/tree/main/config"
    click ED "https://github.com/htcondor/htcondor-ce/tree/main/contrib"
    click Test "https://github.com/htcondor/htcondor-ce/tree/main/tests"
    click WI1 "https://github.com/htcondor/htcondor-ce/tree/main/templates"
    click WI2 "https://github.com/htcondor/htcondor-ce/tree/main/src/htcondorce"
```
