# spring-projects/spring-batch-extensions architecture

> Architecture diagram of the GitHub repository spring-projects/spring-batch-extensions, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/spring-projects/spring-batch-extensions
- Repository: https://github.com/spring-projects/spring-batch-extensions

## Overview

This project is a multi-module library that extends the Spring Batch framework with support for several external systems such as Google BigQuery, Elasticsearch, Microsoft Excel, Apache Geode, Neo4j, and Notion. The main purpose is to provide specialized readers, writers, and builder components that help integrate Spring Batch with these systems in batch processing jobs.

##### Main components
- Core Spring Batch Integration – the central part of the library that hooks into Spring Batch.
- Individual Extension Modules – separate components for each supported technology:
- spring-batch-bigquery: Provides item readers and writers (including builders) tailored to Google BigQuery (e.g., BigQueryQueryItemReader and CSV/JSON writers).
- spring-batch-elasticsearch: Contains components for reading from and writing to Elasticsearch.
- spring-batch-excel: Implements integration with Microsoft Excel, covering both POI and streaming variants, along with necessary mappers and support classes.
- spring-batch-geode: Implements integration with Apache Geode through specialized writers.
- spring-batch-neo4j: Provides components to interact with Neo4j for batch jobs.
- spring-batch-notion: Offers functionality to integrate with Notion using tailored readers, mappers, and filters.

##### Relationships and interactions
- A central node labeled “Spring Batch Extensions Core” or “Spring Batch Integration Layer” which connects to all individual extension modules.
- For each module, include its key elements (e.g., ItemReader, ItemWriter, Builder classes) as sub-components.

##### Architectural patterns and design principles
- That the project follows a modular design with loose coupling. Each extension is packaged as an independent Maven module and can be built, tested, and released independently.
- Point out the use of builder patterns (e.g., BigQueryQueryItemReaderBuilder or Neo4jItemReaderBuilder) that help in constructing complex objects.
- How Spring Batch’s Enterprise Integration patterns (ItemReader, ItemWriter, Job, Step) are extended with technology-specific implementations.
- Optionally, add another layer for CI/CD, showing GitHub workflows (available in the .github/workflows folder) that trigger Maven builds and tests for each module.

##### Relevant technologies and frameworks
- Spring Batch as the core batch-processing engine.
- Maven for build management (build and dependency management is evident with pom.xml files).
- External systems: Google BigQuery, Elasticsearch, Microsoft Excel, Apache Geode, Neo4j, and Notion.

- Sub-components (readers, writers, builders, mappers) within each module.
- Optionally, include a secondary layer to show the build and CI/CD aspect with GitHub workflows linked to each module.

## Diagram (Mermaid)

```mermaid
graph TD
    CORE["Spring Batch Integration Core"]:::core

    subgraph "Spring Batch Extensions - BigQuery"
        BQ["spring-batch-bigquery"]:::module
        BQReader["BigQueryQueryItemReader"]:::extension
        BQBuilder["BigQueryQueryItemReaderBuilder"]:::extension
        ExternalBQ["Google BigQuery"]:::external
        BQBuilder -->|"builds"| BQReader
        BQReader -->|"integrates"| ExternalBQ
    end

    subgraph "Spring Batch Extensions - Elasticsearch"
        ES["spring-batch-elasticsearch"]:::module
        ESReader["ElasticsearchItemReader"]:::extension
        ESWriter["ElasticsearchItemWriter"]:::extension
        ExternalES["Elasticsearch"]:::external
        ESReader -->|"reads"| ExternalES
        ESWriter -->|"writesTo"| ExternalES
    end

    subgraph "Spring Batch Extensions - Excel"
        EX["spring-batch-excel"]:::module
        Poi["PoiItemReader"]:::extension
        Streaming["StreamingXlsxItemReader"]:::extension
        ExternalExcel["Microsoft Excel"]:::external
        Poi -->|"readsExcel"| ExternalExcel
        Streaming -->|"streamsExcel"| ExternalExcel
    end

    subgraph "Spring Batch Extensions - Geode"
        GE["spring-batch-geode"]:::module
        GemfireWriter["GemfireItemWriter"]:::extension
        GemfireBuilder["GemfireItemWriterBuilder"]:::extension
        ExternalGeode["Apache Geode"]:::external
        GemfireBuilder -->|"builds"| GemfireWriter
        GemfireWriter -->|"writes"| ExternalGeode
    end

    subgraph "Spring Batch Extensions - Neo4j"
        NEO["spring-batch-neo4j"]:::module
        NeoReader["Neo4jItemReader"]:::extension
        NeoWriter["Neo4jItemWriter"]:::extension
        NeoBuilder["Neo4jItemReaderBuilder"]:::extension
        ExternalNeo4j["Neo4j"]:::external
        NeoBuilder -->|"builds"| NeoReader
        NeoReader -->|"reads"| ExternalNeo4j
        NeoWriter -->|"writes"| ExternalNeo4j
    end

    subgraph "Spring Batch Extensions - Notion"
        NOT["spring-batch-notion"]:::module
        NotionReader["NotionDatabaseItemReader"]:::extension
        Filter["Filter"]:::extension
        Sort["Sort"]:::extension
        Mapping["Mapping Classes"]:::extension
        ExternalNotion["Notion"]:::external
        NotionReader -->|"mapsTo"| Mapping
        Mapping -->|"uses"| Filter
        NotionReader -->|"reads"| ExternalNotion
    end

    subgraph "CI/CD Pipelines"
        CI["GitHub Workflows"]:::ci
    end

    CORE --> BQ
    CORE --> ES
    CORE --> EX
    CORE --> GE
    CORE --> NEO
    CORE --> NOT

    CI --- BQ
    CI --- ES
    CI --- EX
    CI --- GE
    CI --- NEO
    CI --- NOT

    classDef core fill:#85C1E9,stroke:#1B4F72,stroke-width:2px;
    classDef module fill:#ABEBC6,stroke:#229954,stroke-width:2px;
    classDef extension fill:#F9E79F,stroke:#B7950B,stroke-width:2px;
    classDef external fill:#F5B7B1,stroke:#C0392B,stroke-width:2px;
    classDef ci fill:#D2B4DE,stroke:#7D3C98,stroke-width:2px;

    click CORE "https://github.com/spring-projects/spring-batch-extensions/tree/main/src/main/java/org/springframework/batch/extensions"
    click BQ "https://github.com/spring-projects/spring-batch-extensions/tree/main/spring-batch-bigquery"
    click ES "https://github.com/spring-projects/spring-batch-extensions/tree/main/spring-batch-elasticsearch"
    click EX "https://github.com/spring-projects/spring-batch-extensions/tree/main/spring-batch-excel"
    click GE "https://github.com/spring-projects/spring-batch-extensions/tree/main/spring-batch-geode"
    click NEO "https://github.com/spring-projects/spring-batch-extensions/tree/main/spring-batch-neo4j"
    click NOT "https://github.com/spring-projects/spring-batch-extensions/tree/main/spring-batch-notion"
    click CI "https://github.com/spring-projects/spring-batch-extensions/tree/main/.github/workflows"
```
