# apache/fory architecture

> Architecture diagram of the GitHub repository apache/fory, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/apache/fory
- Repository: https://github.com/apache/fory

## Overview

##### Project type and purpose
- This is a multi-language **serialization framework** (“Apache Fory™”) that provides high-performance, cross-language object serialization/deserialization and a zero-copy row format.
- It ships implementations in Java, C++, Python, Go, Rust, JavaScript, Kotlin, Scala, and Dart—each following a common protocol specification (Xlang, Java-native, Row Format, etc.).

##### File structure
- Top-level directories correspond to language implementations: `java/`, `cpp/`, `python/`, `go/`, `rust/`, `javascript/`, `kotlin/`, `scala/`, `dart/`.
- A `bazel/` directory and root `BUILD`/`WORKSPACE` indicate a Bazel-based, polyglot build setup (for C++, Python, Java, etc.).
- CI & release scripts under `.github/` and `ci/` coordinate multi-language builds, tests, packaging, and deployments.
- Shared documentation in `docs/` and integration tests in `integration_tests/` demonstrate cross-language compatibility.

##### README for additional insights
- The README emphasizes **core features**: JIT compilation (Java), static codegen (C++, Rust, Go), zero-copy row format, meta-sharing, and cross-language protocols.
- It lists **protocol specifications** (Xlang, Java Serialization, Row Format) and **user guides** per language, implying that each language implementation must adhere to the central specifications.

##### Main components and how they interact
a. **Monorepo Root**
- Contains global build definitions (Bazel, Maven/Gradle POMs, Cargo.toml, setup.py, package.json).
- Houses CI workflows that orchestrate building, testing, and packaging each language artifact.
- **Java**: Modules `fory-core`, `fory-format`, `fory-simd`, `fory-extensions`, test suites. Uses Maven/Gradle, JIT compilation (Janino), GraalVM native-image.
- **Python**: Under `python/pyfory`: Cython wrappers around the C++ core plus pure-Python fallbacks. Implements the same protocols via `_serializer`, `_struct`, `format`, etc.
- **Go**: Under `go/fory`: idiomatic Go implementation with static code-generation via `go generate`.
- **Rust**: Under `rust/`: `fory-core`, `fory-derive` (proc-macros), benches/tests.
- **JavaScript**: Under `javascript/packages/fory`: TypeScript implementation plus an optional native extension (`hps`).
- **Kotlin**, **Scala**, **Dart**: Lightweight wrappers on the Java or native core, providing language-specific serializers.

c. **Protocol Specifications & Codegen**
- The **Xlang Serialization Spec** and **Row Format Spec** (under `docs/specification/`) define the wire formats and type-mapping rules.
- Each implementation uses either code generation (Java JIT, Rust derive macro, Go generator, C++ templates) or dynamic resolution (Python, JS) to produce efficient serialization/deserialization code conforming to these specs.

d. **Integration & Testing**
- `integration_tests/` contains cross-language test suites and benchmarks (e.g., `cpython_benchmark`, `graalvm_tests`, JDK compatibility tests).
- Each language’s `test/` folder verifies compliance with the specs and measures performance.

##### Architectural patterns and design principles
- **Polyglot Modularization**: Each language lives in its own directory/module, but they share a unified specification.
- **Code Generation vs. Dynamic**: Static codegen (Rust Proc-Macros, Go generator, C++ templates) vs. dynamic resolution (Python Cython, JS TypeScript).
- **Separation of Concerns**: Build logic separated from language-specific logic; protocol spec separated from implementation.
- **Micro-modules**: Java approach uses small Maven modules (`fory-core`, `fory-format`, etc.) to allow optional features (SIMD, format).

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Monorepo Root"
        MR1["WORKSPACE"] 
        MR2["BUILD"] 
        MR3["bazel/"] 
        MR4[".github/workflows/"] 
        MR5["ci/"] 
    end
    class MR1,MR2,MR3,MR4,MR5 monorepo

    subgraph "Protocol Specifications"
        PS1["xlang_serialization_spec.md"] 
        PS2["row_format_spec.md"] 
        PS3["java_serialization_spec.md"] 
        PS4["xlang_type_mapping.md"] 
    end
    class PS1,PS2,PS3,PS4 spec

    subgraph "Language Implementations"
        subgraph Java
            J["Java"] 
        end
        subgraph Cpp
            CPP["C++"] 
        end
        subgraph Python
            PY["Python"] 
        end
        subgraph Go
            GO["Go"] 
        end
        subgraph Rust
            RUST["Rust"] 
        end
        subgraph JavaScript
            JS["JavaScript"] 
        end
        subgraph Kotlin
            KOT["Kotlin"] 
        end
        subgraph Scala
            SCALA["Scala"] 
        end
        subgraph Dart
            DART["Dart"] 
        end
    end
    class J,CPP,PY,GO,RUST,JS,KOT,SCALA,DART language

    subgraph "Integration Tests & Benchmarks"
        IT["integration_tests/"] 
    end
    class IT integration

    subgraph "CI/CD Pipelines & Release"
        CI["CI/CD (.github & ci)"] 
    end
    class CI ci

    %% Relationships
    PS1 -->|"implements spec"| J
    PS2 -->|"implements spec"| J
    PS3 -->|"implements spec"| J
    PS4 -->|"implements spec"| J

    PS1 -->|"implements spec"| CPP
    PS2 -->|"implements spec"| CPP
    PS4 -->|"implements spec"| CPP

    PS1 -->|"implements spec"| PY
    PS2 -->|"implements spec"| PY
    PS4 -->|"implements spec"| PY

    PS1 -->|"implements spec"| GO
    PS2 -->|"implements spec"| GO
    PS4 -->|"implements spec"| GO

    PS1 -->|"implements spec"| RUST
    PS2 -->|"implements spec"| RUST
    PS4 -->|"implements spec"| RUST

    PS1 -->|"implements spec"| JS
    PS2 -->|"implements spec"| JS
    PS4 -->|"implements spec"| JS

    PS1 -->|"implements spec"| KOT
    PS2 -->|"implements spec"| KOT
    PS4 -->|"implements spec"| KOT

    PS1 -->|"implements spec"| SCALA
    PS2 -->|"implements spec"| SCALA
    PS4 -->|"implements spec"| SCALA

    PS1 -->|"implements spec"| DART
    PS2 -->|"implements spec"| DART
    PS4 -->|"implements spec"| DART

    J -->|"tested by"| IT
    CPP -->|"tested by"| IT
    PY -->|"tested by"| IT
    GO -->|"tested by"| IT
    RUST -->|"tested by"| IT
    JS -->|"tested by"| IT
    KOT -->|"tested by"| IT
    SCALA -->|"tested by"| IT
    DART -->|"tested by"| IT

    CI -->|"builds & releases"| J
    CI -->|"builds & releases"| CPP
    CI -->|"builds & releases"| PY
    CI -->|"builds & releases"| GO
    CI -->|"builds & releases"| RUST
    CI -->|"builds & releases"| JS
    CI -->|"builds & releases"| KOT
    CI -->|"builds & releases"| SCALA
    CI -->|"builds & releases"| DART
    CI -->|"orchestrates tests"| IT

    %% Click Events for Monorepo Root
    click MR1 "https://github.com/apache/fory/tree/main/WORKSPACE"
    click MR2 "https://github.com/apache/fory/tree/main/BUILD"
    click MR3 "https://github.com/apache/fory/tree/main/bazel/"
    click MR4 "https://github.com/apache/fory/tree/main/.github/workflows/"
    click MR5 "https://github.com/apache/fory/tree/main/ci/"

    %% Click Events for Protocol Specifications
    click PS1 "https://github.com/apache/fory/blob/main/docs/specification/xlang_serialization_spec.md"
    click PS2 "https://github.com/apache/fory/blob/main/docs/specification/row_format_spec.md"
    click PS3 "https://github.com/apache/fory/blob/main/docs/specification/java_serialization_spec.md"
    click PS4 "https://github.com/apache/fory/blob/main/docs/specification/xlang_type_mapping.md"

    %% Click Events for Language Implementations
    click J "https://github.com/apache/fory/tree/main/java/fory-core/"
    click CPP "https://github.com/apache/fory/tree/main/cpp/fory/"
    click PY "https://github.com/apache/fory/tree/main/python/pyfory/"
    click GO "https://github.com/apache/fory/tree/main/go/fory/"
    click RUST "https://github.com/apache/fory/tree/main/rust/fory-core/"
    click JS "https://github.com/apache/fory/tree/main/javascript/packages/fory/"
    click KOT "https://github.com/apache/fory/tree/main/kotlin/src/main/java/org/apache/fory/serializer/kotlin/"
    click SCALA "https://github.com/apache/fory/tree/main/scala/src/main/java/org/apache/fory/serializer/scala/"
    click DART "https://github.com/apache/fory/tree/main/dart/packages/fory/"

    %% Click Events for Integration Tests & Benchmarks
    click IT "https://github.com/apache/fory/tree/main/integration_tests/"

    %% Click Events for CI/CD
    click CI "https://github.com/apache/fory/tree/main/.github/workflows/"
    click CI "https://github.com/apache/fory/tree/main/ci/"

    %% Styles
    classDef monorepo fill:#e0ffff,stroke:#333,stroke-width:1px
    classDef spec fill:#90EE90,stroke:#333,stroke-width:1px
    classDef language fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef integration fill:#DDA0DD,stroke:#333,stroke-width:1px
    classDef ci fill:#FFA500,stroke:#333,stroke-width:1px
```
