# michaelt919/kintsugi3dbuilder architecture

> Architecture diagram of the GitHub repository michaelt919/kintsugi3dbuilder, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-04-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/michaelt919/kintsugi3dbuilder
- Repository: https://github.com/michaelt919/kintsugi3dbuilder

## Overview

1. **What this repo is**
   - Kintsugi 3D Builder is a desktop Java 11 application for turning flash-photography image sets and photogrammetry reconstructions into empirically derived material textures.
   - It is not a backend service or web app; the architecture is centered on a JavaFX UI plus an OpenGL rendering and optimization engine.

2. **Top-level runtime shape**
   - The app starts in `kintsugi3d.builder.app.Kintsugi3DBuilder` / `javafx.core.MainApplication`.
   - At runtime, JavaFX owns the windows and workflow UI, while a separate rendering layer drives OpenGL canvases and shader execution.
   - The build/package setup also includes OS-specific launch and installer assets for Windows, macOS, and distribution packaging.

3. **UI and workflow layer**
   - The JavaFX layer is organized by experience/workflow: create project, light calibration, mask options, specular basis fit, specular texture fit, export model, system settings, and help/about.
   - Controllers live under `src/kintsugi3d/builder/javafx/controllers/...`, with FXML in `src/main/resources/fxml/...`.
   - The “paged” controller classes suggest wizard-like multi-step flows, while the `experience` package coordinates modal and workflow screens.

4. **Application state and model boundary**
   - State is split between project data, scene configuration, and user/system preferences.
   - The `state` and `core` packages define the canonical models: project, camera, lighting, environment, object pose, tabs/cards, and general settings.
   - There are synchronized/observable wrappers under `javafx/multithread` and `javafx/internal` to bridge JavaFX thread updates with rendering/background work.

5. **Project and source-data ingestion**
   - Input comes from photogrammetry outputs and capture metadata, mainly through readers for Agisoft Metashape, RealityCapture CSV, and a custom VSET format.
   - The `io` layer also handles project directories, view-set serialization, LOD generation, image export, and texture export.
   - This is the boundary where raw source data becomes the application’s internal `ViewSet`/`ProjectModel` representation.

6. **Rendering subsystem**
   - Rendering is a substantial subsystem under `src/kintsugi3d/builder/rendering`, backed by the custom OpenGL framework in `src/kintsugi3d/gl`.
   - The `gl` package provides the low-level abstraction layer for contexts, framebuffers, textures, shaders, vertex buffers, windowing, and event handling.
   - Higher-level rendering components compose scene elements such as camera widgets, lights, backplates, environment maps, split-screen views, and lit/standard scenes.

7. **Shader-driven visual pipeline**
   - The `shaders/` tree contains the actual GPU programs used for display, reconstruction, color appearance, specular fitting, scene rendering, and test/synthetic cases.
   - Architecturally, the shaders are the core execution units for both visualization and several processing stages; much of the “heavy lifting” is GPU-based rather than CPU-only.
   - The shader groups map closely to functional domains: scene rendering, render modes/material display, specular fit, and helper/common math.

8. **Optimization and reconstruction pipeline**
   - The `optimization` and `builder/fit` packages implement the scientific/reconstruction side of the app.
   - Main stages include specular decomposition, normal optimization, roughness optimization, diffuse/albedo/ORM finalization, and reconstruction/error evaluation.
   - These stages use least-squares, non-negative optimization, SVD, clustering, and shader-based error calculation to derive material parameters from the captured images.

9. **Resource and asset management**
   - The app maintains explicit resource layers for images, cached graphics resources, preview generation, and stream-based render resources.
   - `resources/project` and `resources/project/specular` manage cacheable textures and material resources tied to a project.
   - This boundary separates transient GPU resources from durable project data and exportable outputs.

10. **Interaction and manipulation tools**
   - User interaction with the 3D scene is modularized in `builder/tools`.
   - Tools cover orbit/pan/dolly, first-person navigation, object orientation, light adjustment, environment rotation, picking, and parameter toggles.
   - These tools act as input-to-state adapters: mouse/keyboard events feed the scene models, which then drive rendering updates.

11. **Persistence and preferences**
   - User preferences and system settings are serialized separately from project content.
   - The `preferences` package manages directory settings and global user preferences, with Jackson-based serialization support.
   - Project state itself has its own serialization path through view-set and specular-fit serializers, which is important for drawing a diagram showing durable vs. runtime-only state.

12. **Export and companion-tool integration**
   - Outputs include standard textures, GLTF-related data, screenshots, resampled views, orbit animations, and model exports.
   - The repo is designed to interoperate with external viewers, especially the companion Kintsugi 3D Viewer application, and with downstream tools such as Sketchfab-style pipelines.
   - Packaging scripts and installer configs reflect that the Builder is meant to produce artifacts consumed by another app or by standard 3D ecosystems.

13. **Build/distribution structure**
   - Maven is the primary build system (`pom.xml`), with assembly descriptors for distribution and platform-specific launchers/installers.
   - Windows and macOS packaging are first-class concerns, including bundled runtimes, NSIS installer support, and macOS app resources.
   - The repository also includes sample test assets and documentation PDFs, but those are ancillary to the runtime architecture.

14. **Suggested architectural diagram boundaries**
   - A good diagram should show four main zones: JavaFX UI/workflows, project/state models, rendering/OpenGL+shader pipeline, and fit/optimization/export pipeline.
   - The key data flow is: imported photogrammetry/capture data → internal project/view-set models → interactive visualization and calibration → optimization/reconstruction → exported textures/materials/models.

Main entry points: [`src/kintsugi3d/builder/javafx/core/MainApplication.java`](https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/builder/javafx/core/MainApplication.java) (FX core), [`src/kintsugi3d/builder/javafx/internal/ObservableProjectModel.java`](https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/builder/javafx/internal/ObservableProjectModel.java) (UI state), [`src/kintsugi3d/builder/rendering/ProjectRenderingEngine.java`](https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/builder/rendering/ProjectRenderingEngine.java) (Render engine), [`src/kintsugi3d/builder/state/scene/LightingEnvironmentModel.java`](https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/builder/state/scene/LightingEnvironmentModel.java) (Scene state)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Runtime"]
  node_app_boot["App bootstrap<br/>javafx launcher"]
  node_fx_core["FX core<br/>javafx shell"]
  node_fx_experience["Experiences<br/>workflow coordinator<br/>[Experience.java]"]
  node_fx_pages["Paged UI<br/>wizard pages<br/>[Page.java]"]
  node_fx_scene["Scene UI<br/>scene controllers"]
  node_fx_state["UI state<br/>observable models"]
  node_project_model[("Project model<br/>core state<br/>[ProjectModel.java]")]
  node_scene_state[("Scene state<br/>manipulable models")]
  node_tooling["Interaction tools<br/>input adapters<br/>[ToolBox.java]"]
end

subgraph group_pipeline["Pipeline"]
  node_project_io["Project IO<br/>session persistence<br/>[ProjectIO.java]"]
  node_ingest_io["Import IO<br/>view-set ingest<br/>[ViewSetReader.java]"]
  node_export_io["Export IO<br/>artifact export<br/>[GLTFExporter.java]"]
  node_fit_pipeline["Fit pipeline<br/>reconstruction"]
  node_opt_core["Optimization<br/>math solvers"]
end

subgraph group_graphics["Graphics"]
  node_render_engine["Render engine<br/>viewer runtime"]
  node_render_scene["Scene render<br/>render components<br/>[StandardScene.java]"]
  node_gl_layer["OpenGL layer<br/>graphics framework<br/>[Context.java]"]
  node_shader_suite["Shaders<br/>gpu programs<br/>[subjectMain.glsl]"]
  node_resources["Resources<br/>asset caches"]
end

subgraph group_platform["Platform"]
  node_prefs["Preferences<br/>user settings"]
  node_packaging["Packaging<br/>distribution assets<br/>[pom.xml]"]
end

node_app_boot -->|"starts"| node_fx_core
node_fx_core -->|"opens workflows"| node_fx_experience
node_fx_core -->|"uses pages"| node_fx_pages
node_fx_core -->|"hosts scene UI"| node_fx_scene
node_fx_core -->|"binds state"| node_fx_state
node_fx_state -->|"wraps"| node_project_model
node_fx_state -->|"wraps"| node_scene_state
node_project_io -->|"loads"| node_ingest_io
node_ingest_io -->|"builds"| node_project_model
node_fit_pipeline -->|"drives"| node_opt_core
node_fit_pipeline -->|"uses caches"| node_resources
node_export_io -->|"exports from"| node_resources
node_render_engine -->|"composes"| node_render_scene
node_render_engine -->|"runs on"| node_gl_layer
node_render_scene -->|"executes"| node_shader_suite
node_gl_layer -->|"loads"| node_shader_suite
node_tooling -->|"mutates"| node_scene_state
node_scene_state -->|"feeds"| node_render_engine
node_prefs -->|"configures"| node_fx_core
node_packaging -->|"bundles"| node_app_boot
node_packaging -.->|"ships"| node_prefs

click node_app_boot "https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/builder/app/Kintsugi3DBuilder.java"
click node_fx_core "https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/builder/javafx/core/MainApplication.java"
click node_fx_experience "https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/builder/javafx/experience/Experience.java"
click node_fx_pages "https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/builder/javafx/controllers/paged/Page.java"
click node_fx_scene "https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/builder/javafx/controllers/scene/RootSceneController.java"
click node_fx_state "https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/builder/javafx/internal/ObservableProjectModel.java"
click node_project_model "https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/builder/state/project/ProjectModel.java"
click node_scene_state "https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/builder/state/scene/LightingEnvironmentModel.java"
click node_project_io "https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/builder/javafx/core/ProjectIO.java"
click node_ingest_io "https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/builder/io/ViewSetReader.java"
click node_export_io "https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/builder/io/gltf/GLTFExporter.java"
click node_fit_pipeline "https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/builder/fit/SpecularFitProcess.java"
click node_opt_core "https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/optimization/ShaderBasedOptimization.java"
click node_render_engine "https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/builder/rendering/ProjectRenderingEngine.java"
click node_render_scene "https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/builder/rendering/components/StandardScene.java"
click node_gl_layer "https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/gl/core/Context.java"
click node_shader_suite "https://github.com/michaelt919/kintsugi3dbuilder/blob/main/shaders/subject/subjectMain.glsl"
click node_tooling "https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/builder/tools/ToolBox.java"
click node_resources "https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/builder/resources/project/GraphicsResources.java"
click node_prefs "https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/builder/preferences/GlobalUserPreferencesManager.java"
click node_packaging "https://github.com/michaelt919/kintsugi3dbuilder/blob/main/pom.xml"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_app_boot,node_fx_core,node_fx_experience,node_fx_pages,node_fx_scene,node_fx_state,node_project_model,node_scene_state,node_tooling toneBlue
class node_project_io,node_ingest_io,node_export_io,node_fit_pipeline,node_opt_core toneAmber
class node_render_engine,node_render_scene,node_gl_layer,node_shader_suite,node_resources toneMint
class node_prefs,node_packaging toneRose
```

## Components

### Runtime

Application startup, UI workflow, and interactive scene state

- [App bootstrap](https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/builder/app/Kintsugi3DBuilder.java): javafx launcher · Desktop entrypoint and window synchronization setup. · `src/kintsugi3d/builder/app/Kintsugi3DBuilder.java`
- [FX core](https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/builder/javafx/core/MainApplication.java): javafx shell · Main application, window orchestration, and project/session wiring. · `src/kintsugi3d/builder/javafx/core/MainApplication.java`
- [Experiences](https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/builder/javafx/experience/Experience.java): workflow coordinator · Modal and wizard-like workflow flows for create-project, calibration, fit, export, settings, and about. · `src/kintsugi3d/builder/javafx/experience/Experience.java`
- [Paged UI](https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/builder/javafx/controllers/paged/Page.java): wizard pages · Reusable page/controller framework for multi-step dialogs and data-bound flows. · `src/kintsugi3d/builder/javafx/controllers/paged/Page.java`
- [Scene UI](https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/builder/javafx/controllers/scene/RootSceneController.java): scene controllers · Root scene, camera, lights, environment, and object settings panels. · `src/kintsugi3d/builder/javafx/controllers/scene/RootSceneController.java`
- [UI state](https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/builder/javafx/internal/ObservableProjectModel.java): observable models · JavaFX-facing observable and synchronized models bridging UI, background work, and rendering. · `src/kintsugi3d/builder/javafx/internal/ObservableProjectModel.java`
- [Project model](https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/builder/state/project/ProjectModel.java): core state · Canonical project, view-set, camera, lighting, environment, and pose state. · `src/kintsugi3d/builder/state/project/ProjectModel.java`
- [Scene state](https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/builder/state/scene/LightingEnvironmentModel.java): manipulable models · Interactive camera, light, environment, and object pose models used by the viewer. · `src/kintsugi3d/builder/state/scene/LightingEnvironmentModel.java`
- [Interaction tools](https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/builder/tools/ToolBox.java): input adapters · Orbit, pan, dolly, first-person, picking, light, and object manipulation tools. · `src/kintsugi3d/builder/tools/ToolBox.java`

### Pipeline

Import, reconstruction, optimization, and export

- [Project IO](https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/builder/javafx/core/ProjectIO.java): session persistence · Project loading, recent projects, and session-level file operations. · `src/kintsugi3d/builder/javafx/core/ProjectIO.java`
- [Import IO](https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/builder/io/ViewSetReader.java): view-set ingest · Readers and writers for Metashape XML, RealityCapture CSV, and VSET project data. · `src/kintsugi3d/builder/io/ViewSetReader.java`
- [Export IO](https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/builder/io/gltf/GLTFExporter.java): artifact export · Texture, model, screenshot, animation, resample, and glTF export paths. · `src/kintsugi3d/builder/io/gltf/GLTFExporter.java`
- [Fit pipeline](https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/builder/fit/SpecularFitProcess.java): reconstruction · Specular decomposition, normal/roughness optimization, and final material reconstruction. · `src/kintsugi3d/builder/fit/SpecularFitProcess.java`
- [Optimization](https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/optimization/ShaderBasedOptimization.java): math solvers · Least-squares, non-negative optimization, clustering, SVD, and shader-based error calculation. · `src/kintsugi3d/optimization/ShaderBasedOptimization.java`

### Graphics

Rendering engine, OpenGL layer, and shader programs

- [Render engine](https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/builder/rendering/ProjectRenderingEngine.java): viewer runtime · Project rendering engine and reconstruction/scene viewport coordination. · `src/kintsugi3d/builder/rendering/ProjectRenderingEngine.java`
- [Scene render](https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/builder/rendering/components/StandardScene.java): render components · Base scene, lit scene, split-screen, light calibration, and subject rendering components. · `src/kintsugi3d/builder/rendering/components/StandardScene.java`
- [OpenGL layer](https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/gl/core/Context.java): graphics framework · Contexts, framebuffers, textures, shaders, vertex buffers, windows, and event plumbing. · `src/kintsugi3d/gl/core/Context.java`
- [Shaders](https://github.com/michaelt919/kintsugi3dbuilder/blob/main/shaders/subject/subjectMain.glsl): gpu programs · GPU programs for scene rendering, color appearance, render modes, and specular fitting. · `shaders/subject/subjectMain.glsl`
- [Resources](https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/builder/resources/project/GraphicsResources.java): asset caches · Project graphics resources, previews, image caches, and stream-based render resources. · `src/kintsugi3d/builder/resources/project/GraphicsResources.java`

### Platform

Preferences, packaging, and launch assets

- [Preferences](https://github.com/michaelt919/kintsugi3dbuilder/blob/main/src/kintsugi3d/builder/preferences/GlobalUserPreferencesManager.java): user settings · Global preferences, directory settings, and serialization for persistent user configuration. · `src/kintsugi3d/builder/preferences/GlobalUserPreferencesManager.java`
- [Packaging](https://github.com/michaelt919/kintsugi3dbuilder/blob/main/pom.xml): distribution assets · Maven build, assembly descriptors, platform launchers, and installer resources. · `pom.xml`

## Connections

- App bootstrap → FX core: starts · Bootstrap hands off to the JavaFX application and window lifecycle.
- FX core → Experiences: opens workflows · Main UI launches modal and wizard-like experiences.
- FX core → Paged UI: uses pages · Wizard dialogs are built from reusable paged controllers.
- FX core → Scene UI: hosts scene UI · Main window embeds camera, light, environment, and object panels.
- FX core → UI state: binds state · JavaFX models mirror application and project state across threads.
- UI state → Project model: wraps · Observable project state is a UI-facing view of canonical project data.
- UI state → Scene state: wraps · Observable scene models bridge interaction controls and rendering.
- Project IO → Import IO: loads · Project/session IO relies on source-data readers and writers.
- Import IO → Project model: builds · Imported capture data becomes internal project and view-set models.
- Fit pipeline → Optimization: drives · Reconstruction stages use the solver and error-evaluation core.
- Fit pipeline → Resources: uses caches · Optimization stages consume cached project graphics and specular resources.
- Export IO → Resources: exports from · Exporters read from project graphics/material resources.
- Render engine → Scene render: composes · The engine assembles the scene-specific render components.
- Render engine → OpenGL layer: runs on · Rendering is backed by the custom OpenGL abstraction layer.
- Scene render → Shaders: executes · Scene components are shader-driven GPU render passes.
- OpenGL layer → Shaders: loads · Programs and shader objects are built from repository shader sources.
- Interaction tools → Scene state: mutates · Interaction tools update camera, object, and lighting state.
- Scene state → Render engine: feeds · Manipulated scene state is rendered in the viewport and calibration views.
- Preferences → FX core: configures · User preferences affect application startup and runtime settings.
- Packaging → App bootstrap: bundles · Build and installer assets package the desktop entrypoint.
- Packaging → Preferences: ships · Distribution includes platform-specific configuration and persisted settings support.
