# commreteris/nind-denoise architecture

> Architecture diagram of the GitHub repository commreteris/nind-denoise, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/commreteris/nind-denoise
- Repository: https://github.com/commreteris/nind-denoise

## Overview

##### Project Type and Purpose
- This is a command-line image-denoising tool (nind-denoise) built on PyTorch.
- It wraps a workflow combining Darktable CLI + gmic for preprocessing and an RL-deblur Lua script, then passes the result through a neural network (Unet/UtNet variants) for denoising.
- It also includes training utilities (dataset builders, trainers, config files) to reproduce or fine-tune models.

##### File-Structure Analysis
- Top-level:
- denoise.py            — CLI entry point for inference
- pyproject.toml, requirements.in — Python build/config files
- src/config/operations.yaml — high-level pipeline definition
- src/nind_denoise/:
- common/libs/…         — shared helper modules (I/O, JSON, image ops, PyTorch utilities)
- configs/*.yaml        — default & per-experiment training/validation configs
- datasets & dataset_torch_3.py — CSV-based dataset loader for training/validation
- denoise_dir.py, denoise_image.py — batched & single-image inference drivers
- loss.py, nnModules.py, nn_common.py, graph_utils.py — training/inference logic
- networks/…            — neural-network architectures (Unet, UtNet, third-party nets)
- nn_train.py           — training orchestrator (loads config, dataset, model, runs training loop)
- tools/…               — scripts to download/crop datasets, pick validation set, graph results
- models/…                — timestamped model checkpoints + training metadata
- src/lua-scripts/        — RL-deblur implementation in Lua, invoked via gmic or Darktable CLI

##### README Insights
- Describes end-user workflow: `python3 src/denoise.py <raw_image> [options]`
- External dependencies: Darktable CLI, gmic, Lua script, GPU/CPU acceleration via `torch.accelerator`
- Automatic download of pretrained models from a Backblaze B2 bucket
- Training requirements and dataset generation steps

##### Detail Is Key
- Every major step: preprocessing scripts, config load, model fetch, inference call, output save
- In training flow, show where common/libs utilities plug in (loss calculation, graphing)
- Don’t lump “all code” into a single monolith—split by directory/function (common, networks, tools, trainer)

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Inference Pipeline
    subgraph "Inference Pipeline"
        direction TB
        User["User"]:::external
        CLI["CLI Frontend"]:::python
        ConfigMgr["Config Manager"]:::python
        Darktable["Darktable CLI"]:::external
        GMIC["GMIC + RL-Deblur"]:::external
        BatchedInf["Batched Inference Driver"]:::python
        SingleInf["Single-Image Inference Driver"]:::python
        DataLoader["Inference Dataset Loader"]:::python
        DenoiseEngine["Denoising Engine"]:::python
        ModelStore["Model Store"]:::storage
        OutputWriter["Output Writer"]:::python
    end

    %% Training Pipeline
    subgraph "Training Pipeline"
        direction TB
        TrainConfig["Experiment Configs"]:::python
        DatasetBuilder["Dataset Builder & Tools"]:::python
        Trainer["Trainer Orchestrator"]:::python
        TrainModelStore["Model Store"]:::storage
    end

    %% Inference Flow
    User -->|"runs"| CLI
    CLI -->|"loads"| ConfigMgr
    ConfigMgr -->|"preprocess steps"| Darktable
    Darktable -->|"TIFF"| GMIC
    GMIC -->|"TIFF"| DataLoader
    ConfigMgr -->|"select model"| ModelStore
    ModelStore -->|"weights"| DenoiseEngine
    DataLoader -->|"tensors"| DenoiseEngine
    DenoiseEngine -->|"denoised tensors"| OutputWriter
    OutputWriter -->|"JPEG"| User

    %% Inference Drivers integration
    CLI --> BatchedInf
    CLI --> SingleInf
    BatchedInf --> DataLoader
    SingleInf --> DataLoader

    %% Training Flow
    TrainConfig --> DatasetBuilder
    DatasetBuilder --> Trainer
    Trainer --> TrainModelStore

    %% Click Events
    click CLI "https://github.com/commreteris/nind-denoise/blob/master/src/denoise.py"
    click BatchedInf "https://github.com/commreteris/nind-denoise/blob/master/src/nind_denoise/denoise_dir.py"
    click SingleInf "https://github.com/commreteris/nind-denoise/blob/master/src/nind_denoise/denoise_image.py"
    click ConfigMgr "https://github.com/commreteris/nind-denoise/blob/master/src/config/operations.yaml"
    click TrainConfig "https://github.com/commreteris/nind-denoise/tree/master/src/nind_denoise/configs/"
    click GMIC "https://github.com/commreteris/nind-denoise/blob/master/src/lua-scripts/nind_denoise_rl.lua"
    click DataLoader "https://github.com/commreteris/nind-denoise/blob/master/src/nind_denoise/dataset_torch_3.py"
    click ModelStore "https://github.com/commreteris/nind-denoise/tree/master/src/nind_denoise/models/"
    click TrainModelStore "https://github.com/commreteris/nind-denoise/tree/master/src/nind_denoise/models/"
    click Trainer "https://github.com/commreteris/nind-denoise/blob/master/src/nind_denoise/nn_train.py"
    click DatasetBuilder "https://github.com/commreteris/nind-denoise/tree/master/src/nind_denoise/tools/"

    %% Styles
    classDef python fill:#d0e7ff,stroke:#036,stroke-width:1px;
    classDef external fill:#d0ffd0,stroke:#070,stroke-width:1px;
    classDef storage fill:#ffe0b2,stroke:#c66,stroke-width:1px;
```
