silveroxides/convert_to_quant

1. What this repository is

- convert_to_quant is a Python package and CLI tool that converts model weights stored as safetensors into quantized weights for ComfyUI inference.

- The central architectural flow is: load model weights → choose quantization format/preset → calibrate/optimize → write a new quantized safetensors file with ComfyUI-compatible metadata.

2. Primary entry points

- There are two front doors into the same pipeline: the CLI in convert_to_quant/cli/main.py and the module API exposed from convert_to_quant/__init__.py and convert_to_quant/convert_to_quant.py.

- The CLI argument parser (convert_to_quant/cli/argument_parser.py) maps flags like --int8, --mxfp8, --nvfp4, --comfy_quant, and model presets into the conversion pipeline.

3. Core orchestration layer

- convert_to_quant/convert_to_quant.py acts as the top-level coordinator.

- It wires together model loading, layer selection, calibration, conversion, metadata generation, and output serialization.

- This is the boundary where user intent becomes a concrete conversion plan.

4. Format-specific conversion subsystem

- The convert_to_quant/formats/ package contains the actual quantization-format logic:

- fp8_conversion.py

- int8_conversion.py

- mxfp8_conversion.py

- nvfp4_conversion.py

- hybrid_mxfp8_conversion.py

- format_migration.py

- These modules encapsulate how weights are encoded for each target format and how mixed/hybrid layer strategies are represented.

- legacy_utils.py suggests backward-compatibility handling for older metadata or format layouts.

5. Optimization and learned rounding

- The convert_to_quant/converters/ package contains converter classes and optimization variants:

- base_converter.py defines the shared conversion interface/behavior.

- learned_rounding.py, learned_mxfp8.py, and learned_nvfp4.py implement error-reduction strategies.

- mxfp8_converter.py and nvfp4_converter.py specialize the format conversion path.

- The README’s “learned rounding” and “bias correction” features are implemented as a distinct optimization layer on top of raw format conversion, not as part of the storage format itself.

6. Calibration and numeric analysis

- calibrate_activation_scales.py indicates a calibration stage used to measure activation/scale behavior before finalizing quantization.

- Utility modules such as utils/float_utils.py, utils/tensor_utils.py, and utils/convrot.py support numerical transforms, error analysis, and rounding/rotation-related preprocessing.

- This is the part of the pipeline that improves accuracy, especially when calibration samples and synthetic data are used.

7. Model-specific filtering and layer policy

- config/layer_config.py, constants.py, and the README’s preset flags (--flux2, --t5xxl, --hunyuan, --wan) define which layers should be kept in higher precision or excluded from quantization.

- The repo supports both preset-based filtering and explicit regex-based exclusions, which means the conversion pipeline is not “quantize everything”; it is policy-driven at the layer boundary.

8. ComfyUI compatibility boundary

- convert_to_quant/comfy/ and utils/comfy_quant.py are the integration layer for ComfyUI.

- comfy/float.py, comfy/quant_ops.py, and comfy/int8_kernels.py appear to provide custom tensor/operation behavior needed for ComfyUI to consume the produced weights correctly.

- The output format includes metadata and optional ComfyUI-specific layer information so the converted file can be loaded like a normal model in that ecosystem.

9. Memory and runtime constraints

- utils/memory_efficient_loader.py and pinned_transfer.py show that the pipeline is designed for large model weights and constrained GPU/CPU memory.

- The README’s --low-memory mode and the presence of OOM recovery tests indicate that memory management is a first-class concern, not an afterthought.

- For INT8, the architecture also depends on optional Triton-backed kernels, which affects runtime behavior and performance on supported systems.

10. Validation and error measurement

- The tests/ directory is unusually important here: it includes functional tests, CLI consistency checks, OOM recovery, hybrid-format coverage, and INT8/convrot-specific tests.

- The quantization error measurement scripts and documentation in tests/ indicate that numerical fidelity is a major acceptance criterion.

- In architectural terms, the tests validate both the conversion logic and the “does this still work in ComfyUI?” integration boundary.

11. Overall data flow for an architecture diagram

- Input: safetensors model file plus CLI/module options.

- Policy stage: layer presets, exclusions, and custom format selection.

- Calibration/optimization stage: activation-scale measurement, learned rounding, bias correction.

- Conversion stage: format-specific encoder path for FP8, INT8, NVFP4, MXFP8, or hybrid layouts.

- Output: quantized safetensors file with metadata and ComfyUI compatibility artifacts.

12. How to think about the repository boundary

- This repo is not a model trainer; it is a post-training weight conversion and optimization tool.

- Its main external dependencies are PyTorch, safetensors, CUDA-capable hardware, and optionally Triton/comfy-kitchen depending on the target format.

- The key architectural seam is between generic model weights and format-aware, ComfyUI-ready quantized weights.