How satoshi-kosugi/pg-ia-nilut works
Main Components
• Core Model & Lookup Tables: Represented by the “adaint” directory, especially the model implementation (model.py) and prompt guidance loss, as well as the “ailut_transform” extension that implements the neural implicit lookup tables (with C++/CUDA code in “csrc”).
• Dataset and Preprocessing: Include the “dataset.py”, “transforms.py”, and annotation files (in “annfiles”). This component handles dataset ingestion (FiveK and PPR10K) and data augmentation.
• Configuration & Training Scripts: Show the configuration files under “configs”, and training/testing routines in “tools” (e.g., tools/train.py, tools/test.py) that drive the experiments.
• MMEditing Toolbox: Represented by the “mmedit” directory. This is an external toolbox modified to support CLIP-based evaluation (CLIP-IQA integration) and provides additional model architectures, common layers, optimizers, schedulers, dataset handling pipelines, etc.
• Pretrained Models & Demo: The “pretrained” directory and demo scripts (demo.py in “adaint” and tools/demo.py) should be highlighted since they play a role in both inference and visualization.
• External Dependencies: Indicate that compiled extensions are built via setup scripts (setup.py in “ailut_transform” and setup_demo.py), and that the project relies on specific versions of PyTorch, CUDA, mmcv, and other libraries.
Architectural Patterns and Design Principles
• Modularity: Emphasize the separation between the core “adaint” functionality and the externally extended “mmedit” toolbox. Each major folder represents a separate layer or module.
• High-performance Extensions: Indicate that “ailut_transform” includes compiled code (with separate branches for CPU and CUDA implementations) that plug-in into the higher-level architecture.
Relevant Technologies, Frameworks, and Libraries
• Python 3.7.11, PyTorch 1.11.0 for deep learning.
• CUDA 11.3 for GPU acceleration.
• mmcv and MMEditing libraries as key dependencies.
• CLIP-based methods integrated within the MMEditing extensions for image quality assessment.