# hezongyao/lmf architecture

> Architecture diagram of the GitHub repository hezongyao/lmf, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hezongyao/lmf
- Repository: https://github.com/hezongyao/lmf

## Overview

##### Main Components

a) Data Flow Pipeline:
- Input Layer (LR Images)
- Feature Extraction Module
- LMF Processing Core
- Latent Space Processing (HD-MLP)
- Render Space Processing (LD-MLP)
- Output Layer (HR Images)

b) Training System:
- Data Loading & Preprocessing
- Model Architecture
- Loss Computation
- Optimization Loop

##### Architectural Components

**[Input Image] → [Feature Extractor (EDSR/RDN/SwinIR)]**
- ↓
**[LMF Core] → [Latent Space HD-MLP] → [Modulation Generation]**
- ↓
**[Render Space] → [LD-MLP] → [Coordinate-based Rendering]**
- ↓
**[Output HR Image]**

##### Implementation Details

- Data flow between components
- Training vs. inference paths
- CMSR optimization loop
- Model configuration loading

##### Technical Specifications

Annotations for:
- Input/output dimensions
- Computational complexity reduction points
- Memory optimization locations
- Scale factor handling

##### Component Relationships

- Forward pass data flow
- Backward pass during training
- Configuration loading paths
- Model parameter updates

##### Additional Elements

- Configuration management (/configs folder)
- Model storage (/save folder)
- Dataset handling (/datasets folder)
- Testing infrastructure

##### The CMSR optimization process

Use a hierarchical layout with:
- Top level: Overall system flow
- Mid level: Component interactions
- Detail level: Specific module implementations

- Component type indicators
- Scale indicators

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph Legend
        L1[Input/Output]:::io
        L2[Feature Extraction]:::feature
        L3[LMF Core Processing]:::lmf
        L4[Rendering]:::render
        L5[Training]:::training
        L6[Support Modules]:::support
    end

    subgraph "Input Layer"
        Input[LR Images]:::io
        DataLoader["Data Loading & Preprocessing"]:::io
    end

    subgraph "Feature Extraction"
        FE["Feature Extractor"]:::feature
        subgraph "Backbone Options"
            EDSR["EDSR"]:::feature
            RDN["RDN"]:::feature
            SwinIR["SwinIR"]:::feature
        end
    end

    subgraph "LMF Core"
        HDMLP["Latent Space HD-MLP"]:::lmf
        ModGen["Modulation Generation"]:::lmf
        LDMLP["Render Space LD-MLP"]:::lmf
    end

    subgraph "Rendering System"
        CoordRender["Coordinate-based Rendering"]:::render
        Output["HR Images"]:::io
    end

    subgraph "Training Components"
        TrainLoop["Training Loop"]:::training
        LossComp["Loss Computation"]:::training
        Optimizer["Optimization"]:::training
    end

    subgraph "CMSR System"
        CMSRCtrl["CMSR Controller"]:::support
        ScaleMap["Scale-to-Modulation Mapping"]:::support
    end

    subgraph "Support Infrastructure"
        Config["Configuration Management"]:::support
        ModelStore["Model Storage"]:::support
        TestInfra["Testing Infrastructure"]:::support
    end

    Input --> DataLoader
    DataLoader --> FE
    FE --> |Select One| EDSR
    FE --> |Select One| RDN
    FE --> |Select One| SwinIR
    EDSR & RDN & SwinIR --> HDMLP
    HDMLP --> ModGen
    ModGen --> LDMLP
    LDMLP --> CoordRender
    CoordRender --> Output
    
    CMSRCtrl --> ScaleMap
    ScaleMap --> ModGen
    
    TrainLoop --> LossComp
    LossComp --> Optimizer
    Optimizer --> FE

    Config --> TrainLoop
    Config --> CMSRCtrl
    ModelStore --> FE
    TestInfra --> CoordRender

    %% Component Mappings
    click EDSR "https://github.com/HeZongyao/LMF/blob/main/models/edsr.py"
    click RDN "https://github.com/HeZongyao/LMF/blob/main/models/rdn.py"
    click SwinIR "https://github.com/HeZongyao/LMF/blob/main/models/swinir.py"
    click HDMLP "https://github.com/HeZongyao/LMF/blob/main/models/lmmlp.py"
    click LDMLP "https://github.com/HeZongyao/LMF/blob/main/models/lmlte.py"
    click DataLoader "https://github.com/HeZongyao/LMF/blob/main/datasets/datasets.py"
    click TrainLoop "https://github.com/HeZongyao/LMF/blob/main/train.py"
    click TestInfra "https://github.com/HeZongyao/LMF/blob/main/test.py"
    click CMSRCtrl "https://github.com/HeZongyao/LMF/blob/main/init_cmsr.py"
    click Config "https://github.com/HeZongyao/LMF/tree/main/configs/train-lmf/"
    click ModelStore "https://github.com/HeZongyao/LMF/tree/main/save/pretrained_models/"

    classDef io fill:#3498db
    classDef feature fill:#2ecc71
    classDef lmf fill:#f1c40f
    classDef render fill:#e67e22
    classDef training fill:#9b59b6
    classDef support fill:#95a5a6
```
