# openimaginglab/timelens-xl architecture

> Architecture diagram of the GitHub repository openimaginglab/timelens-xl, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/openimaginglab/timelens-xl
- Repository: https://github.com/openimaginglab/timelens-xl

## Overview

##### Project Type & Purpose
- This is a deep-learning research framework (“TimeLens-XL”) for event-based video frame interpolation (Event-VFI) with large motion in real time.
- Key functionalities: dataset preparation & loading, model definitions (multiple architectures), training/evaluation loops, loss computation, parameter configuration, utilities for debugging & performance.

##### File Structure Analysis
- Top-level directories:
- dataset – all data loaders for various datasets (BSERGB, GOPRO, HQEVFI, etc.)
- models – model implementations grouped by architecture (TimeLens variants, CBMNet, RIFE, SuperSlomo, etc.)
- losses – loss functions (common, LPIPS, DISTS, SSIM)
- params – configuration files for datasets, model hyperparameters, paths
- tools – helper scripts (data conversion, ffmpeg interface, GPU memory tracker, scheduler init, profiling)
- docs – project documentation & static site
- prepare_dataset.py, run_network.py – main CLI entrypoints
- requirements.txt – dependencies

- Patterns:
- Modular separation by concern (data / model / loss / config / utils)
- Config-driven via params/\*.py and command-line flags
- Single-monolith script (run_network.py) invoking dataset loaders, model constructors, training/eval loops, loss modules

A. Data Ingestion Layer (dataset/)
- Raw video/event data → specialized loader classes → batched tensors
- prepare_dataset.py for one-off dataset transforms

B. Configuration Layer (params/)
- BaseParams & per-dataset/per-experiment .py files
- run_network.py reads param_name to inject hyperparameters

C. Model Repository (models/)
- Each subfolder = distinct architecture
- BaseModel → concrete implementations (TLXNet, TLXNet+, Expv8_large, etc.)
- Forward pass: events + RGB frames → feature extractor → fusion → interpolation → output frames

D. Loss & Metrics Layer (losses/)
- Loss functions (L1/L2, SSIM, LPIPS, DISTS)
- run_network.py selects appropriate losses based on config

E. Training/Evaluation Engine (run_network.py)
- Constructs DataLoader ← dataset + params
- Instantiates Model ← models + params
- Selects Losses ← losses + params
- Defines Optimizer & Scheduler ← tools/initOptimScheduler
- Loop: forward → compute losses → backward → optimizer.step → scheduler.step
- Logging & checkpointing (tools/TimeTracker, GPU mem track, model_deparse)
- Metrics & visualization hooks

F. Utilities (tools/)
- Data acceleration, ffmpeg conversions, profiler hooks, model size estimation
- Interface & model deparse for introspection

G. Outputs
- Trained model weights (.pt)
- Evaluation metrics (LPIPS, PSNR, SSIM)
- Logs, figures, demos

##### Architectural Patterns & Principles
- Modular, plug-and-play: swap in/out models, losses, datasets via config
- Dependency injection via params files
- Single responsibility modules (each folder handles one layer)
- CLI orchestration for reproducibility

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Core Scripts
    prepare["prepare_dataset.py"]:::core
    run["run_network.py"]:::core

    %% Data Ingestion Layer
    subgraph "Data Ingestion Layer" 
        dataset["dataset/"]:::data
    end

    %% Configuration Layer
    subgraph "Configuration Layer"
        params_all["params/"]:::config
        base_params["BaseParams.py"]:::config
    end

    %% Model Repository
    subgraph "Model Repository"
        base_model["BaseModel.py"]:::model
        ours_base["OursBaseModel.py"]:::model
        subgraph "Architectures"
            arch1["CBMNet/"]:::model
            arch2["EVDI/"]:::model
            arch3["Expv8_Lights3/"]:::model
            arch4["Expv8_large/"]:::model
            arch5["QVI/"]:::model
            arch6["REFID/"]:::model
            arch7["RIFE/"]:::model
            arch8["SuperSlomo/"]:::model
            arch9["raft/"]:::model
            arch10["timelens/"]:::model
            arch11["timelens_flow/"]:::model
        end
    end

    %% Loss & Metrics Layer
    subgraph "Loss & Metrics Layer"
        losses["losses/"]:::loss
    end

    %% Utilities & Tools
    subgraph "Utilities & Tools"
        tools["tools/"]:::tools
    end

    %% Project Meta
    requirements["requirements.txt"]:::external
    docs["docs/"]:::external

    %% Connections
    prepare -->|"produce transformed data"| dataset
    dataset -->|"batch data"| run
    params_all -->|"hyperparams"| run
    base_params -->|"base config"| params_all
    run -->|"loads model definitions"| base_model
    run -->|"loads model definitions"| ours_base
    run -->|"loads architectures"| arch1
    run -->|"loads architectures"| arch2
    run -->|"loads architectures"| arch3
    run -->|"loads architectures"| arch4
    run -->|"loads architectures"| arch5
    run -->|"loads architectures"| arch6
    run -->|"loads architectures"| arch7
    run -->|"loads architectures"| arch8
    run -->|"loads architectures"| arch9
    run -->|"loads architectures"| arch10
    run -->|"loads architectures"| arch11
    run -->|"selects loss fns"| losses
    run -->|"uses utilities"| tools
    run -->|"ensures deps"| requirements
    run -->|"generates docs/static site"| docs

    %% Click Events
    click dataset "https://github.com/openimaginglab/timelens-xl/tree/main/dataset/"
    click prepare "https://github.com/openimaginglab/timelens-xl/blob/main/prepare_dataset.py"
    click params_all "https://github.com/openimaginglab/timelens-xl/tree/main/params/"
    click base_params "https://github.com/openimaginglab/timelens-xl/blob/main/params/BaseParams.py"
    click base_model "https://github.com/openimaginglab/timelens-xl/blob/main/models/BaseModel.py"
    click ours_base "https://github.com/openimaginglab/timelens-xl/blob/main/models/OursBaseModel.py"
    click arch1 "https://github.com/openimaginglab/timelens-xl/tree/main/models/CBMNet/"
    click arch2 "https://github.com/openimaginglab/timelens-xl/tree/main/models/EVDI/"
    click arch3 "https://github.com/openimaginglab/timelens-xl/tree/main/models/Expv8_Lights3/"
    click arch4 "https://github.com/openimaginglab/timelens-xl/tree/main/models/Expv8_large/"
    click arch5 "https://github.com/openimaginglab/timelens-xl/tree/main/models/QVI/"
    click arch6 "https://github.com/openimaginglab/timelens-xl/tree/main/models/REFID/"
    click arch7 "https://github.com/openimaginglab/timelens-xl/tree/main/models/RIFE/"
    click arch8 "https://github.com/openimaginglab/timelens-xl/tree/main/models/SuperSlomo/"
    click arch9 "https://github.com/openimaginglab/timelens-xl/tree/main/models/raft/"
    click arch10 "https://github.com/openimaginglab/timelens-xl/tree/main/models/timelens/"
    click arch11 "https://github.com/openimaginglab/timelens-xl/tree/main/models/timelens_flow/"
    click losses "https://github.com/openimaginglab/timelens-xl/tree/main/losses/"
    click tools "https://github.com/openimaginglab/timelens-xl/tree/main/tools/"
    click requirements "https://github.com/openimaginglab/timelens-xl/blob/main/requirements.txt"
    click docs "https://github.com/openimaginglab/timelens-xl/tree/main/docs/"

    %% Styles
    classDef data fill:#bbdefb,stroke:#1e88e5,color:#0d47a1;
    classDef model fill:#c8e6c9,stroke:#43a047,color:#1b5e20;
    classDef loss fill:#ffe0b2,stroke:#fb8c00,color:#e65100;
    classDef config fill:#e1bee7,stroke:#8e24aa,color:#4a148c;
    classDef tools fill:#eeeeee,stroke:#757575,color:#424242;
    classDef core fill:#fff9c4,stroke:#fbc02d,color:#f57f17;
    classDef external fill:#f5f5f5,stroke:#9e9e9e,color:#616161;
```
