# torchopc/torchlitho architecture

> Architecture diagram of the GitHub repository torchopc/torchlitho, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/torchopc/torchlitho
- Repository: https://github.com/torchopc/torchlitho

## Overview

##### Project type and purpose
- This is an open-source research framework for differentiable computational lithography built in Python.
- It exposes configurable imaging models (Abbe, Hopkins, TCC), analytical resist modeling, and vector lithography routines.
- It leverages PyTorch (+ Lightning) for GPU acceleration and Hydra for flexible experiment management.

##### File structure
- Top-level:
- configs/ – hundreds of YAMLs for callbacks, data, trainer settings, loggers, experiments
- src/ – core code: data modules, model definitions, training & eval scripts, utilities
- scripts/ – wrapper scripts (e.g., schedule.sh)
- notebooks/, misc/ – Jupyter demos, slide decks, figures
- tests/ – pytest suites for configs, data, training, evaluation
- environment.yaml, requirements.txt, setup.py/pyproject.toml – dependency/build spec
- src/data: PyTorch Lightning DataModule(s), data components
- src/models:
- components/simple_dense_net.py (ML baseline)
- litho/: GDSII parsing, layer/material definitions, numerical functions (Calculate*Image, TCC matrices)
- mnist_module.py, gdsii, litho imaging classes (FilmStack, ImagingModel, Layer, Mask, Resist, Source, etc.)
- src/train.py & src/eval.py – entrypoints wired via Hydra
- src/utils & configs/hydra: configuration instantiation, logging, CLI integration

##### README
- Confirms heavy use of PyTorch Lightning, Hydra 1.3, multiple logger backends (MLflow, WandB, Comet, Aim).
- Usage patterns: “python src/train.py trainer.max_epochs=20 data.batch_size=64”
- Emphasis on first‐of‐its‐kind differentiable lithography imaging & resist modeling.

1. Configuration Layer (Hydra)
2. Data Layer
– MNIST DataModule
– GDSII parsing & dataset components
3. Model Layer
– SimpleDenseNet (baseline)
– Lithography Imaging Model
- Optics: FilmStack, ProjectionObjective, Source
- Resist: Resist, Receipe, Numerics
- GDSII handling: record ↔ element ↔ structure
- Imaging functions (Calculate1DAerialImage, …)
4. Training Orchestrator
– PyTorch Lightning Trainer
– Callbacks (early stopping, checkpoints, progress bars)
– Logger plugins (TensorBoard, MLflow, WandB, Comet, Aim)
5. Evaluation & Visualization
– src/eval.py
– Jupyter notebooks for demos (“Imaging.ipynb”)
6. CLI & Scripts
– train.py / eval.py entrypoints
– schedule.sh for job orchestration
- Hydra config → injects into DataModule, Model, Trainer
- DataModule → yields batched tensors to Model
- Model.forward → returns predictions → loss → Trainer
- Trainer → invokes Callbacks & Logger → writes checkpoints & metrics
- Eval script or notebook → loads checkpoint → runs forward → generates visualizations
c. Architectural patterns & design principles:
– Modular layering: separation of data, model, training orchestration, evaluation
– Plugin architecture via Hydra (config‐over‐code)
– Abstraction of training loop via Lightning Trainer & callbacks
– Extensible logger interface (multiple backends)
– Functional decomposition of imaging routines
d. Key technologies, frameworks, libraries:
– Python 3.8–3.10, PyTorch ≥2.0, PyTorch Lightning ≥2.0
– Hydra 1.3 for config management
– pytest for testing
– CI via pre-commit, black style
– Logging: TensorBoard, MLflow, WandB, Comet, Aim

##### Final advice
– Be exhaustive: split “Model” into its subcomponents (Optics, Resist, GDSII).
– Capture both control-plane (Hydra/Cli) and data-plane (tensors/parameters) flows.
– Review the “configs/experiment” folder to show how different experiments slot into this architecture seamlessly.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Configuration Layer"
        HydraConfig["Hydra Config"]:::config
    end

    subgraph "Configs"
        ExpCfg["configs/experiment/"]:::config
        TrainerCfg["configs/trainer/"]:::config
        CallbackCfg["configs/callbacks/"]:::config
        LoggerCfg["configs/logger/"]:::config
        DataCfg["configs/data/"]:::config
        HydraRuntime["configs/hydra/default.yaml"]:::config
    end

    subgraph "Data Layer"
        MNIST["MNIST DataModule"]:::core
        GDSII["GDSII Parser & Dataset"]:::core
    end

    subgraph "Model Layer"
        subgraph "Baseline Model"
            DenseNet["SimpleDenseNet"]:::core
            MnistModule["MNIST LightningModule"]:::core
        end
        subgraph "Lithography ImagingModel"
            ImagingModel["ImagingModel"]:::core
            subgraph Optics
                FilmStack["FilmStack"]:::core
                ProjectionObjective["ProjectionObjective"]:::core
                Source["Source"]:::core
            end
            subgraph Resist
                Resist["Resist"]:::core
                Recipe["Receipe"]:::core
            end
            subgraph Numerics
                Numerics["Numerics Orchestrator"]:::core
                Functions["Aerial/Resist Imaging & TCC Functions"]:::core
            end
            LayerComp["Layer Abstraction"]:::core
            MaskComp["Mask / Masks Abstraction"]:::core
            Material["Material Definitions"]:::core
        end
    end

    subgraph "Training Orchestrator"
        Trainer["Lightning Trainer"]:::framework
        Callbacks["Callbacks"]:::core
        Loggers["Logger Plugins"]:::core
        TrainScript["train.py"]:::script
    end

    subgraph "Evaluation & Visualization"
        EvalScript["eval.py"]:::script
        Notebooks["Demo Notebooks"]:::script
    end

    subgraph "CLI & Scripts"
        Schedule["schedule.sh"]:::script
    end

    subgraph "Utilities"
        Instantiator["instantiators.py"]:::core
        LoggingUtils["logging_utils.py"]:::core
        PyLogger["pylogger.py"]:::core
        RichUtils["rich_utils.py"]:::core
    end

    %% Relationships
    HydraConfig -->|injects config| MNIST
    HydraConfig -->|injects config| GDSII
    HydraConfig -->|injects config| DenseNet
    HydraConfig -->|injects config| ImagingModel
    HydraConfig -->|injects config| Trainer
    HydraConfig -->|injects config| TrainScript

    ExpCfg --> HydraConfig
    TrainerCfg --> HydraConfig
    CallbackCfg --> HydraConfig
    LoggerCfg --> HydraConfig
    DataCfg --> HydraConfig
    HydraRuntime --> HydraConfig

    MNIST -->|batches| MnistModule
    MNIST -->|batches| ImagingModel
    GDSII -->|geometry data| ImagingModel

    DenseNet -->|forward| Trainer
    MnistModule -->|forward| Trainer
    ImagingModel -->|forward| Trainer

    TrainScript --> Trainer
    Instantiator --> HydraConfig
    TrainScript --> Instantiator
    EvalScript --> Instantiator

    Trainer -->|uses| Callbacks
    Trainer -->|uses| Loggers

    Trainer -->|checkpoint| EvalScript
    EvalScript -->|visualizes| Notebooks

    Schedule --> TrainScript
    Schedule --> EvalScript

    %% Click Events
    click HydraConfig "https://github.com/torchopc/torchlitho/tree/main/configs/"
    click ExpCfg "https://github.com/torchopc/torchlitho/tree/main/configs/experiment/"
    click TrainerCfg "https://github.com/torchopc/torchlitho/tree/main/configs/trainer/"
    click CallbackCfg "https://github.com/torchopc/torchlitho/tree/main/configs/callbacks/"
    click LoggerCfg "https://github.com/torchopc/torchlitho/tree/main/configs/logger/"
    click DataCfg "https://github.com/torchopc/torchlitho/tree/main/configs/data/"
    click HydraRuntime "https://github.com/torchopc/torchlitho/blob/main/configs/hydra/default.yaml"
    click MNIST "https://github.com/torchopc/torchlitho/blob/main/src/data/mnist_datamodule.py"
    click GDSII "https://github.com/torchopc/torchlitho/tree/main/src/models/gdsii/"
    click DenseNet "https://github.com/torchopc/torchlitho/blob/main/src/models/components/simple_dense_net.py"
    click MnistModule "https://github.com/torchopc/torchlitho/blob/main/src/models/mnist_module.py"
    click ImagingModel "https://github.com/torchopc/torchlitho/blob/main/src/models/litho/ImagingModel.py"
    click FilmStack "https://github.com/torchopc/torchlitho/blob/main/src/models/litho/FilmStack.py"
    click ProjectionObjective "https://github.com/torchopc/torchlitho/blob/main/src/models/litho/ProjectionObjective.py"
    click Source "https://github.com/torchopc/torchlitho/blob/main/src/models/litho/Source.py"
    click Resist "https://github.com/torchopc/torchlitho/blob/main/src/models/litho/Resist.py"
    click Recipe "https://github.com/torchopc/torchlitho/blob/main/src/models/litho/Receipe.py"
    click Numerics "https://github.com/torchopc/torchlitho/blob/main/src/models/litho/Numerics.py"
    click Functions "https://github.com/torchopc/torchlitho/tree/main/src/models/litho/functions/"
    click LayerComp "https://github.com/torchopc/torchlitho/blob/main/src/models/litho/Layer.py"
    click MaskComp "https://github.com/torchopc/torchlitho/blob/main/src/models/litho/Mask.py"
    click Material "https://github.com/torchopc/torchlitho/blob/main/src/models/litho/Material.py"
    click TrainScript "https://github.com/torchopc/torchlitho/blob/main/src/train.py"
    click EvalScript "https://github.com/torchopc/torchlitho/blob/main/src/eval.py"
    click Schedule "https://github.com/torchopc/torchlitho/blob/main/scripts/schedule.sh"
    click Instantiator "https://github.com/torchopc/torchlitho/blob/main/src/utils/instantiators.py"
    click LoggingUtils "https://github.com/torchopc/torchlitho/blob/main/src/utils/logging_utils.py"
    click PyLogger "https://github.com/torchopc/torchlitho/blob/main/src/utils/pylogger.py"
    click RichUtils "https://github.com/torchopc/torchlitho/blob/main/src/utils/rich_utils.py"

    %% Styles
    classDef core fill:#AED6F1,stroke:#1B4F72,stroke-width:1px
    classDef framework fill:#A9DFBF,stroke:#196F3D,stroke-width:1px
    classDef config fill:#F9E79F,stroke:#B9770E,stroke-width:1px
    classDef script fill:#D5D8DC,stroke:#424949,stroke-width:1px
```
