# matersim/lego-xtal architecture

> Architecture diagram of the GitHub repository matersim/lego-xtal, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-11-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/matersim/lego-xtal
- Repository: https://github.com/matersim/lego-xtal

## Overview

##### Project Type and Purpose
- LEGO-Xtal is an open-source, research-grade pipeline for AI-driven crystal structure generation.
- It consists of a data-processing layer, generative-model training layer, structure-optimization layer, energy-ranking layer, and database/visualization layer.

##### High-Level Components
a. Data Layer
- Source databases (data/source/*.db)
- Training CSVs (data/train/*.csv)
b. Generative-Model Layer
- Model scripts (1_train_sample.py)
- Core libraries (`lego/GAN.py`, `lego/VAE.py`, `lego/base.py`, `lego/builder.py`)
c. Pre-Relaxation Layer
- Script (2_relax.py)
- Batch routines (`lego/Batch_Pre-relaxation/*`)
d. Energy-Ranking Layer
- Script (3_energy.py)
- Energy modules (`lego/basinhopping.py`, `lego/util.py`)
e. Database & Visualization Layer
- DB management (`misc/dbs/*.py`)
- Figures & demos (`misc/Figs/*.py`, `misc/demos/*.py`)
f. Utility & Deployment
- SLURM scripts (`util/slurm/*.sh`)
- VASP interface (`util/vasp/vasp.py`)
- Env config (environment.yml)

##### Simplified Workflow with Code-Stage Purposes
Stage 1: Data Collection & Augmentation
- Code: `0_make_traindata.py`
- Purpose: Extract structural descriptors from source DBs; encode space group, cell parameters, Wyckoff sites; augment symmetry-equivalent representations.
- Helpers: `lego/data_transformer.py`, `misc/dbs/db_fetch.py`, `symdata-aug.py`.

Stage 2: Generative-Model Training
- Code: `1_train_sample.py`
- Purpose: One-hot encode augmented features; train GANs/VAEs (via `lego/GAN.py`, `lego/VAE.py`) to learn crystal-feature distributions.
- Config: environment.yml, any hyperparameters in script.

Stage 3: Pre-Relaxation of Generated Structures
- Code: `2_relax.py`
- Purpose: Decode model outputs into trial crystal structures; optimize local environments using L-BFGS/Adam through `lego/batch_opt.py` and `SO3.py`.
- Batch scripts: `util/slurm/run_relax.sh` for HPC submission.

Stage 4: Energy Ranking & Database Update
- Code: `3_energy.py`
- Purpose: Evaluate pre-relaxed structures with ReaxFF, MACE, or DFT (via `util/vasp/vasp.py`); rank by energy; update results DB.
- DB scripts: `misc/dbs/db_register_vasp.py`, `db_add.py`.

Stage 5: Visualization & Demos
- Code: `misc/Figs/*.py`, `misc/demos/*.py`
- Purpose: Generate publication-quality figures (energy frameworks, distance plots, convergence) and demo scripts for optimization workflows.

##### Architectural Patterns & Design Principles
- Layered pipeline: clear separation of concerns (data → model → optimization → ranking → visualization).
- Modular Python packages under `lego/` for core algorithms.
- Script wrappers at top level for orchestration.
- Utility subfolders (`util/`, `misc/`) for cross-cutting concerns (job submission, database, plotting).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Layer
    subgraph "Data Layer"
        DB["Source Databases"]:::data
        CSV["Training CSVs"]:::data
    end

    %% Data Prep & Augmentation
    subgraph "Data Prep & Augmentation"
        TDATA["0_make_traindata.py<br/><i>extracts & augments descriptors</i>"]:::compute
        DTRANS["lego/data_transformer.py"]:::compute
        DBFETCH["misc/dbs/db_fetch.py"]:::compute
        SYMDATA["misc/dbs/symdata-aug.py"]:::compute
    end

    %% Generative-Model Training
    subgraph "Model Training"
        TRAIN["1_train_sample.py<br/><i>trains GAN/VAE on features</i>"]:::compute
        GAN["lego/GAN.py"]:::compute
        VAE["lego/VAE.py"]:::compute
        BASE["lego/base.py"]:::compute
        BUILDER["lego/builder.py"]:::compute
        ENV["environment.yml"]:::compute
    end

    GS["Generated Structures"]:::data

    %% Pre-Relaxation
    subgraph "Pre-Relaxation"
        RELAX["2_relax.py<br/><i>optimizes structures</i>"]:::compute
        BOPT["lego/Batch_Pre-relaxation/batch_opt.py"]:::compute
        SO3["lego/Batch_Pre-relaxation/SO3.py"]:::compute
        BSYM["lego/Batch_Pre-relaxation/batch_sym.py"]:::compute
        WYLK["lego/Batch_Pre-relaxation/wyckoff_list.csv"]:::compute
        RUNRELAX["util/slurm/run_relax.sh"]:::external
    end

    OPTS["Optimized Structures"]:::data

    %% Energy Evaluation
    subgraph "Energy Evaluation"
        ENERGY["3_energy.py<br/><i>evaluates & ranks by energy</i>"]:::compute
        BHOP["lego/basinhopping.py"]:::compute
        UTILMOD["lego/util.py"]:::compute
        VASPUTIL["util/vasp/vasp.py"]:::external
        DBREG["misc/dbs/db_register_vasp.py"]:::compute
        DBADD["misc/dbs/db_add.py"]:::compute
    end

    RANK["Ranked Results"]:::data

    %% Database & Visualization
    subgraph "Database & Visualization"
        DBM["Database (results)"]:::db
        FIGS["misc/Figs/*.py<br/><i>plots figures</i>"]:::output
        DEMOS["misc/demos/*.py<br/><i>demo workflows</i>"]:::output
    end

    %% Utility & Deployment
    subgraph "Utility & Deployment"
        RUNTRAIN["util/slurm/run_train.sh"]:::external
        VASPDEP["util/vasp/vasp.py"]:::external
        UTSH["util/*.sh"]:::external
        CHECK["util/check_*.py"]:::external
        QUERY["util/query.py"]:::external
        ENV2["environment.yml"]:::compute
    end

    %% Data Flow
    DB -->|"fetch"| TDATA
    TDATA -->|"augment"| CSV

    CSV -->|"train"| TRAIN
    TRAIN --> GAN
    TRAIN --> VAE
    TRAIN --> BASE
    TRAIN --> BUILDER
    TRAIN --> ENV
    TRAIN --> GS

    GS -->|"optimize"| RELAX
    RELAX --> BOPT
    RELAX --> SO3
    RELAX --> BSYM
    RELAX --> WYLK
    RELAX --> RUNRELAX
    RELAX --> OPTS

    OPTS -->|"evaluate"| ENERGY
    ENERGY --> BHOP
    ENERGY --> UTILMOD
    ENERGY --> VASPUTIL
    ENERGY --> DBREG
    ENERGY --> DBADD
    ENERGY --> RANK

    RANK -->|"store"| DBM
    DBM -->|"plot"| FIGS
    DBM -->|"demo"| DEMOS

    %% Utility interactions
    RUNTRAIN -.-> TRAIN
    RUNRELAX -.-> RELAX
    VASPUTIL -.-> ENERGY
    ENV2 -.-> TRAIN

    %% Click Events
    click DB "https://github.com/matersim/lego-xtal/blob/main/data/source/*.db"
    click CSV "https://github.com/matersim/lego-xtal/blob/main/data/train/*.csv"
    click TDATA "https://github.com/matersim/lego-xtal/blob/main/0_make_traindata.py"
    click DTRANS "https://github.com/matersim/lego-xtal/blob/main/lego/data_transformer.py"
    click DBFETCH "https://github.com/matersim/lego-xtal/blob/main/misc/dbs/db_fetch.py"
    click SYMDATA "https://github.com/matersim/lego-xtal/blob/main/misc/dbs/symdata-aug.py"
    click TRAIN "https://github.com/matersim/lego-xtal/blob/main/1_train_sample.py"
    click GAN "https://github.com/matersim/lego-xtal/blob/main/lego/GAN.py"
    click VAE "https://github.com/matersim/lego-xtal/blob/main/lego/VAE.py"
    click BASE "https://github.com/matersim/lego-xtal/blob/main/lego/base.py"
    click BUILDER "https://github.com/matersim/lego-xtal/blob/main/lego/builder.py"
    click ENV "https://github.com/matersim/lego-xtal/blob/main/environment.yml"
    click RELAX "https://github.com/matersim/lego-xtal/blob/main/2_relax.py"
    click BOPT "https://github.com/matersim/lego-xtal/blob/main/lego/Batch_Pre-relaxation/batch_opt.py"
    click SO3 "https://github.com/matersim/lego-xtal/blob/main/lego/Batch_Pre-relaxation/SO3.py"
    click BSYM "https://github.com/matersim/lego-xtal/blob/main/lego/Batch_Pre-relaxation/batch_sym.py"
    click WYLK "https://github.com/matersim/lego-xtal/blob/main/lego/Batch_Pre-relaxation/wyckoff_list.csv"
    click RUNRELAX "https://github.com/matersim/lego-xtal/blob/main/util/slurm/run_relax.sh"
    click ENERGY "https://github.com/matersim/lego-xtal/blob/main/3_energy.py"
    click BHOP "https://github.com/matersim/lego-xtal/blob/main/lego/basinhopping.py"
    click UTILMOD "https://github.com/matersim/lego-xtal/blob/main/lego/util.py"
    click VASPUTIL "https://github.com/matersim/lego-xtal/blob/main/util/vasp/vasp.py"
    click DBREG "https://github.com/matersim/lego-xtal/blob/main/misc/dbs/db_register_vasp.py"
    click DBADD "https://github.com/matersim/lego-xtal/blob/main/misc/dbs/db_add.py"
    click FIGS "https://github.com/matersim/lego-xtal/blob/main/misc/Figs/*.py"
    click DEMOS "https://github.com/matersim/lego-xtal/blob/main/misc/demos/*.py"
    click RUNTRAIN "https://github.com/matersim/lego-xtal/blob/main/util/slurm/run_train.sh"
    click VASPDEP "https://github.com/matersim/lego-xtal/blob/main/util/vasp/vasp.py"
    click UTSH "https://github.com/matersim/lego-xtal/blob/main/util/*.sh"
    click CHECK "https://github.com/matersim/lego-xtal/blob/main/util/check_*.py"
    click QUERY "https://github.com/matersim/lego-xtal/blob/main/util/query.py"
    click ENV2 "https://github.com/matersim/lego-xtal/blob/main/environment.yml"

    %% Styles
    classDef data fill:#D0E6FF,stroke:#0366d6,stroke-width:1px
    classDef compute fill:#DFFFE0,stroke:#2c662d,stroke-width:1px,stroke-dasharray: 5 2
    classDef external fill:#E0E0E0,stroke:#606060,stroke-width:1px,stroke-dasharray: 2 2
    classDef db fill:#FFD8A8,stroke:#d96e00,stroke-width:1px
    classDef output fill:#FFF5D0,stroke:#d6ac00,stroke-width:1px,stroke-dasharray: 3 3
```
