How panda000001/windassimilation works
Project Type and Purpose
• This is a research-oriented machine learning pipeline, not a web or mobile app.
• It integrates Computational Fluid Dynamics (CFD) simulation (via SOWFA), data ingestion, PINN training (PyTorch), hyperparameter search (Weights & Biases), inference/reconstruction, transfer learning, and visualization.
File Structure
• Top-level directories and files:
• SOWFA.script: contains CFD case setup, mesh definition, boundary/initial conditions, solver control dictionaries, mesh refinement, sampling plans, and job submission scripts (sub1_pre.sh, sub2_run.sh).
• Sensor.U8.XY.h5: HDF5 outputs from SOWFA containing raw flow data for various probes and lidar measurements.
• Visualization: Jupyter notebooks or static assets for figures and GIFs.
• Python scripts:
• read_data.py: ingests and preprocesses CFD/HDF5 data.
• pinn_model.py: defines the PINN architecture and physics constraints in PyTorch.
• main.py: orchestrates training loop, logging, hyperparameter search (wandb).
• pred_write.py: runs inference to reconstruct flow fields and writes HDF5 outputs.
• transfer.py: implements online transfer learning for real-time data assimilation.
• requirement.txt: lists dependencies (PyTorch, h5py, wandb, etc.)
Extract Insights from README
• Workflow stages:
a. Generate or use pre-computed CFD data via SOWFA.script.
b. Read and preprocess data (read_data.py).
c. Train PINN with physics constraints (main.py + pinn_model.py), using wandb for HPO.
d. Reconstruct or predict flow fields (pred_write.py).
e. Deploy and continually fine-tune with transfer learning on incoming measurements (transfer.py).
• External integrations: SOWFA CFD engine, HDF5 data format, wandb service.
Main Components and Their Interactions
• Component A: SOWFA Simulation Subsystem
• Generates ground-truth flow fields.
• Outputs HDF5 files under Sensor.U8.XY.h5.
• Component B: Data Ingestion & Preprocessing
• read_data.py reads HDF5, extracts probe/velocity/pressure fields, builds tensors.
• Component C: PINN Model Definition
• pinn_model.py defines neural network layers and physics‐based loss components (RANS equations, eddy viscosity).
• Component D: Training Orchestrator
• main.py sets up training loops, optimizer, scheduling, wandb logging and HPO.
• Component E: Inference & Reconstruction
• pred_write.py loads trained model, runs on spatial grid or new data, writes reconstructed fields to HDF5.
• Component F: Transfer Learning Module
• transfer.py ingest streaming data, fine-tunes the pre-trained PINN for online deployment.
• Component G: Results Storage & Visualization
• Visualization assets generate static plots and animations (GIFs) from HDF5 results.
• External Service: wandb for hyperparameter search, experiment tracking.
Relationships and Data Flow
• SOWFA.script → HDF5 Data Store
• HDF5 → read_data.py → Tensor Datasets
• Datasets → main.py + pinn_model.py → Trained Model
• Trained Model → pred_write.py → Reconstructed HDF5
• Streaming Measurements → transfer.py + Trained Model → Updated Model → New HDF5
• All models, metrics, logs → wandb
• Final HDF5 → Visualization tools → Figures/GIFs
Architectural Patterns and Principles
• Pipeline Pattern: clear sequential stages from data generation to visualization.
• Modular Design: separate scripts for ingestion, model definition, training, inference, transfer learning.
• Single Responsibility: each component has one well-defined role.
• Extensibility: new measurement types or physics constraints can be added by extending read_data or pinn_model.