How hsraktu/major_project works
Project Type and Purpose
• This is an open-source Python ML pipeline for predicting CFD (computational fluid dynamics) fields.
• Key functionality: ingest raw CFD data, preprocess into slices/masks, train 3D-CNN/PointNet models, test/infer, and demo results.
File Structure and Map to Architectural Components
• Top-level scripts:
• data_prepare.py – initial data‐preparation orchestration.
• run_demo.ipynb – end‐to‐end demo notebook.
• preprocessing/ – module that slices, pads, and visualizes CFD data.
• training/ – dataset loader and training orchestration (train.py).
• models/ – model definitions (cd_predictor_3dcnn.py) and experimental notebooks.
• testing/ – test data, test harness scripts, and validation notebooks.
• requirements*.txt – lists Python dependencies (e.g., Paddle or PyTorch, NumPy, etc.).
README
• Confirms the goal: build and evaluate a learning-based CFD predictor.
Sketch the Main Components and Their Interactions
a. Data Sources
• Raw CFD simulation outputs (e.g., CSV or binary files under testing/).
b. Preprocessing Module
• Scripts in preprocessing/ and data_prepare.py
• Tasks: slice generation, padding/masking, visualization.
c. Storage Layer
• Temporary file store for processed slices (on-disk folders).
d. Training Module
• training/dataset_loader.py loads processed slices.
• training/train.py runs model training, emits trained model artifacts.
e. Model Definitions
• models/cd_predictor_3dcnn.py (3D-CNN architecture)
• Experiment notebooks for PointNet2D.
f. Testing/Inference Module
• testing/test.py, model_setup.ipynb, test notebooks
• Loads trained model, runs inference on new CFD inputs, outputs paddle_tensor files or metrics.
g. Demo/Visualization
• run_demo.ipynb – ties together preprocessing, model loading, and downstream visualization.
Architectural Patterns and Design Principles
• Pipeline Pattern: clear staging from raw data → preprocessing → training → inference.
• Modularity: separate directories per concern (preprocessing, models, training, testing).
• Extensibility: drop-in new model definitions under models/, new preprocessing scripts under preprocessing/.
• Reproducibility: notebooks (run_demo, model experiments, tests) document end-to-end flows.
Technologies and Frameworks to Annotate
• Python 3.x
• ML frameworks: Paddle or PyTorch (as per requirements.txt)
• Data libraries: NumPy, pandas (possibly)
• Visualization: Matplotlib or custom scripts in preprocessing/visualize_slices.py
• Jupyter Notebooks for experiments and demos.
Final Advice
• Pipeline sequence and data handoffs between modules.
• The result should clearly reflect the file structure under these categories, making the architecture self‐explanatory.