How toenshoff/anycsp works
Project Type and Purpose
• This is a research-oriented machine learning pipeline for solving various constraint satisfaction problems (3SAT, MaxCut, MaxSAT, Graph Coloring (COL), and RB).
• It provides an end-to-end workflow: configuration → data generation/ingestion → model training → evaluation.
High-Level Architecture Components
a. Orchestration Layer
• Shell scripts (scripts/train_*.sh, scripts/eval_*.sh) invoke the training and evaluation pipelines.
• Entrypoints: train.py for training, evaluate.py (and specialized variants like evaluate_maxcut.py) for evaluation.
b. Configuration Store
• JSON files under configs/<PROBLEM>/<PROBLEM>_default.json. Each defines hyperparameters, dataset paths, training settings.
c. Data Layer
• src/data/ module:
• dataset.py, generators.py, tuple_generator.py: sample or load problem instances.
• xparser.py: parse external data formats.
• intension.py: handle constraint expressions.
d. CSP Abstraction Layer
• src/csp/ module:
• csp_data.py: wraps CSP instances into a common format.
• constraints.py: defines constraint types and interfaces.
e. Model Layer
• src/model/ module:
• model.py: assembles neural network model based on configuration.
• layers.py: custom GNN or other layers.
• loss.py: problem-specific loss functions.
f. Utility Layer
• src/utils/ module:
• config_utils.py: loads and validates JSON configs.
• data_utils.py: common preprocessing helpers.
• rb_utils.py: specialized utilities for “RB” problem.
g. External Dependencies
• PyTorch (+ GPU support), Python 3.10 environment managed by Conda.
• Pre-downloaded datasets (unzipped into project root) from provided Google Drive link.
Relationships and Data Flow
1. User runs a shell script (e.g., bash scripts/train_maxcut.sh).
2. Script calls train.py --config configs/MAXCUT/MAXCUT_default.json.
3. train.py uses config_utils to read JSON → instantiates data generator from src/data → csp_data to convert to graph or tensor inputs.
4. Inputs feed into the model assembled by src/model/model.py, which uses layers in src/model/layers.py.
5. Training loop applies loss from src/model/loss.py, updates parameters via PyTorch.
6. Checkpoints and logs are written to disk.
7. For evaluation, a similar flow via evaluate.py (or problem-specific variant), producing metrics.
Architectural Patterns and Design Principles
• Modular layering: separation of concerns between configuration, data ingestion, domain logic (CSP), model definition, and utilities.
• Config-driven runtime: JSON files define behavior, enabling easy experiment swapping.
• Reusable components: data generators and CSP abstractions usable across all problem types.
• CLI entrypoints for reproducibility (shell scripts + Python scripts).