# artemiymitsel/claude_hack3 architecture

> Architecture diagram of the GitHub repository artemiymitsel/claude_hack3, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-05-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/artemiymitsel/claude_hack3
- Repository: https://github.com/artemiymitsel/claude_hack3

## Overview

1. This repository contains two independent forecasting pipelines for the same wind-farm task. They share one input dataset at the project root, but otherwise do not depend on each other. Each pipeline has its own training script, tuning script(s), dependency list, and output CSV.

2. The shared data boundary is `dataset/`, which contains:
   - `train_dataset.csv`: historical hourly observations from 2022-01 to 2025-12, including the target.
   - `valid_features.csv`: the 2026-01 to 2026-03 forecast horizon, with the target hidden.
   Both solutions read these files relative to their own folder paths and write their final submission file back into their own folder.

3. The first subsystem is `physics/`, a hybrid model that combines domain physics with machine learning. Its core idea is to compute a physical baseline power estimate from wind inputs and the Siemens Gamesa SG 3.4-132 turbine power curve, then model remaining error with a small ensemble of learned regressors.

4. `physics/predict.py` is the main entry point for Solution A. It:
   - computes standalone physics-based predictions,
   - evaluates time-based backtests,
   - trains a fixed ensemble of CatBoost and one histogram gradient boosting model,
   - blends their outputs using tuned weights,
   - and writes `physics/predictions.csv`.
   This is the final production path for that solution.

5. The physics solution has a staged tuning pipeline around it:
   - `tune_physics.py` searches physical constants with Optuna,
   - `phase_c_validate.py` checks candidates end-to-end in the ensemble,
   - `phase_d_optimize_weights.py` re-optimizes blend weights with SLSQP.
   The repository includes JSON artifacts from these phases, which act as saved tuning results and selected configurations.

6. The second subsystem is `catboost_pure/`, a direct supervised learning pipeline with no explicit physics residual stack. It builds engineered features from wind statistics, lags, rolling windows, wind shear/veer, gust ratios, ramps, and a derived capacity-factor feature, then trains a single CatBoost regressor.

7. `catboost_pure/train_predict.py` is the main entry point for Solution B. It:
   - performs 5-fold `TimeSeriesSplit` validation,
   - reports fold-level RMSE/MAE/MAPE,
   - fits a final model on the full training set,
   - and writes `catboost_pure/predictions.csv`.
   The file `best_params.json` stores the tuned CatBoost hyperparameters used by this pipeline.

8. `catboost_pure/tune.py` is the optimization companion for Solution B. It runs an Optuna search to find CatBoost hyperparameters and persists the best configuration back to `best_params.json`. This makes the tuning step distinct from the final training step.

9. Both pipelines are time-series aware and avoid random train/test splits. Validation is based on temporal backtesting, which is important for preserving chronological causality in hourly forecasting. The physics solution uses quarter-based backtests with weighted historical years; the pure CatBoost solution uses rolling folds over time.

10. Operationally, the repo assumes a Python environment with separate dependency sets:
   - `physics/requirements.txt` adds the scientific optimization stack needed for physics calibration and weight fitting.
   - `catboost_pure/requirements.txt` covers the direct model pipeline.
   CatBoost training caches are kept inside each solution folder, and both pipelines fix `SEED = 42` for repeatability.

11. The final architectural output of either path is the same contract: a one-column `predictions.csv` named `Выработка. Результирующий расчет`, with 2126 rows matching the order of `valid_features.csv` and clipped to the farm’s capacity range. The two solutions are independent candidates for submission rather than stages of a single combined system.

Main entry points: [`physics/predict.py`](https://github.com/artemiymitsel/claude_hack3/blob/main/physics/predict.py) (Predict), [`physics/phase_d_optimize_weights.py`](https://github.com/artemiymitsel/claude_hack3/blob/main/physics/phase_d_optimize_weights.py) (Opt weights), [`physics/tune_physics.py`](https://github.com/artemiymitsel/claude_hack3/blob/main/physics/tune_physics.py) (Tune physics), [`catboost_pure/train_predict.py`](https://github.com/artemiymitsel/claude_hack3/blob/main/catboost_pure/train_predict.py) (Train predict)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_shared_data["Shared data"]
  node_dataset_train[("Train data<br/>csv time series<br/>[train_dataset.csv]")]
  node_dataset_valid[("Valid features<br/>csv horizon<br/>[valid_features.csv]")]
end

subgraph group_physics["Physics pipeline"]
  node_physics_deps["Physics deps<br/>requirements<br/>[requirements.txt]"]
  node_physics_tune["Tune physics<br/>optuna search<br/>[tune_physics.py]"]
  node_physics_validate["Phase C validate<br/>backtest validator"]
  node_physics_opt_weights["Opt weights<br/>slsqp blend fit"]
  node_physics_predict["Predict<br/>hybrid forecast<br/>[predict.py]"]
  node_physics_results["Tuning results<br/>json artifacts<br/>[tune_results.json]"]
  node_physics_phase_c_results["Phase C results<br/>json artifacts"]
  node_physics_phase_d_results_narrow["Phase D narrow<br/>json artifacts"]
  node_physics_phase_d_results_wide["Phase D wide<br/>json artifacts"]
  node_physics_phase_d_results_compromise["Phase D compromise<br/>json artifacts"]
  node_physics_predictions["Predictions<br/>csv submission<br/>[predictions.csv]"]
end

subgraph group_catboost["CatBoost pipeline"]
  node_catboost_deps["CatBoost deps<br/>requirements<br/>[requirements.txt]"]
  node_catboost_tune["Tune CatBoost<br/>optuna search<br/>[tune.py]"]
  node_catboost_params["Best params<br/>json config<br/>[best_params.json]"]
  node_catboost_train["Train predict<br/>timeseries cv<br/>[train_predict.py]"]
  node_catboost_predictions["Predictions<br/>csv submission"]
end

node_dataset_train -->|"train data"| node_physics_tune
node_dataset_valid -->|"forecast horizon"| node_physics_predict
node_physics_deps -->|"supports"| node_physics_tune
node_physics_tune -->|"writes"| node_physics_results
node_physics_tune -->|"configures"| node_physics_validate
node_physics_validate -->|"writes"| node_physics_phase_c_results
node_physics_phase_c_results -->|"candidates"| node_physics_opt_weights
node_physics_opt_weights -->|"writes"| node_physics_phase_d_results_narrow
node_physics_opt_weights -->|"writes"| node_physics_phase_d_results_wide
node_physics_opt_weights -->|"writes"| node_physics_phase_d_results_compromise
node_physics_results -->|"tuned params"| node_physics_predict
node_physics_phase_d_results_narrow -->|"blend weights"| node_physics_predict
node_physics_phase_d_results_wide -->|"blend weights"| node_physics_predict
node_physics_phase_d_results_compromise -->|"blend weights"| node_physics_predict
node_physics_predict -->|"writes"| node_physics_predictions
node_dataset_train -->|"train data"| node_catboost_tune
node_dataset_train -->|"train data"| node_catboost_train
node_dataset_valid -->|"forecast horizon"| node_catboost_train
node_catboost_deps -->|"supports"| node_catboost_tune
node_catboost_tune -->|"writes"| node_catboost_params
node_catboost_params -->|"configures"| node_catboost_train
node_catboost_train -->|"writes"| node_catboost_predictions

click node_dataset_train "https://github.com/artemiymitsel/claude_hack3/blob/main/dataset/train_dataset.csv"
click node_dataset_valid "https://github.com/artemiymitsel/claude_hack3/blob/main/dataset/valid_features.csv"
click node_physics_deps "https://github.com/artemiymitsel/claude_hack3/blob/main/physics/requirements.txt"
click node_physics_tune "https://github.com/artemiymitsel/claude_hack3/blob/main/physics/tune_physics.py"
click node_physics_validate "https://github.com/artemiymitsel/claude_hack3/blob/main/physics/phase_c_validate.py"
click node_physics_opt_weights "https://github.com/artemiymitsel/claude_hack3/blob/main/physics/phase_d_optimize_weights.py"
click node_physics_predict "https://github.com/artemiymitsel/claude_hack3/blob/main/physics/predict.py"
click node_physics_results "https://github.com/artemiymitsel/claude_hack3/blob/main/physics/tune_results.json"
click node_physics_phase_c_results "https://github.com/artemiymitsel/claude_hack3/blob/main/physics/phase_c_results.json"
click node_physics_phase_d_results_narrow "https://github.com/artemiymitsel/claude_hack3/blob/main/physics/phase_d_results_optuna_narrow_top1.json"
click node_physics_phase_d_results_wide "https://github.com/artemiymitsel/claude_hack3/blob/main/physics/phase_d_results_optuna_wide_top1.json"
click node_physics_phase_d_results_compromise "https://github.com/artemiymitsel/claude_hack3/blob/main/physics/phase_d_results_physical_compromise.json"
click node_physics_predictions "https://github.com/artemiymitsel/claude_hack3/blob/main/physics/predictions.csv"
click node_catboost_deps "https://github.com/artemiymitsel/claude_hack3/blob/main/catboost_pure/requirements.txt"
click node_catboost_tune "https://github.com/artemiymitsel/claude_hack3/blob/main/catboost_pure/tune.py"
click node_catboost_params "https://github.com/artemiymitsel/claude_hack3/blob/main/catboost_pure/best_params.json"
click node_catboost_train "https://github.com/artemiymitsel/claude_hack3/blob/main/catboost_pure/train_predict.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_dataset_train,node_dataset_valid toneBlue
class node_physics_deps,node_physics_tune,node_physics_validate,node_physics_opt_weights,node_physics_predict,node_physics_results,node_physics_phase_c_results,node_physics_phase_d_results_narrow,node_physics_phase_d_results_wide,node_physics_phase_d_results_compromise,node_physics_predictions toneAmber
class node_catboost_deps,node_catboost_tune,node_catboost_params,node_catboost_train,node_catboost_predictions toneMint
```

## Components

### Shared data

Root dataset used by both independent pipelines.

- [Train data](https://github.com/artemiymitsel/claude_hack3/blob/main/dataset/train_dataset.csv): csv time series · Historical hourly observations with target values. · `dataset/train_dataset.csv`
- [Valid features](https://github.com/artemiymitsel/claude_hack3/blob/main/dataset/valid_features.csv): csv horizon · Forecast horizon features with hidden target. · `dataset/valid_features.csv`

### Physics pipeline

Hybrid physical baseline plus learned residual ensemble.

- [Physics deps](https://github.com/artemiymitsel/claude_hack3/blob/main/physics/requirements.txt): requirements · Scientific and optimization stack for calibration and blending. · `physics/requirements.txt`
- [Tune physics](https://github.com/artemiymitsel/claude_hack3/blob/main/physics/tune_physics.py): optuna search · Searches physical constants for the turbine baseline. · `physics/tune_physics.py`
- [Phase C validate](https://github.com/artemiymitsel/claude_hack3/blob/main/physics/phase_c_validate.py): backtest validator · Checks tuned candidates end to end in the ensemble. · `physics/phase_c_validate.py`
- [Opt weights](https://github.com/artemiymitsel/claude_hack3/blob/main/physics/phase_d_optimize_weights.py): slsqp blend fit · Re-optimizes ensemble weights after validation. · `physics/phase_d_optimize_weights.py`
- [Predict](https://github.com/artemiymitsel/claude_hack3/blob/main/physics/predict.py): hybrid forecast · Main production path: physics baseline, residual models, blending, and output. · `physics/predict.py`
- [Tuning results](https://github.com/artemiymitsel/claude_hack3/blob/main/physics/tune_results.json): json artifacts · Saved tuning outcomes and selected configurations. · `physics/tune_results.json`
- [Phase C results](https://github.com/artemiymitsel/claude_hack3/blob/main/physics/phase_c_results.json): json artifacts · End-to-end validation outputs for candidate selection. · `physics/phase_c_results.json`
- [Phase D narrow](https://github.com/artemiymitsel/claude_hack3/blob/main/physics/phase_d_results_optuna_narrow_top1.json): json artifacts · Optimized blend results from the narrow Optuna run. · `physics/phase_d_results_optuna_narrow_top1.json`
- [Phase D wide](https://github.com/artemiymitsel/claude_hack3/blob/main/physics/phase_d_results_optuna_wide_top1.json): json artifacts · Optimized blend results from the wide Optuna run. · `physics/phase_d_results_optuna_wide_top1.json`
- [Phase D compromise](https://github.com/artemiymitsel/claude_hack3/blob/main/physics/phase_d_results_physical_compromise.json): json artifacts · Physical compromise configuration and fit summary. · `physics/phase_d_results_physical_compromise.json`
- [Predictions](https://github.com/artemiymitsel/claude_hack3/blob/main/physics/predictions.csv): csv submission · Final physics-solution submission file. · `physics/predictions.csv`

### CatBoost pipeline

Direct supervised forecasting pipeline with engineered features.

- [CatBoost deps](https://github.com/artemiymitsel/claude_hack3/blob/main/catboost_pure/requirements.txt): requirements · Dependency set for the direct supervised pipeline. · `catboost_pure/requirements.txt`
- [Tune CatBoost](https://github.com/artemiymitsel/claude_hack3/blob/main/catboost_pure/tune.py): optuna search · Finds CatBoost hyperparameters and persists them. · `catboost_pure/tune.py`
- [Best params](https://github.com/artemiymitsel/claude_hack3/blob/main/catboost_pure/best_params.json): json config · Stored tuned CatBoost hyperparameters used by training. · `catboost_pure/best_params.json`
- [Train predict](https://github.com/artemiymitsel/claude_hack3/blob/main/catboost_pure/train_predict.py): timeseries cv · Builds features, runs rolling validation, fits final model, and writes submission. · `catboost_pure/train_predict.py`
- **Predictions**: csv submission · Final CatBoost-solution submission file.

## Connections

- Train data → Tune physics: train data · Historical data used for physical calibration.
- Valid features → Predict: forecast horizon · Features to score for the final submission.
- Physics deps → Tune physics: supports · Provides optimization and scientific libraries.
- Tune physics → Tuning results: writes · Persists tuned physical constants and search outcomes.
- Tune physics → Phase C validate: configures · Feeds candidate physical settings into validation.
- Phase C validate → Phase C results: writes · Stores end-to-end validation outputs.
- Phase C results → Opt weights: candidates · Validation outputs inform later weight optimization.
- Opt weights → Phase D narrow: writes · Persists optimized blend weights for the narrow search.
- Opt weights → Phase D wide: writes · Persists optimized blend weights for the wide search.
- Opt weights → Phase D compromise: writes · Persists compromise configuration and fit summary.
- Tuning results → Predict: tuned params · Selected calibration results guide final forecasting.
- Phase D narrow → Predict: blend weights · Optimized ensemble weights are applied in production.
- Phase D wide → Predict: blend weights · Alternate optimized weights can inform production.
- Phase D compromise → Predict: blend weights · Compromise fit can inform production.
- Predict → Predictions: writes · Produces the final physics submission CSV.
- Train data → Tune CatBoost: train data · Historical data used for hyperparameter search.
- Train data → Train predict: train data · Historical data used for feature engineering and validation.
- Valid features → Train predict: forecast horizon · Features to score for the final submission.
- CatBoost deps → Tune CatBoost: supports · Provides the CatBoost runtime and tuning stack.
- Tune CatBoost → Best params: writes · Persists the best hyperparameter configuration.
- Best params → Train predict: configures · Training loads tuned parameters from the saved JSON.
- Train predict → Predictions: writes · Produces the final CatBoost submission CSV.