Author: Linmiao Xu
Date: Sat Dec 30 11:08:03 2023 +0100
Timestamp: 1703930883

Update default net to nn-b1e55edbea57.nnue

Created by retraining the master big net `nn-0000000000a0.nnue` on the same
dataset with the ranger21 optimizer and more WDL skipping at training time.

More WDL skipping is meant to increase lambda accuracy and train on fewer
misevaluated positions where position scores are unlikely to correlate
with game outcomes. Inspired by:
- repeated reports in discord #events-discuss about SF misplaying due to wrong endgame
evals, possibly due to Leela's endgame weaknesses reflected in training data
- an attempt to reduce the skewed dataset piece count distribution where there
are much more positions with less than 16 pieces, since the target piece count
distribution in the trainer is symmetric around 16

The faster convergence seen with ranger21 is meant to:
- prune experiment ideas more quickly since fewer epochs are needed to reach elo maxima
- research faster potential trainings by shortening each run

```yaml
experiment-name: 2560-S7-Re-514G-ranger21-more-wdl-skip
training-dataset: /data/S6-514G.binpack
early-fen-skipping: 28

start-from-engine-test-net: True
nnue-pytorch-branch: linrock/nnue-pytorch/r21-more-wdl-skip

num-epochs: 1200
lr: 4.375e-4
gamma: 0.995
start-lambda: 1.0
end-lambda: 0.7
```

Experiment yaml configs converted to easy_train.sh commands with:
https://github.com/linrock/nnue-tools/blob/4339954/yaml_easy_train.py

Implementations based off of Sopel's NNUE training & experimentation log:
https://docs.google.com/document/d/1gTlrr02qSNKiXNZ_SuO4-RjK4MXBiFlLE6jvNqqMkAY
- Experiment 336 - ranger21 https://github.com/Sopel97/nnue-pytorch/tree/experiment_336
- Experiment 351 - more WDL skipping

The version of the ranger21 optimizer used is:
https://github.com/lessw2020/Ranger21/blob/b507df6/ranger21/ranger21.py

The dataset is the exact same as in:
https://github.com/official-stockfish/Stockfish/pull/4782

Local elo at 25k nodes per move:
nn-epoch619.nnue : 6.2 +/- 4.2

Passed STC:
https://tests.stockfishchess.org/tests/view/658a029779aa8af82b94fbe6
LLR: 2.93 (-2.94,2.94) <0.00,2.00>
Total: 46528 W: 11985 L: 11650 D: 22893 Elo +2.50
Ptnml(0-2): 154, 5489, 11688, 5734, 199

Passed LTC:
https://tests.stockfishchess.org/tests/view/658a448979aa8af82b95010f
LLR: 2.94 (-2.94,2.94) <0.50,2.50>
Total: 265326 W: 66378 L: 65574 D: 133374 Elo +1.05
Ptnml(0-2): 153, 30175, 71254, 30877, 204

This was additionally tested with the latest DualNNUE and passed SPRTs:

Passed STC vs. https://github.com/official-stockfish/Stockfish/pull/4919
https://tests.stockfishchess.org/tests/view/658bcd5c79aa8af82b951846
LLR: 2.93 (-2.94,2.94) <0.00,2.00>
Total: 296128 W: 76273 L: 75554 D: 144301 Elo +0.84
Ptnml(0-2): 1223, 35768, 73617, 35979, 1477

Passed LTC vs. https://github.com/official-stockfish/Stockfish/pull/4919
https://tests.stockfishchess.org/tests/view/658c988d79aa8af82b95240f
LLR: 2.95 (-2.94,2.94) <0.50,2.50>
Total: 75618 W: 19085 L: 18680 D: 37853 Elo +1.86
Ptnml(0-2): 45, 8420, 20497, 8779, 68

closes https://github.com/official-stockfish/Stockfish/pull/4942

Bench: 1304666
see source

Was passiert wenn man das Dual Netz verwendet und in der manuellen Daueranalyse der eigenen Partien ständig vor und zurück geht?