Hier die Bekanntgabe auf Discord:

The first (or rather zeroth) release candidate in the v0.29 series is finally out. You can find it in <https://github.com/LeelaChessZero/lc0/releases/tag/v0.29.0-rc0>. In this release we have:
* Initial support for attention policy, only cuda backend and partially in blas/dnnl/eigen (good enough for T79).
* Non multigather (legacy) search code and `--multigather` option are removed.
* 15b default net is now 753723.
* The onnx backend now allows selecting gpu to use.
* Improved error messages for unsupported network files.
* Some assorted fixes.

Dazu noch das neueste ap-mish net:

https://training.lczero.org/get_network?sha=546f078df4d25b9aada7faa7880ed6b2a621e789fe28290f2505917be3e2ecd3

Und das bisher beste T78 Netz:

https://training.lczero.org/get_network?sha=a83975cd70bb6e5ed3832aefb38ab70bcb4511bd6dca8e9857e6cc985d9ed1e2

Lothar

Hier die bisherigen Parameterbereiche für ap-mish-Netze:

https://media.discordapp.net/attachments/539960268982059008/958763909270110309/landscape-20220330-175352-98.png

https://media.discordapp.net/attachments/539960268982059008/959860372989415454/optima-20220402-184530-176.png

Lothar

Die Generierung von Netz T80 hat heute begonnen:

**T80** Started: Copied train/match parameters from T79.  Initial net is from 2.016M steps point of T79.9 trained on second LR using T79 data. Contributions require 0.29-rc0 (dev will not be accepted) (cuda or ~~blas~~). Shadow data is T79 and will be removed as the training window fills.
```
%YAML 1.2
---
name: '512x15-t80'
gpu: 1

dataset:
  num_chunks: 3000000
  allow_less_chunks: true
  train_ratio: 0.90
  input_train:
    - '/data/d/data/dev1/split/train/' # supports glob
    - '/data/d/data/shadow1/data-rescored/' # supports glob
    - '/data/d/data/shadow1/data-rescored2/' # supports glob
    - '/data/d/data/shadow1/data-rescored3/' # supports glob
  input_test:
    - '/data/d/data/dev1/split/test/' # supports glob
    - '/data/d/data/shadow1/data-test/'  # supports glob
    - '/data/d/data/shadow1/data-test2/'  # supports glob
    - '/data/d/data/shadow1/data-test3/'  # supports glob
  input_validation: '/data/d/data/dev1/validate/'
  train_workers: 32
  test_workers: 8

training:
    swa: true
    swa_output: true
    swa_steps: 100
    swa_max_n: 10
    mask_legal_moves: true
    lookahead_optimizer: true
    renorm: true
    renorm_max_r: 1.0
    renorm_max_d: 0.0
    diff_focus_min: 0.025
    diff_focus_slope: 3.0
    q_ratio: 0.0
    max_grad_norm: 2.5
    batch_size: 1024
    num_batch_splits: 2
    test_steps: 500
    validation_steps: 500
    num_test_positions: 40000
    train_avg_report_steps: 200  
    total_steps: 2000 
    warmup_steps: 1000
    checkpoint_steps: 4000
    shuffle_size: 500000  
    lr_values: 
        - 0.04
        - 0.04
    lr_boundaries:
        - 100
    policy_loss_weight: 1.0
    value_loss_weight:  1.0 
    moves_left_loss_weight: 1.0  
    reg_term_weight: 0.5
    path: '/data/d/data/dev1/networks'   

model:
  filters: 512
  residual_blocks: 15
  se_ratio: 16
  default_activation: 'mish'
  policy: 'attention'
  pol_encoder_layers: 0
...
```

Lothar