ich frag mich, wieviel schneller dieser Trainingsprozess in Zukunkt, in ein paar Jahren ablaufen wird.
Das ist noch ziemlich neu und sieht nicht sehr effektiv aus
Da halte ich einen Verbesserungsfaktor von 10-100 fuer moeglich.
1005 Euro , gaming PC , Ryzen 1300 , GTX1080
945 Euro , gaming PC , Ryzen 1300 , GTX1070
669 Euro , gaming PG , Ryzen 1300 , GTX1060
505 Euro , gaming PC , Ryzen 1300 , GTX1050
Elo-sf=(3*Elo=se-2558.5) /5
gooogle's 2nd generation TPU gets 180 teraflops
up to 11.5 petaflops of peak performance.
This TPU2 board has four of the TPU2 units, each board capable of a maximum peak throughput of 45 teraflops with the system board having an aggregate of 180 teraflops as we have said above. (We presume that this is using 16-bit half-precision floating point.)
https://news.ycombinator.com/item?id=16358557Market rate is close to $1 per TB outbound. Your rate is $80-$120 per TB.
training ImageNet on a p3.16xlarge cost $358, when this post claims it'll cost
less than $200. (EDIT: never mind; the benchmark uses ImageNet-152, and
Google compares TPU performance against ImageNet-50)
--------------------------------
jemand postete diese Formel : Elo-sf=(3*Elo=se-2558.5) /5
um die LC0 self-play Elos in richtige Elos umzurechnen.
Aber Elos sind linear, warum kommt jetzt der Faktor von 0.6 ??
--------------------------------------
google spielte 44M Trainingspartien auf 5000 1st-generation-TPUs in 9 Stunden
, a TPU costs a little more than 2x as much as a Volta on AWS P3,
and delivers a little less than 2x the performance (180 TOPs for the TPU, 100 for Volta)
Nvidias Tesla-V100 $8000 at ebay
A0 : $ 80M hardware for 9h , 44M games --> 1.5 games per day per $
LC0 : $300K for 150d 400 times more time , 333 times less worth hardware
https://github.com/gcp/leela-zero/issues/3051070 , 600 games per day
$90 , GTX660=350 games/day --> 3.9 games per day per $
I run two threads on a GTX 970 (about the same as a 1060) that have a combined speed of 1000-1200 nps
The Google Colab performance is about 1100-1300 nps.
On my laptop, the GPU client runs about 550-750 nps on a GTX 965M.