critter 0.70

By Swaminathan N Date 2010-05-25 04:41

https://sites.google.com/site/strategictestsuite/test-results

#2 of 63 engines tested! 8-)

Critter 0.6 had 741/100.

Conditions:

10 seconds per position,
1000 positions,
Q6600, 2GB RAM, 2.4GHZ,
Engine used 1 CPU.

By Peter Martan Date 2010-05-25 06:09

[quote="Swaminathan N"]
#2 of 63 engines tested! 8-)
[/quote]

Thanks for the news!
So only Stockfish 1.7.1 scored better, right?
Two more questions: you didn't test Rybka 3, did you? Guess you'ld rather wait for R4?
Still I'm not quite sure about the classification, from A to D? S is even better than A+?
Thanks in advance and thanks for the great work again.
Regards
Peter.

By Swaminathan N Date 2010-05-25 06:36

Hi Peter, I don't have Rybka 3 (only Dann has) but I could ofcourse test Rybka 2.2n2.. these tests are currently in progress...I'll add more test results into the site as I go along. Now testing Komodo 1.2JA.

S is the topmost grade which is equivalent to "superior" or "outstanding".
This is the grading system in place, which is implemented into STS Stat software:

85+ S
80+ A+
75+ A
70+ A- 
65+ B+
60+ B
55+ C+
50+ C
45+ D
40+ E
<40 F

Best Wishes,
Swami

By Peter Martan Date 2010-05-25 07:06

One more big thank you and congratulations.
Now we'ld just need another one test series dealing with tactical postions as well organized and itemized as yours for strategic themes and we could evaluate engines' abilities overall much better than by "ELO" only. And we wouldn't have to calibrate "ELO" of engines again and again, which will get harder and harder with more and more engines of approximately the same high "ELO" level.
The problem of these counting to a declining curve would be solved.

At least it is a very valuable measurement in addition.

By Swaminathan N Date 2010-05-25 07:28 Edited 2010-05-25 07:33

Hi Peter, you're welcome! yes, you made a lot of sense with that. Rating list based on games appears to be little inconclusive as they are all closely few ELO apart.

I think someone has to collect all the tactical positions by searching through "Test position" posts on forums. (CCC,CSS,Rybkaforum) and we can have final list of all tough "tactical" set. I believe there's enough positions in the archives for the past 8 years of the forum's existence. Tactics (that's challenging for top engine) is usually very hard to find.

As for strategy, the suite is still under development, I believe there is need for 20 STS chapters, which could give balanced assessment and report of true positional understanding of engines under test. STS 11 is about King Activity and we're collecting more positions and testing them.

By Peter Martan Date 2010-05-25 07:57

[quote="Swaminathan N"]
I think someone has to collect all the tactical positions by searching through "Test position" posts on forums. (CCC,CSS,Rybkaforum) and we can have final list of all tough "tactical" set. I believe there's enough positions in the archives for the past 8 years of the forum's existence. Tactics (that's challenging for top engine) is usually very hard to find.

As for strategy, the suite is still under development, I believe there is need for 20 STS chapters, which could give balanced assessment and report of true positional understanding of engines under test. STS 11 is about King Activity and we're collecting more positions and testing them.
[/quote]

Hi Swaminathan!
All right, if tactical test postitions were nearly as many and as well itemized as your strategical ones, engine authors might well tune up there programs to those too.
It was always critized this would lead up to spezialized engines, nevermind spezial tuning for about as many positions as within your suite.