Similarity Move Test

By Sedat Canbaz Date 2012-06-11 22:00 Edited 2012-06-11 22:02

Hello Dear Friends,

Nowadays there are a lot of clone/derivative/original work discussions over chess forums
That's why,this time i decided to create a different list,where all engines are played exactly at same position

Similarity Move Test Conditions:
----------------------------------
Processor:QX 9650@3.66GHz
Win XP x64 Prof (service pack 2)
Chess GUIs:Fritz 12 / Arena 3.0
HashTable size:128 MB
Time Control:3 min game
Engine CPUs:All engines are played with 1 core
Before starting each test: engine's hastables are cleaned
Test Opening Position: B12 Caro-Kann,Advance Variation (Side to move:Blacks)
r2qkb1r/pp1nnpp1/2p1p2p/3pPb2/3P4/1N3N2/PPP1BPPP/R1BQ1RK1 b kq - 0 8

Some Notes:
---------------
-The current test is done mainly for recognizing which engines are each other relative
-The Similarity Test can be useful about which engines use same/similar ideas as well
-Similarity Move Test is not done for proving which engines are clone/derivative/original work
-Almost all Top Engines (Houdini;Komodo;Strelka;Stockfish,Critter,Ivanhoe... prefered 'Qc7' move
-Its interesting to note that Rybka 4.1 x64 1c prefered to play 'g5' move (not sure why ?!)
-Fruit 2.1 (open source engine) prefered especially to play 'Ng6'
-Note also 'Ng6' move is the most prefered move by many chess engines
-Zappa 1.0 and Fruit 2.0 are prefered to play 'a5' move (again similarity !)
-Sjeng c't 2010;Spike 1.4;BigLion 1.8f;Junior 6;Sos 5.1 played completely different moves than all

***************************************

59 Engine versions prefered 'Ng6' move

Amateur 2.82 - Ng6
Abrok 5.0 - Ng6
Ant 2006 - Ng6
Arasan 13.4 1c - Ng6
BeoWolf 2.4 - Ng6
Betsy 6.51 - Ng6
Bison 10.1 x64 - Ng6
Brutus 8.05 x64 - Ng6
CapaChess - Ng6
Chezzz 1.03 - Ng6
Comet B68 - Ng6
Crafty 23 CB 1c - Ng6
Cyrano 0.6b17 x64 - Ng6
Danasah 4.45 UCI - Ng6
Danasah 4.45 WB - Ng6
DayDreamer 1.75 x64 - Ng6
Deep Ftritz 13 1c - Ng6
Deep Sjeng 1.6 1c - Ng6
Delphil 26a - Ng6
Dirty 25082011 x64 - Ng6
Dragon 4.43 - Ng6
DrunkenMaster 1.2 - Ng6
EXchess 6.10b x64 - Ng6
Franchesca 0.19 - Ng6
Fruit 2.1 - Ng6
GES 1.36 - Ng6
Gull 1.2 x64 - Ng6
GreKo 9.2 - Ng6
Gromit 3.1 CB - Ng6
GnuChess4TM - Ng6
GnuChess 6.0.2 - Ng6
Glaurung 1.0.1 - Ng6
Jonny 2.7 - Ng6
Junior 7 - Ng6
Junior 8 - Ng6
Hiarcs 7.32 - Ng6
Hiarcs 9 - Ng6
Hiarcs 13.2 - Ng6
Ktulu 5.1 - Ng6
Ktulu 9 - Ng6
LambChop 10.99 - Ng6
LittleThought 052 - Ng6
Loop 2007 x64 - Ng6
Movei 0.08.438 1c - Ng6
Naum 4.2 x64 1c - Ng6
Onno 1.2.70 x64 1c - Ng6
Phalanx XXII - Ng6
Pharaon 3.5.1 1c - Ng6
Popochin 3.2 - Ng6
Shredder 9 - Ng6
Spike 0.9a - Ng6
Supra 101 - Ng6
Toga II 1.2.1 - Ng6
Toga II 1.4 beta5c - Ng6
Tornado 4.88 x64 1c - Ng6
Ufirm 8.02 - Ng6
Umko 1.2 x64 1c - Ng6
Zappa Mexico II x64 1c - Ng6
Zarkov 6.50 - Ng6

***************************************

57 Engine versions prefered 'Qc7' move

Amy 0.87 - Qc7
Anmon 5.75 WB - Qc7
Aristarch 4.50 - Qc7
AliChess 4.20 - Qc7
BobCat 3.25 x64 - Qc7
Buzz 008 x64 - Qc7
Boot 5.1.0 - Qc7
Cerebro 3.0d - Qc7
Chess Tiger 14 - Qc7
Chess Tiger 15 - Qc7
Chess Tiger 2007.1 - Qc7
Critter 0.8 x64 1c - Qc7
Critter 1.4 x64 1c - Qc7
Chronos 1.99 x64 - Qc7
Deep Junior 13 1c - Qc7
Deuterium 110229107 - Qc7
ETchess 130108 - Qc7
El Cinito 3.25 - Qc7
Elf 1.30 - Qc7
EXchess 4.03 - Qc7
Frenzee 3.5.19 - Qc7
FreeChess 2.17b - Qc7
Fritz in Bahrein - Qc7
Gandalf 5.0 - Qc7
Fire 2.2 xTreme x64 1c - Qc7
Fairy-Max v4.8 - Qc7
Fritz 5.32 - Qc7
Fritz 6 - Qc7
Fritz 7 - Qc7
Fruit 1.0 - Qc7
Fruit 090705 x64 1c - Qc7
Gambit Tiger 2.0- Qc7
GreKo 2.817 - Qc7
Ivanhoe B46fE 1c - Qc7
Hannibal 1.1 x64 - Qc7
Houdini 2.0c - Qc7
Igorrit 0086v9 x64 - Qc7
Insomniac 0.69 - Qc7
Komodo 4 x64 - Qc7
MinkoChess 1.3 x64 1c - Qc7
Nejmet 3.06 - Qc7
Nemo 1.01b x64 - Qc7
Philou 3.70 x64 - Qc7
Pseudo 0.7 - Qc7
Quazar 0.4 x64 - Qc7
Rybka 1.0b x64 1c - Qc7
RobboLito 010 - Qc7
Rufian 2.0.2 - Qc7
Rufian 2.1.0 - Qc7
SOS CB - Qc7
Strelka 5.1 x64 - Qc7
Strelka 5.5 x64 - Qc7
Stockfish 2.2.2 x64 1c - Qc7
TSCP 1.81 - Qc7
Thinker 5.4Di x64 1c - Qc7
Xpdnt 061120 - Qc7
Yace 0.99.87 - Qc7
Vitrivius 1.0C - Qc7

***************************************

25 Engine versions prefered 'Nc8' move

Anatoli 035k - Nc8
Amyan 1.59 - Nc8
Amyan 1.72 - Nc8
Arion 1.7 - Nc8
AliBaba 1.18 - Nc8
Bionic 4.01 - Nc8
Cheng3 1.07 - Nc8
Counter 1.2 - Nc8
Colossus 2008b - Nc8
Crafty 19.19 1c - Nc8
Delfi 5.4 1c - Nc8
Faile 1.44 CB - Nc8
Hagrid 0.7.56 - Nc8
Hamsters 0.8.2 1c - Nc8
Icarus 0.18 CB - Nc8
Leila 052f - Nc8
List 512 - Nc8
Mizar 1.0 - Nc8
Nimzo 8 - Nc8
Pepito 1.59 - Nc8
Predator 2.21 x64 - Nc8
Rotor 0.5 - Nc8
Trace 1.37a - Nc8
WildCat 4 WB - Nc8
WildCat 8 UCI - Nc8

***************************************

24 Engine versions prefered 'a5' move

Ayito 02994 - a5
Bugchess2 1.9 x64 - a5
Cheese 1.3 - a5
Chiron 1.0a x64 1c - a5
Gaviota 0.84 x64 - a5
Deep Shredder 12 x64 1c - a5
Equinox 1.10 x64 1c - a5
Fritz 8 - a5
Fruit 2.0 - a5
Goliat Light 1.5 - a5
Kiwi 0.6d x64 - a5
King of Kings 2.57 - a5
Matacz14 HT74 - a5
Quark 2.35 - a5
Pawny 0.2 x64 - a5
Patzer 3.11a - a5
Protector 1.4.0 x64 1c - a5
Prodeo 1.0 - a5
ProDeo 1.74 - a5
SlowChess Blitz 2.1 1c - a5
SmarThink 017a - a5
SmarThink 1.20 x64 - a5
Spark 1.0 x64 1c - a5
Zappa 1.0 - a5

***************************************

21 Engine versions prefered 'g5' move

Anaconda 2.2.2 - g5
Crafty 23.3 x64 1c - g5
Crafty 18.10 CB - g5
ElTurco 0.93 - g5
Eeyore 1.48 - g5
Jonny 4 x64 - g5
Gandalf 6.0 - g5
Natwarlal 0.14 - g5
Naum 1.8 - g5
Patriot 1.3.0 - g5
RedQueen 1.0.0 x64 - g5
Rybka v2.1c - g5
Rybka 4.1 x64 1c - g5
Sjeng WC2008 x64 1c - g5
Scorpio 2.7 x64 1c - g5
Testina 2.2 - g5
Texel 101 x64 - g5
The Baron 2.2.2 - g5
The Baron 2.2.3 - g5
The King 3.33 - g5
The King 3.50 x64 1c - g5
Thinker 4.7a - g5

***************************************
3 Engine versions prefered 'Rc8' move

GarboChess 3 x64 - Rc8
Litle Goliath Revival 3.11 - Rc8
RomiChess P3k x64 - Rc8

***************************************

3 Engine versions prefered 'Bh7' move

Tao 5.7 - Bh7
The Crazy Bishop CB - Bh7
The Crazy Bishop 052 - Bh7

***************************************

2 Engine versions prefered 'Bg4' move

Gaia 3.5 x64 - Bg4
LordKing 2006 Serafinus - Bg4

***************************************

1 Engine versions prefered 'b6' move

Sjeng c't 2010 - b6

***************************************
1 Engine version prefered 'Bg6' move

Spike 1.4 1c - Bg6

***************************************

1 Engine version prefered 'Be4' move

Sos 5.1 For Arena - Be4
***************************************

1 Engine version prefered 'f6' move

BigLion 1.8f - f6
***************************************

1 Engine version prefered 'Rg8' move

Junior 6 - Rg8

***************************************

Best Regards,
Sedat Canbaz

By Ernest Bonnem Date 2012-06-12 23:44

[quote="Sedat Canbaz"]-Almost all Top Engines (Houdini;Komodo;Strelka;Stockfish,Critter,Ivanhoe... prefered 'Qc7' move[/quote]
Hi Sedat,

It seems that in this position, several moves are very close.
On infinite analysis, here is what I get for Houdini (1st and only test):

r2qkb1r/pp1nnpp1/2p1p2p/3pPb2/3P4/1N3N2/PPP1BPPP/R1BQ1RK1 b kq - 0 1
Analysis by Houdini 2.0c w32 Nlm:
..................
1...Ng6 2.Bd3 Bxd3 3.Qxd3 Be7 4.Bd2 0-0 5.Rfe1 Qc7 6.Rac1 Nb6 7.Na5 Na4 8.Rb1 Rad8 9.g3
= (0.02)   Depth: 13/29   00:00:00 663kN
1...Ng6 2.Bd3 Bxd3 3.Qxd3 Be7 4.Bd2 0-0 5.Rac1 Qc7 6.g3 Nb6 7.Na5 Na4 8.Rb1 Rad8 9.Nb3
= (0.04)   Depth: 14/29   00:00:00 811kN
1...Ng6 2.Bd3
= (0.10 --)   Depth: 15/33   00:00:01 1319kN
1...Ng6 2.Bd3
= (0.24 --)   Depth: 15/33   00:00:01 2419kN
1...Qc7 2.h3 c5 3.Nxc5 Nxc5 4.dxc5 Nc6 5.Be3 Be4 6.Nd4 Qxe5 7.f4 Qb8 8.Nxc6 bxc6 9.Bd4 Qb4 10.a3 Qb7
= (0.19)   Depth: 15/36   00:00:03 7411kN
1...Qc7 2.a4 Be4 3.Be3 c5 4.Nxc5 Nxc5 5.dxc5 Nc6 6.Bd3 Be7 7.Bxe4 dxe4 8.Nd2 Qxe5 9.c3 Rd8 10.Qg4 h5 11.Qxe4 Bxc5 12.Rad1 Qxe4 13.Nxe4 Rxd1 14.Rxd1 Bxe3 15.fxe3
= (0.17)   Depth: 16/44   00:00:09 20597kN
1...Qc7 2.a4 Be4 3.Be3 a6 4.Nfd2 Bh7 5.c4 dxc4 6.Nxc4 Nd5 7.Bd3 Bxd3 8.Qxd3 Be7 9.Bd2 0-0 10.Qg3 Kh8 11.Nd6 b6 12.Rac1 Bxd6 13.exd6
= (0.18)   Depth: 17/44   00:00:11 25493kN
1...Qc7 2.a4 Be4 3.Be3 a6 4.Nfd2 Bh7 5.c4 dxc4 6.Nxc4 Nd5 7.Bd3 Bxd3 8.Qxd3 Be7 9.Bd2 0-0 10.Qg3 Kh8 11.Nd6 b6 12.Rac1 Bxd6 13.exd6
= (0.18)   Depth: 18/44   00:00:18 41387kN
1...Qc7 2.h3 a5 3.a4 b6 4.Bf4 c5 5.c4 Rd8 6.Rc1 Be4 7.Nbd2 Nc6 8.cxd5 Bxd5 9.Nc4 Bxf3 10.Bxf3 Nxd4 11.Re1 Nxf3+ 12.Qxf3
= (0.18)   Depth: 19/50   00:00:50 120mN
1...Qc7 2.h3 a5 3.a4 Be4 4.Be3 b6 5.Nbd2 Bh7 6.c4 Nf5 7.Bf4 Be7 8.cxd5 exd5 9.g4 Nh4 10.Nxh4 Bxh4 11.Nf3 Be7 12.Bd3 Be4 13.Rc1
= (0.18)   Depth: 20/50   00:01:18 186mN
1...Bg6
= (0.11 !)   Depth: 20/53   00:01:42 242mN
1...Bg6 2.Nh4 Bh7 3.Be3 g5 4.Nf3 Qc7 5.a4 0-0-0 6.Bd2 Kb8 7.h3 Rg8 8.a5 Be4 9.a6 b6 10.Bd3 Bxd3 11.cxd3 Nf5 12.Qc2 g4 13.hxg4 Rxg4
= (0.12)   Depth: 20/53   00:01:43 246mN
1...Bg6 2.Nh4 Bh7 3.Be3 g5 4.Nf3 Qc7 5.a4 0-0-0 6.Bd2 Kb8 7.h3 Rg8 8.a5 Be4 9.a6 b6 10.Bd3 Bxd3 11.cxd3 Nf5 12.Qc2 g4 13.hxg4 Rxg4
= (0.12)   Depth: 21/53   00:01:52 266mN
1...Bg6 2.Nh4 Bh7 3.Be3 g5 4.Nf3 Qc7 5.Bd2 a5 6.a4 Bg7 7.h4 gxh4 8.Nxh4 c5 9.Nxc5 Nxc5 10.dxc5 Nc6 11.Nf3 Nxe5 12.Nxe5 Bxe5 13.Bb5+ Kf8 14.Rb1 Qxc5 15.Qh5 Bg7 16.Bd3 Bxd3 17.cxd3
= (0.11)   Depth: 22/53   00:02:22 341mN
1...Bg6 2.Nh4 Bh7 3.Bd3 Bxd3 4.Qxd3 Qc7 5.Nf3 Rc8 6.Bd2 c5 7.Nxc5 Nxc5 8.dxc5 Qxc5 9.Qb3 b6 10.c3 Nc6 11.Be3 Qa5 12.Rad1 Bc5 13.c4 Ne7 14.Bxc5 Qxc5 15.Qa4+ Qc6 16.Qxc6+ Rxc6
= (0.11)   Depth: 23/53   00:03:26 498mN
1...Bg6 2.Nh4 Bh7 3.Bd3 Bxd3 4.Qxd3 Qc7 5.Bd2 c5 6.Nxc5 Nxc5 7.dxc5 Qxc5 8.Nf3 Rc8 9.Qb3 b6 10.c3 Nc6 11.Be3 Qa5 12.Rad1 Bc5 13.c4 Ne7 14.Bxc5 Qxc5 15.Qa4+ Qc6 16.Qxc6+ Rxc6
= (0.11)   Depth: 24/53   00:05:32 814mN
1...Bg6 2.Nh4 Bh7 3.Bd3 Bxd3 4.Qxd3 Qc7 5.f4 g6 6.a4 a5 7.Be3 Bg7 8.Bf2 0-0 9.Nf3 h5 10.Qd2 Nf5 11.Nh4 Ne7 12.Qd3 Rfe8 13.Qe3 Rf8 14.Nd2 Qb6 15.Rab1 Qb4
= (0.10)   Depth: 25/55   00:11:30 1720mN

By Ernest Bonnem Date 2012-06-13 01:30

...and a second test, with multivariation:

(at depth 26)

r2qkb1r/pp1nnpp1/2p1p2p/3pPb2/3P4/1N3N2/PPP1BPPP/R1BQ1RK1 b kq - 0 1
Analysis by Houdini 2.0c w32 Nlm:

1. = (0.13): 1...a5 2.a4 Bg6 3.Nh4 Bh7 4.Be3 g5 5.Nf3 Bg7 6.c4 0-0 7.Bd3 Bxd3 8.Qxd3 dxc4 9.Qxc4 Nf5 10.Rfc1 Nb6 11.Qe2 Nd5 12.g4 Nfxe3 13.fxe3 h5 14.gxh5 g4 15.Nfd2 Qg5 16.Nc4 Qxh5 17.Nbxa5 Ra7 18.Rf1 Rd8
2. = (0.13): 1...Bg6 2.Nh4 Bh7 3.Be3 g5 4.Nf3 Qc7 5.a4 0-0-0 6.Bd2 Nf5 7.Ba5 b6 8.Bc3 a5 9.h3 Kb8 10.Bd3 Be7 11.Qe2 Rdg8 12.Nh2 Re8 13.Ng4 h5 14.Ne3 g4 15.hxg4 Nxe3 16.fxe3 Bxd3 17.cxd3 hxg4 18.Rxf7
3. = (0.16): 1...Qc7 2.Bd2 Bg6 3.a4 c5 4.Nxc5 Nxc5 5.dxc5 Nc6 6.Be3 Be7 7.c3 0-0 8.Bd4 Nxd4 9.cxd4 b6 10.b4 a5 11.cxb6 Qxb6 12.b5 Rac8 13.Rc1 Be4 14.Qd2 Ba3 15.Rxc8 Rxc8 16.Qf4 Bb2 17.Rd1 Rc2 18.Bf1 Bf5 19.h3

By Sedat Canbaz Date 2012-06-13 11:38

Thanks for the testings dear Ernest,

Just i'd like to mention that any engine's move choice is highly depending on played conditions:
-Hardware Speed
-Engine's Cores
-Time Control
-Opening Position
-Hashtable Size
-Endgames
...

Note also that many engines, which are based on free open source (Ippo, Stockfish, Fruit...) play same/similar moves as well

Btw,just my 2 cents more over this issue

Nowadays we are talking about engine move similarities,yes...there are tons of clone discussions,but still we have different views
In my opinion,the problems are appearing due to many engines use similar stuff/ideas or include same copied code as well
On the other hand,i believe the free open source engines are the main reason about why we see a lot clone discussions and also why the engines are so relative

Actually,i think its now too late to find which engines are 100% original, because there is almost no a such original work

One thing more,I have the same view as Alan's statement:
An open source version of any engine can be created by anyone with a good decompiler. Open source does not confer legitimacy.

Let's imagine about opening book moves/lines similarities...
Can we find a X book,which is completely different than all other books (i mean based on 100% own ideas) ?

And last,
Its time for revolution,we need to give a chance for the 'Young Talents',otherwise there will be a big mistake/injustice over them
Note:I mean for those 'Young Talents', who has capacity to create at least 50 Elo improvement over free open source engines

Kind Regards,
Sedat

By Roland del Rio Date 2012-06-15 01:41

Hi Sedat.

Very interesting test. Thanks a lot for your work! For me, stuff like this is much more meaninful than tons of 40/4 engine matches.
I did some test like yours on my own, but never with that huge amount of different engines. I not 100% sure if I like the position
in your test, because the favorite moves are likely leading to the same positions in some moves with slightly different move orders.
This might be the reasons why e.g. Houdini is very close in MV-evaluations. I always try to use positions where the next move
decides on different plans and move transpositions aren't easily possible.

Greetings

Roland

By Sedat Canbaz Date 2012-06-15 16:55 Edited 2012-06-15 16:58

[quote="Roland del Rio"]
Hi Sedat.

Very interesting test. Thanks a lot for your work! For me, stuff like this is much more meaninful than tons of 40/4 engine matches.
I did some test like yours on my own, but never with that huge amount of different engines. I not 100% sure if I like the position
in your test, because the favorite moves are likely leading to the same positions in some moves with slightly different move orders.
This might be the reasons why e.g. Houdini is very close in MV-evaluations. I always try to use positions where the next move
decides on different plans and move transpositions aren't easily possible.

Greetings

Roland
[/quote]

Hello dear Roland,

Thanks a lot for your kind words and for your interest

Yes...the current similarity list is just a idea,a new concept

Of course for more reliable list, we need more positions and more engines
Once more i'd like to mention that i tested all engines manually (probably more than 20 hours i spent for the current position)

And its will be great,if there will be a new automatic similarity GUI, where we can test and watch directly the played moves

One thing more, actually there is a easier way and its possible to include many positions by using Chess GUIs

For this, we need to create a engine tournament with hundreds of starting positions
And for not wasting many CPU time, we need to adjudicate manually the played moves

Greetings,
Sedat