POLL:Man vs Machine ?

By Michael Scheidl Date 2012-02-20 13:12

Zitat:

1)More than 500 Elo
2)Less than 500 Elo

I think the answer to this extreme question is easy:

According to the official FIDE Elo tables, +501 Elo means a performance difference of 96%. That would mean 4% for the best humans, or ~1 one draw only, in twelve games. Comps are stronger but not that much stronger, compared to the top humans.

(I could be wrong...)

By Benno Hartwig Date 2012-02-20 15:39

[quote="Michael Scheidl"]According to the official FIDE Elo tables, +501 Elo means a performance difference of 96%. That would mean 4% for the best humans, or ~1 one draw only, in twelve games. Comps are stronger but not that much stronger, compared to the top humans.[/quote]And what, if we really find, that some engines reach 250 ELO against other engines, which really reach 250 ELO against the strongest humans?
The ELO-system might get looked at with several views.

Benno

By Kurt Utzinger Date 2012-02-20 13:48

Hi Sedat
I think that my answer to this question will not please you. In my opinion a top GM
will not lose if only playing for a draw and using a very boring "do-nothing-but-do-
it-well-strategy". Even players much beyond GM-strength will be able to produce
such boring drawn games. LOL about the crazy high Elo-ratings of the top engines.
Kurt

By Peter Martan Date 2012-02-20 15:26

Hi Kurt!
I support your opinion, at least if engines would play without any books, in which case the human masters could prepare much better for certain engines as for openings, as they are allowed to do against human opponents they know they are going to play against too.

Engines playing with books from human masters' theory of course is quite a different question but not the really fair one if pure strength of engines is asked for, to get to know this, engines should be playing fully by themselves. (As or the human players there is the difference, that they have to understand and remember their opening variants by themselves too and are not allowed to use databases during the match neither.)

Btw, if there were such vital interests in this question, there would sponsors have been found even till now to pay good human masters their good trophy money necessary to get them play against the engines.

By Sedat Canbaz Date 2012-02-20 21:18

[quote="Kurt Utzinger"]
Hi Sedat
I think that my answer to this question will not please you. In my opinion a top GM
will not lose if only playing for a draw and using a very boring "do-nothing-but-do-
it-well-strategy". Even players much beyond GM-strength will be able to produce
such boring drawn games. LOL about the crazy high Elo-ratings of the top engines.
Kurt
[/quote]

Dear Kurt,

Thank you for interesting comments...

Of course its just estimations...i dont claim 100 % that i am right about there will be more than 500 Elo difference

I can be wrong...or maybe you can be wrong too

Just a simple question:
-Do you believe that Top GMs of 2700-2800 Elo will perform 200-300 Elo points stronger than Fritz in Bahrain,Rebel,Chess Tiger 2007.1 ?

Please check the bellow table,which is based on GMs vs Machines results - played 6-7 years ago,where the difference is 190 Elo:
http://en.wikipedia.org/wiki/Human-computer_chess_matches

Note:only Junior chess engine was defeated (only two loses by Junior engine,it seems Junior team was not so lucky)

See also the performance of Chess Tiger,Rebel... vs Humans:
http://www.rebel.nl/resu.htm

I know very well that for a better conclusion,thousands of games are needed,but anyway i impressed by the performance of the top chess programs
And i dont think that we can find a such sponsor,who will make a such event of running thousands of games between Man vs Machine

Personally for me it's will be a BIG surprise,in case of Houdini,Rybka,Critter will be performed 300-400 Elo stronger than top GMs

Btw,who laughs last, laughs best

Best,
Sedat

By Kurt Utzinger Date 2012-02-20 22:05

Hi Sedat

Comparisons with matches in the past are of no great help. I have studied all man vs machine
games at sensful time controls. In most of the (too) few games I had feelings like these ones:

- man almost never used a "do-nothing-but-do-it-well-strategy"
- man had or wanted to prove something to the public (stress, nerves)
- man often played too risky
- man did in too many case choose the "wrong" openings leading to dynamic and equal but unbalanced positions
- some men showed indeed bad understanding of computer chess
- ....

You are surely aware of the fact that you can find in almost every opening enough very boring lines without
pawn majorities, with only one (or two) open line(s), with no space advantage for both players, and so on.
And in such games no real fight will (can) occur. To play such games, you do not need to be a GM and in such
games 6, 8 or even 12 cores do not much help the engines.

For the reasons mentioned above I still think that the best chess programs on very fast hardware would
be unable to reach a level with a Elo differerence of 500 Elo above men.

Kurt

By Sedat Canbaz Date 2012-02-20 22:55 Edited 2012-02-20 22:59

[quote="Kurt Utzinger"]
Hi Sedat

Comparisons with matches in the past are of no great help. I have studied all man vs machine
games at sensful time controls. In most of the (too) few games I had feelings like these ones:

- man almost never used a "do-nothing-but-do-it-well-strategy"
- man had or wanted to prove something to the public (stress, nerves)
- man often played too risky
- man did in too many case choose the "wrong" openings leading to dynamic and equal but unbalanced positions
- some men showed indeed bad understanding of computer chess
- ....

You are surely aware of the fact that you can find in almost every opening enough very boring lines without
pawn majorities, with only one (or two) open line(s), with no space advantage for both players, and so on.
And in such games no real fight will (can) occur. To play such games, you do not need to be a GM and in such
games 6, 8 or even 12 cores do not much help the engines.

For the reasons mentioned above I still think that the best chess programs on very fast hardware would
be unable to reach a level with a Elo differerence of 500 Elo above men.

Kurt
[/quote]

Dear Kurt,

I have different point of view (not like yours)

Some notes about the previous played matches-Man vs Machine:

1)In 1990 years = GMs were stronger than Chess Engines
2)In the early of 2000 years = GMs were equal to Chess Engines
3)In 2003/2004/2005/2006 = Chess Engines performed approx. 190 Elo better than GMs
4)Now we are in 2012 = We have Houdini,Rybka,Criiter and much faster hardwares than the past

Probably you missed my previous question:
-Do you believe that Top GMs of 2700-2800 Elo will perform 200-300 Elo points stronger than Fritz in Bahrain,Rebel,Chess Tiger 2007.1 ?

Btw,see the Elo difference between 4 core and 6 core:
http://www.sedatcanbaz.com/chess/ratings/scct-auto232/

Hope this helps

Best,
Sedat

By Benno Hartwig Date 2012-02-21 08:22

[quote="Sedat Canbaz"]3)In 2003/2004/2005/2006 = Chess Engines performed approx. 190 Elo better than GMs
4)Now we are in 2012 = We have Houdini,Rybka,Criiter and much faster hardwares than the past[/quote]Ja.
Aber kann eine heutige Engine, die ggü. der 2003-2006-Engine in den Computer-Rating-Listen um 400 ELO besser geführt wird, wirklich gegen GMs das Ergebnis erspielen, welches 190+400=590 ELO Differenz entspricht?

Ggf. ist die Arithmetik des ELO-Systems hier grundsätzlich in einem Grenzbereich
Und ggf. wirken Techniken und Ideen, die gegen die 2006-Engine recht erfolgreich war, irgendwelche Formen von anti-Computer-Strategien, auch gegen eine heutige Engine besser, als es die Computer-ELO-Differenz erwarten lässt.

Eine Engine, die bei ähnlichen Bewertugskriterien zwei plys tiefer rechnet, mag gegen die andere recht erfolgreich sein.
Aber kann sie in ähnlichem Maße auch erfolgreicher gegen Menschen sein? Aus meinem Bauch heraus: nein!

Benno

By Kurt Utzinger Date 2012-02-21 09:48

[quote="Sedat Canbaz"]
[...]

Probably you missed my previous question:
-Do you believe that Top GMs of 2700-2800 Elo will perform 200-300 Elo points stronger than Fritz in Bahrain,Rebel,Chess Tiger 2007.1 ?
Best,
Sedat
[/quote]

Hi Sedat
No, I didn't miss your previous question ... a question I simply can't answer. But
this question is of no interest to me. As I pointed out I mean that top GM's and
even players much below GM-level can produce boring an drawn games at
their will against todays best programs on fast hardware. If my assumption is
correct - I do not doubt - your question is of no relevance.
Best regards
Kurt

By Sedat Canbaz Date 2012-02-21 12:14 Edited 2012-02-21 12:24

[quote="Kurt Utzinger"]
[quote="Sedat Canbaz"]
[...]

Probably you missed my previous question:
-Do you believe that Top GMs of 2700-2800 Elo will perform 200-300 Elo points stronger than Fritz in Bahrain,Rebel,Chess Tiger 2007.1 ?
Best,
Sedat
[/quote]

Hi Sedat
No, I didn't miss your previous question ... a question I simply can't answer. But
this question is of no interest to me. As I pointed out I mean that top GM's and
even players much below GM-level can produce boring an drawn games at
their will against todays best programs on fast hardware. If my assumption is
correct - I do not doubt - your question is of no relevance.
Best regards
Kurt
[/quote]

Dear Kurt,

Thank you for your replay

I respect your opinion,but as i mentioned before i don't agree with you

Honestly your current comments are no big interest for me,due to there is no real facts

Note also that engines of 2200-2300 Elo can play draws vs GMs

In other words,there is only one way-who is right ?!

I challenge any Top GM (including GMs of +2600 Elo) to play with me

I mean,my conditions will be:

Hardware:i7 980X @4.33GHz
Engine:Houdini/Rybka/Critter
Book:Perfect 16
TC:15m+10s or 60m+15s (i can't accept 120/40,due to we can't produce many games)

Who are interested for a such challenge, please feel free to contact me:
http://www.sedatcanbaz.com/chess/contact/

Best Regards,
Sedat Canbaz

By Kurt Utzinger Date 2012-02-21 14:17

[quote="Sedat Canbaz"]
[...]
I challenge any Top GM (including GMs of +2600 Elo) to play with me

I mean,my conditions will be:

Hardware:i7 980X @4.33GHz
Engine:Houdini/Rybka/Critter
Book:Perfect 16
TC:15m+10s or 60m+15s (i can't accept 120/40,due to we can't produce many games)

Who are interested for a such challenge, please feel free to contact me:
http://www.sedatcanbaz.com/chess/contact/

Best Regards,
Sedat Canbaz
[/quote]

Hi Sedat

I would like to act as consultant for any player who is willing to
accept your challenge

but TC 15m+10s must be avoided and
the suggested 60m+15s the minimum time needed for such a match.
Kurt

By Joe Nettelbeck Date 2012-02-20 15:31

Pablo Restrepo ("Father") hat ja auf dem CB-Server immer wieder gezeigt, dass man zumindest Remis halten und ab und an auch gewinnen kann - allerdings mit Bullet-Time. Ich denke nicht, dass er dasselbe bei längeren Bedenkzeiten hinbekäme. Allerdings glaube ich auch nicht, dass die ELO-Differenz mehr als 500 betrüge...

By Clemens Keck Date 2012-02-20 16:56

far Less 500 ELO

even a 2200 ELO player can play draws vs the top engines. Humans can have the advantage of experience and knowlledge.

This is a main reason why I cannot accept rating lists with 3100 ELO and more. Even 3000 is still verry high.

Regards, Clemens

By Kurt Utzinger Date 2012-02-20 17:43

[quote="Clemens Keck"]
[...]This is a main reason why I cannot accept rating lists with 3100 ELO and more. Even 3000 is still verry high.
Regards, Clemens
[/quote]

Hi Clemens
So I do.
Kurt

By Sedat Canbaz Date 2012-02-20 18:28 Edited 2012-02-20 18:34

[quote="Clemens Keck"]
far Less 500 ELO

even a 2200 ELO player can play draws vs the top engines. Humans can have the advantage of experience and knowlledge.

This is a main reason why I cannot accept rating lists with 3100 ELO and more. Even 3000 is still verry high.

Regards, Clemens
[/quote]

Just i'd like to mention again and to make it more clear...

When i said, if we run the top engines on 'latest fast machines' i was not meaning about 1 core engines or quads or dual processors or 10 years old processors
Or i was not meaning that we will use the same engines which are played 10-15 yeas ago...
Or i was not meaning too what will be the Elo engine performance using own books...
Or i was not meaning also about Rybka Cluter's performance vs GMs

I mean especially for top engine performance on latest fastest i7 6 core or i7 12 core high-overclocked machines
Plus,in case of top mp engines are using superior private strong book

Actually it's not so hard to estimate about what will be the Elo points between Human vs Engines

And i can give you another example about the current issue

I think the bellow engines are around on the same level as Top GM of 2700-2800 Elo human points,right ?:
Chess Tiger 2007,Fritz in Bahrain,Rebel...

So...in other words,if we will start to test Chess Tiger 2007 engine against Rybka,Houdini,Criiter
(i mean if we run the Top MP engines on latest i7 6 core or 12 core machines via Auto232 mode or via online)

And after a such test at 40/120,can we expect the Elo difference to be less than 500 Elo ?
For example,engine match between Houdini 2.0c x64 6 core against Chess Tiger 2007 w32 1c

Lets take as example SSDF rating,which is very useful for a such comparison:
http://ssdf.bosjo.net/list.htm

Deep Rybka 4 x64 2GB Q6600 2,4 GHz   3216  32  -29  642  78%  3001
Chess Tiger 2007 256MB Athlon 1200 MHz   2786  22  -23  966  39%  2862

Note:the current SSDF Elo difference is 430 Elo,what will be the Rybka's Elo difference on 3 times faster machine ??

For example, if Rybka,Houdini Critter will be played on decent fast hardwares,e.g on i7 990X @ 4.60GHz
Note that in case of running MP engines, i7 990X @4.60 GHz is approx.3 times faster than Q6600 2.40GHz

That means you will get extra approx.130-150 Elo points

In other words,i expect the SSDF Rating to be:

1.Houdini 2.0c Pro x64 6 core i7 990X @4.60 GHz 3400 Elo
2.Deep Rybka 4.1 x64 6 core i7 990X @4.60 GHz   3350 Elo
3.Critter 1.4 x64 6 core i7 990X @4.60 GHz      3350 Elo
4.Chess Tiger 2007 256MB Athlon 1200 MHz   2786 Elo

Btw,SCCT Auto232's rating has almost same Elo points

I really wonder too,in case of participating the Top Human GM in SSDF Rating (e.g Kasparov,Kramnik,Carlsen,Topalov,Anand... )
Probably then, i expect the Top Human Players would be rated around 2700-2850 SSDF Elo points !

More than 10 years ago,we noticed that even the top single processor engines are started to play on GM levels,here is another proof:

Note:the above red comparison table (which is based on around 2000 years) is quite good indicator
It show us that 2000 years engines are started to play with same level as Top GM of 2700-2800 Elo human points

More details::
-Now we are in 2012 that means the current top engines play much stronger than 2000 years engines
-Many of the Top Engines have been improved to play stronger approx.200-300 Elo points (in that period of time)
-Plus 10 yeas ago,the processors were much slower... .e.g in those years,Fritz Benchmark kns values were around 500-1000 kns
-But nowadays, (on latest fast i7 machines) Fritz Benchmark tool generates around 20.000 kns values

In my estimation,the engines which are rated about 2700-2850 Elo points (based on SSDF,CCRL,SCCT... ratings) are expected to be rated almost same level as Top GMs

Just i'd like to mention again that 10-12 years ago, the top engines elo performance were around 2650-2750 Elo points

But nowadays,i mean exactly the same engine versions are improved to play stronger at least 200-300 Elo points

In 10 (ten) years period of time,the processors are become at least 15-20 times more faster
That means the top mp engines can gain extra at least 350-400 Elo points

Just imagine in 2012...what will be Elo difference between Top MP Engines vs GMs

Actually our comments are just estimations...

The best answer:its will be great,if there will be a serious mach (played with many games) -Man vs Machine 2012

Personally i guess,the Top MP Engines will be performed more than 500 Elo

We should not forget to mention the opening book's factor (not only the hardware speed or engine strength can effect the engine's performance)

From my experience i can say that the opening book is the key,which is playing a very important role to reach high elo engine performance
Depending on what kind of opening book usage,the engines can be performed approx. 0-200 elo points or even more...

And last,remember also that even some 2200-2300 Elo chess engines can play draws vs the Top GMs of 2700-2800 Elo

Hope this helps...

Greetings,
Sedat

By Peter Martan Date 2012-02-20 18:38

[quote="Sedat Canbaz"]
Plus,in case of top mp engines are using superior private strong book
[/quote]

Hi Sedat!

Here your "private" in "superior private strong book" has to be accent-bearing.

It's already a different thing if you give the engine's book that's to be used to the master quite a time before the match to prepare for it, and as for my personal point of view it would be more fair a contest, if the engine would play without a book at all.

At least, the master should be allowed to use an opening database of his own during the match too, if the engine may use such.

By Sedat Canbaz Date 2012-02-20 20:13

[quote="Peter Martan"]
[quote="Sedat Canbaz"]
Plus,in case of top mp engines are using superior private strong book
[/quote]

Hi Sedat!

At least, the master should be allowed to use an opening database of his own during the match too, if the engine may use such.

[/quote]

Dear Peter,

No problem... GMs can use any opening database as they wish

And in any case (with the above conditions,which i stated) the top mp engines are expected to be performed min 500 Elo difference

Regards,
Sedat

By Peter Martan Date 2012-02-20 20:40 Edited 2012-02-20 20:48

Well, I guess, Sedat, we wont get to know unless we raise the money to make a master try.
But even if we'll manage to do so, whom's Elo would be the ones to be counted as reference, the master's or the engine's ones?
I mean did you mean the master would perform 500 less then 3300 Celo or the engine more than 2600 human Elo?
Get me right, in both cases the Elo of the two contrahents would be fully meaningless compared to each other's normal collective.

You might think, nevermind, but you see, 6 or 10 or 12 games as a baseline for calculation of Elo is o.k. for human rankings, but only because of a traditional long term agreement and of course it was a one-match-performance only, as for computer games you'ld need very many games for a relevant ranking and the master wouldn't earn any Elo counting against human opponents, otherwise it would be some less expensive to get such a match going probably.

Sorry for being oversophisticated, I can understand your question some more simple of course and your interest in it is very intelligible, btw it's a rather old one, I just don't think it to be answered easily nowadays, if at least approximately at all.
Or to say it much more simple: I don't believe in Elo, not for humans and not for engines to measure playing strenght.
Such can be measured always only relatively, relatively to certain single chess positions and to certain opponents and to certain conditions like time and hardware.

By Michael Scheidl Date 2012-02-21 11:45

I think it would make a big difference if the GMs knew whom, or what they are playing against or not. A kind of "blind" test tournament should be done, with GM/IM and engines mixed, without the human players knowing their opponents each. In addition to a (small) appearance fee, extra price money for wins should be awarded.

By that, chances would be bigger that "normal" chess is being played resulting in realistic performances, and not to have the humans strictly going for draws only, which seems somewhat artificial and not typical. - I would not doubt that comps can achieve more than 3000 Elo, e.g. 80% (!!) against Carlsen would be a perf. of ~3075(*). But for more, I'd have to see it yet before I believe it.

*) Note that 3075 would require a more than 90% performance against 2700.

A debateable point is that we measure strength increases in cc. based on comp vs. comp only, anymore. It is not at all clear if increases since the year 2000 are that big too, versus humans. I cannot prove the opposite but I doubt it.

By Sedat Canbaz Date 2012-03-01 00:29

Dear Chess Friends,

My final words over this issue...

The bellow 2007/2008 data is based on handicapped games played by Rybka (without pawn) vs GMs average 2600 Elo

Dzindzichashvili vs Rybka
4.0 - 4.0 8 games
2.5 - 1.5 4 games
http://en.wikipedia.org/wiki/Roman_Dzindzichashvili

-----------------------------------------------------
Rybka: 4.5
Benjamin: 3.5
http://rybkaforum.net/cgi-bin/rybkaforum/topic_show.pl?tid=1883

-----------------------------------------------------
Rybka: 10.0
Ehlvest: 4.0
http://en.wikipedia.org/wiki/Jaan_Ehlvest

-----------------------------------------------------

More Details (based on my testings):

Rybka 3 using a strong opening book performed approx. +200 Elo better than without using book
http://sedatchess.110mb.com/index.php?p=1_35

Rank Name                 Elo    +    -   games score oppo. draws
01 Ultra Blitz 3          3319   23   23   560   60%  3257   50% 
70 NoBook                 3100   25   25   460   37%  3176   53%

*Note:NoBook is empty.ctg (Rybka 3 is played without using any opening book)

-----------------------------------------------------

Rybka with Full Performance (with all pieces) performed approx. +250 Elo better
http://www.sedatcanbaz.com/chess/ratings/scct-auto232/

Rank Name                         Elo    +    -   games score oppo. draws
  36 Rybka 4.1 x64 1c             3201   13   13  1725   48%  3217   49% 
  52 Rybka 4.1 WP x64 1c          2954   29   29   470   61%  2875   27%

*Note:Rybka 4.1 WP x64 1c is played without using a full pawn

-----------------------------------------------------

In my estimation (based on the above data/sources)
1)Rybka +75 Elo
2)Private Strong Book / Endgames +150 Elo
3)Faster Hardware +150 Elo
4)Newer/Stronger version +150 Elo
5)Full Performance (with all pieces) +250 Elo
-----------------------------------------------------
FINAL CONCLUSION:
-Rybka 4.1 mp (on 2x Intel Xeon X5667 @ 4.60GHz)is expecting to be rated approx.750-800 Elo stronger than GMs of 2600 Elo points

Best Regards,
Sedat Canbaz