Hallo zusammen,
im letzten Monat haben Hauke Lutz und meine Wenigkeit 2 neue Eröffnungs-Datenbanken entwickelt, die die Remisquote in Engine-Engine Matches reduzieren sollen, ohne die Ergebnisse zu verzerren. Wir nennen sie SALC-Sets (Short and Long Castling). In allen Stellungen haben Weiß und Schwarz auf die gegenüberliegende Seite rochiert.
Die SALC-Sets können ab sofort auf meiner Website heruntergeladen werden und ich benutze sie ab sofort für meine Tests.
http://spcc.beepworld.deIch hänge hier das dazugehörige ReadMe-File an, in dem ich alles ausführlich erläutere (sowohl Idee als auch die Realisierung).
Stefan
This folder contains 2 PGN-databases and 2 EPD-databases:
- 10moves_SALC_500.pgn = 500 opening positions, well edited and mixed for serious testwork.
- 12moves_SALC_10k.pgn = 10000 opening positions for big tournaments or randomized opening selection.
- 10moves_SALC_500.epd and 12moves_SALC_10k.epd: Same (final) positions as in the .pgn-files but as EPD (final
board-positions only, without moves).
Use that files, when you use the LittleBlitzerGUI for testing, because the LittleBlitzerGUI has an
en-passant-bug (captured en-passant-pawns are not deleted by the LittleBlitzerGUI, if an en-passant-move
appears in the moves of the opening-PGN file (!!!))
Goal: Reduce the draw rate in engine-engine matches/testruns/tournaments (castling to opposite directions
with queens still on the board makes nice king-attacks possible...), because the faster the computers get,
the higher the quality of computerchess get and the higher the draw-rate in engine-engine-matches get...so
the computerchess is in danger to die the "draw-death" in the near future.
But we didnt want to go the simple way of using strange gambit-openings or positions with great (material)
imbalance. Take a look a the working steps-protocol below, where you can see, which filter-methods Hauke Lutz
used, in order to get only (nearly) balanced positions.
Idea and testwork/verification: Stefan Pohl
All work (editing, sorting) done by Hauke Lutz (using PGNscanner 0.92 (a really nice tool by Gabriel
Guillory) and EXCEL)
All games taken from Adam Hair's 12-moves-PGN openings database and his 10-moves-PGN openings-database.
So a big THANX to Adam Hair and Gabriel Guillory!!!
Here the protocol of the working steps:
Step 1 (by Stefan Pohl using the FritzGUI): Filter all games, where (at move 12 / 10) both sides still
have a queen and both sides castled to opposite directions.
= 17665 positions (12 moves deep) (out of 397457 games)
= 4602 positions (10 moves deep) (out of 199041 games)
Working steps 2-10 by Hauke Lutz (PGNscanner: thinking-time/position: 5 seconds (singlecore,
4.5 GHz (i7-4930k, Fritzmark 3367)):
Step 2: Checked both databases for duplicate games with the PGNscanner. Found nothing (nice work, Adam Hair!)
Step 3: Checked the 12moves-database with Komodo 8 (using PGNscanner (eval-interval of +/-0.50)) and deleted
all games with an evaluation outside the eval-interval.
Step 4: Deleted some games of the 12moves-database with ECO-code B and some games with white long castlings
for a better balance. Reduced the number of games to 10000. Used EXCEL for this.
Step 5: Mixed the games of the 12moves-database (by hand) by the castling-direction (we didnt want
some thousand games with white long castlings in a row followed by some thousand games with white short
castlings...)
Step 6: Checked the 10moves-database with Komodo 8, Houdini 4, Gull 3 (using PGNscanner (eval-interval
of +/-0.40)) and deleted all games if one engine-evaluation was outside the eval-interval.
Step 7: Checked the 10moves-database with Komodo 8 and Stockfish 5 (using PGNscanner (eval-interval
of +/-0.20)) and deleted all games if one engine-evaluation was inside the eval-interval, because we didnt
want positions which are too drawish.
Step 8: Counted/Analyzed the ECO-codes of the 10moves-database with EXCEL and deleted some ECO B+C positions
for a better ECO-code balance (and reduced the number of games/positions to 500).
Step 9: Mixed the 10moves-database (by hand) for a (nearly) uniform mixture of ECO-codes for better results,
if only a part of the database is used for an engine-testrun.
Step 10: 5 Bullet-testruns (singlecore, 20''+200ms, Stockfish 5 against Gull 3), using the complete 500
positions of the 10moves-database, and mixed the 10moves-database a second time, based on the
testrun-results (in 50 positions-blocks).
Step 11 (by Stefan Pohl): Changed the results of all games in the PGN-files to 1/2-1/2, deleted all
annotations (created by the PGNscanner) and created the EPD-files for using the SALC-openings in the
LittleBlitzerGUI.
A final gauntlet-testrun (singlecore, 70''+700ms) of Stockfish 140928 (1000 games against Houdini 4,
Komodo 7a, Gull 3, Fire 3 and Rybka 4.1 (=5000 games)) using the 10moves_500_SALC opening-positions-set
lowered the draw-rate down to 39.0% (original testrun (same conditions but using a "normal"
opening-positions-set (fq500n.pgn) with 500 positions) had a draw-rate of 47.9%.
So the number of draws was more than 18.5% lower with the SALC-set (!). And the overall score of Stockfish
was nearly the same (SALC-set: 1% lower (-7 Elo) = clearly inside the errorbars). And the aggressive
playing Stockfish-engine did not benefit from the SALC-positions (we were not sure about that...).
So the goal of the creation of the SALC-opening-positions-set was reached: a significant lower draw-rate,
while keeping the overall score nearly the same.
And - as a nice side effect - the testrun with the 10moves_500_SALC opening-positions-set took only 93 hours,
instead of the 100 hours, which the testrun with the "normal" opening-positions-set (fq500n.pgn) took.
That means around 7% less time- and power-consumption for the same number of played games...And all games
were adjusted as draw at move 120. With all games played to the end (technical draw), the timesaving would be
definitly higher (around 10%, we guess)...
Enjoy this next step of chess-engine matchplay and testwork. Less draws, more spectacular games/mates,
without distorting the test-results and scores !