Engine – Process test set

 

 

Solving a series of test positions is an objective way of evaluating the strength of an engine. But, it is a tedious task to enter each of the positions, start the engine, and then protocol the solution times.

 

You can run a whole database of test positions automatically. The solution times and values are automatically recorded together with the moves. Even the engine name is noted behind the first move of the solution.

 

After installing the program, you will find a test set of 200 difficult tactical positions for you to experiment with. It is in the directory ChessBase\Testsets. This is how to run a test set:

 

   Switch to the database window and open a database of test positions. The database should only contain positions, and no games. Each position should have the first move of the solution. If there are two solutions, then the alternative move should be given as a variation.

 

   In the board window, click Analysis – Process test set.

 

The following dialog box appears:

 

 

Maximum time: Defines the maximum length of time the program should work on each position.

 

Extra ply: When the engine finds the solution, then it should not stop immediately. The correct move may have come up by chance. The program should at least finish calculating the current ply depth. This corresponds to the setting Extra ply = 1. If you want to be sure, you can make the program go one or two plies deeper. This may, however, greatly extend the processing time.

 

First position: Allows you to specify from which position the program should start. Thus, if you have interrupted a session, you can easily continue at a later time.

 

Previous results: The program keeps a record of the results achieved by each engine that has run the test. It displays these in a sorted list. This contains the following results:

 

   The number of positions solved in the allocated maximum time per position.

   The average solution time of all the positions that were solved.

   The average solution time of all the positions (including those not solved).

 

This last value, which of course is influenced by the maximum time set, is the most illuminating. While the program is working you see the current results (how many positions solved, percentage, average time) in the area just below the status bar at the top of the board window.

In the displayed ranking list (click “Previous results”), you can delete the results of individual engines, or all results. You can clip the results (i.e., copy them into the clipboard and insert them in a spreadsheet program).

 

The unsolved positions are marked in the database with “medals”. Positions that were not solved in the current test run are marked with black medals for “Tactical blunders”. Those not solved by any program, get the “User” medal (blue-green). You can see the medals in the database list and search for them with the search mask. This is the best way to get a list of unsolved positions. To load one position after another, switch to the board window and press F10.

 

Note: You can define a position as solved if one or more moves are not played. This is done by annotating them with a "?". For instance if the notation says 1.Ne5? (1.Ng5?) then the position is solved when neither of these moves is in the main line.