Caissa v1.21

I'm excited to announce Caissa version 1.21, the latest release packed with improvements to make gameplay even stronger and more stable.

Progression test

TC=60+0.6s, Book=UHO_Lichess_4852_v1.epd

Elo   | 35.09 +- 4.16 (95%)
Conf  | 60.0+0.60s Threads=1 Hash=128MB
Games | N: 6278 W: 1758 L: 1126 D: 3394
Penta | [2, 446, 1624, 1052, 15]

TC=8+0.08s, Book=UHO_Lichess_4852_v1.epd

Elo   | 22.65 +- 3.68 (95%)
Conf  | 8.0+0.08s Threads=1 Hash=8MB
Games | N: 10000 W: 2812 L: 2161 D: 5027
Penta | [31, 974, 2402, 1499, 94]

TC=10+0.1s, Book=DFRC.epd

Elo   | 18.98 +- 5.35 (95%)
Conf  | 10.0+0.10s Threads=1 Hash=16MB
Games | N: 5002 W: 1114 L: 841 D: 3047
Penta | [38, 462, 1258, 675, 68]

Changes

• Fixed problems detected by sanitizers that could lead to potential crashes.
• New neural network trained on a total of 13 billion positions.
• Improved eval correction (Stockfish style).
• Tuned parameters at long time control.
• Various search improvements.

Special thanks to @aronpetko for invaluable access to the OpenBench instance.

Caissa v1.20

Progression test

TC=40+0.4s, Book=UHO_Lichess_4852_v1.epd

Elo   | 22.15 +- 5.24 (95%)
Conf  | 40.0+0.40s Threads=1 Hash=64MB
Games | N: 7808 W: 2054 L: 1557 D: 4197
Penta | [7, 696, 2007, 1181, 13]

TC=10+0.1s, Book=UHO_Lichess_4852_v1.epd

Elo   | 18.95 +- 2.35 (95%)
Conf  | 10.0+0.10s Threads=1 Hash=16MB
Games | N: 40014 W: 10644 L: 8464 D: 20906
Penta | [98, 3904, 9948, 5834, 223]

TC=1+0s, Book=UHO_Lichess_4852_v1.epd

Elo   | 71.57 +- 4.99 (95%)
Conf  | 1.0+0.00s Threads=1 Hash=1MB
Games | N: 10240 W: 3832 L: 1752 D: 4656
Penta | [89, 731, 1889, 1833, 578]

Changes

• Bigger neural net (11 king buckets instead of 5) trained on total 12.6B positions
• Improved performance in ultra short time controls without increment

Caissa v1.19

Progression test

TC=60+0.6, Book=UHO_Lichess_4852_v1.epd

Elo   | 22.12 +- 7.60 (95%)
Conf  | 60.0+0.60s Threads=1 Hash=64MB
Games | N: 3790 W: 1016 L: 775 D: 1999
Penta | [1, 344, 969, 575, 6]

Changes

• New neural net trained on total 12.5B positions
• Various search improvements
• Search parameter tuning at LTC

Caissa v1.18

Progression test

TC=8+0.08, Book=UHO_Lichess_4852_v1.epd

Elo   | 21.60 +- 3.48 (95%)
Conf  | 8.0+0.08s Threads=1 Hash=8MB
Games | N: 18582 W: 5085 L: 3931 D: 9566
Penta | [62, 1822, 4474, 2766, 167]

TC=60+0.6, Book=UHO_Lichess_4852_v1.epd

Elo   | 21.98 +- 5.45 (95%)
Conf  | 60.0+0.60s Threads=1 Hash=128MB
Games | N: 7266 W: 1924 L: 1465 D: 3877
Penta | [10, 654, 1850, 1105, 14]

Changes

• New neural net trained on total 9.3B positions. Introduced more positions from regular chess games (instead of DFRC games).
• Various search improvements

Caissa v1.17

Progression test

TC=8+0.08, Book=UHO_Lichess_4852_v1.epd

Elo   | 20.60 +- 4.85 (95%)
Conf  | 8.0+0.08s Threads=1 Hash=8MB
Games | N: 9690 W: 2670 L: 2096 D: 4924
Penta | [38, 947, 2337, 1449, 74]

TC=60+0.6, Book=UHO_Lichess_4852_v1.epd

Elo   | 19.83 +- 6.47 (95%)
Conf  | 60.0+0.60s Threads=1 Hash=64MB
Games | N: 5000 W: 1272 L: 987 D: 2741
Penta | [4, 442, 1332, 709, 13]

TC=8+0.08, Book=DFRC.epd

Elo   | 15.59 +- 6.38 (95%)
Conf  | 8.0+0.08s Threads=1 Hash=8MB
Games | N: 4394 W: 948 L: 751 D: 2695
Penta | [28, 433, 1110, 566, 60]

Changes

• New neural net trained on total 7.1B positions. Introduced more high quality games from SPRT tests to the dataset (~220M positions) and random endgame positions scored with 7-man TB (~40M positions). Finetuning previous net for over 50B iterations.
• Small speedup
• Search improvements:
  • Prevent search explosions in LMR
  • Simplify LMR history formula
  • Additional history bonus based on score difference
  • Store eval in TT as soon as possible
  • Higher RFP margin if opponent is threating a capture
• Use threats info to generate less illegal king moves
• Tweak transposition table replacement scheme

Caissa v1.16

Progression test

TC=10+0.1, Book=UHO_4060_v2.epd

Elo   | 36.65 +- 4.71 (95%)
Conf  | 10.0+0.10s Threads=1 Hash=16MB
Games | N: 10000 W: 2917 L: 1866 D: 5217
Penta | [25, 788, 2387, 1711, 89]

TC=60+0.6, Book=UHO_4060_v2.epd

Elo   | 34.15 +- 7.41 (95%)
Conf  | 60.0+0.60s Threads=1 Hash=128MB
Games | N: 3848 W: 1065 L: 688 D: 2095
Penta | [4, 276, 991, 645, 8]

TC=10+0.1, Book=DFRC.epd

Elo   | 27.03 +- 7.27 (95%)
Conf  | 10.0+0.10s Threads=1 Hash=16MB
Games | N: 3568 W: 866 L: 589 D: 2113
Penta | [23, 311, 879, 508, 63]

Changes

• New neural net trained on total 6.9B positions. Introduced high quality games from SPRT tests to the dataset (~350M positions).
• SPSA parameter tuning at long time controls
• Eval correction improvements
• Smaller transposition table entries
• Few speedups (around 4% in total)
• Various search improvements

Caissa v1.15

Progression test

TC=8+0.08, Book=UHO_4060_v2.epd

Elo   | 31.34 +- 6.59 (95%)
Conf  | 8.0+0.08s Threads=1 Hash=16MB
Games | N: 5002 W: 1398 L: 948 D: 2656
Penta | [19, 427, 1189, 817, 49]

TC=60+0.6, Book=UHO_4060_v2.epd

Elo   | 25.89 +- 10.85 (95%)
Conf  | 60.0+0.60s Threads=1 Hash=128MB
Games | N: 1882 W: 519 L: 379 D: 984
Penta | [3, 156, 485, 292, 5]

TC=8+0.08, Book=8moves_v3.epd

Elo   | 21.64 +- 3.28 (95%)
Conf  | 8.0+0.08s Threads=1 Hash=16MB
Games | N: 10000 W: 1485 L: 863 D: 7652
Penta | [33, 580, 3192, 1122, 73]

TC=8+0.08, Book=DFRC.epd

Elo   | 29.25 +- 6.94 (95%)
Conf  | 8.0+0.08s Threads=1 Hash=16MB
Games | N: 4000 W: 1001 L: 665 D: 2334
Penta | [39, 335, 951, 601, 74]

Changes

This release focuses on bug fixes and search improvements. Unfortunately I wasn't able to train a better net than the one used in the previous release.

• Fix checkmate on fifty move (#16)
• Fix "nodes" stat being counted twice in quiescence search
• Prevent the search stack from overflowing (could happen at very long time controls)
• Fix hang when reaching max depth with threads > 1
• Fix bug in KPvK evaluation
• Normalize reported score so that +100 cp means 50% chance of winning
• Improve time management
• Use only one killer move
• Introduce probcut
• Introduce countermove heuristics
• SPSA tuning of all parameters
• Many other search improvements and speedups

Caissa v1.14.1

Progression test

TC=8+0.08, Book=UHO_4060_v2.epd

Elo   | 35.98 +- 7.68 (95%)
Conf  | 8.0+0.08s Threads=1 Hash=32MB
Games | N: 4002 W: 1222 L: 809 D: 1971
Penta | [15, 350, 904, 671, 61]

TC=60+0.6, Book=UHO_4060_v2.epd

Elo   | 39.43 +- 14.73 (95%)
Conf  | 60.0+0.60s Threads=1 Hash=128MB
Games | N: 1000 W: 289 L: 176 D: 535
Penta | [2, 75, 235, 184, 4]

TC=180+1.8, Book=UHO_4060_v2.epd

Elo   | 37.14 +- 10.37 (95%)
Conf  | 180.0+1.80s Threads=1 Hash=256MB
Games | N: 2000 W: 569 L: 356 D: 1075
Penta | [0, 149, 495, 350, 6]

Changes

• New neural net trained on total 4.5 billion positions generated in Caissa self-play games
  • experimenting with trainer hyperparameters, switched to Adam optimizer
  • ~70B iterations
  • introduced positions from games started from a position sampled from high-quality tournament games with other engines (TCEC, CCRL, SP-CC)
  • introduced positions from SPRT games (both STC and LTC)
• Introduce separate BMI2 binary version (uses more efficient PEXT move generation)
• Improved continuation history heuristics
• SPSA tuning search and move ordering parameters
• Numerous search improvements:
  • better repetition handling
  • tweaking heuristics (LMR, IIR)
  • improved continuation history heuristics
• Time manager tweaks (noticeable improvement in time control with big time increment)
• Numerous small speedups
• Started using PGO (profile-guided optimizations) for AVX2 and BMI2 builds

Big thanks to Andrew for creating great OpenBench testing platform as well as Gabe and Sazgr for providing some cores.

Caissa v1.13.1

Hotfix for a bug with EvalFile option causing no net to be used

Caissa v1.13

Changes

• New neural net trained on 3.5B positions
• Use threats in history heuristics (based on Berserk)
• Average base score used for aspiration windows
• Embed neural network in executable
• Switch from MSVC to GCC compiler for Windows release

Elo gain

Book=UHO_4060_v2, TC=40+0.4

Score of Caissa 1.13 AVX2 vs Caissa 1.12 AVX2: 459 - 325 - 877  [0.540] 1661
...      Caissa 1.13 AVX2 playing White: 450 - 6 - 376  [0.767] 832
...      Caissa 1.13 AVX2 playing Black: 9 - 319 - 501  [0.313] 829
...      White vs Black: 769 - 15 - 877  [0.727] 1661
Elo difference: 28.1 +/- 11.5, LOS: 100.0 %, DrawRatio: 52.8 %

Book=DFRC, TC=8+0.08

Score of Caissa 1.13 AVX2 vs Caissa 1.12 AVX2: 326 - 231 - 773  [0.536] 1330
...      Caissa 1.13 AVX2 playing White: 229 - 64 - 372  [0.624] 665
...      Caissa 1.13 AVX2 playing Black: 97 - 167 - 401  [0.447] 665
...      White vs Black: 396 - 161 - 773  [0.588] 1330
Elo difference: 24.9 +/- 12.1, LOS: 100.0 %, DrawRatio: 58.1 %

Book=UHO_4060_v2, TC=8+0.08

Score of Caissa 1.13 AVX2 vs Caissa 1.12 AVX2: 555 - 436 - 1121  [0.528] 2112
...      Caissa 1.13 AVX2 playing White: 530 - 22 - 506  [0.740] 1058
...      Caissa 1.13 AVX2 playing Black: 25 - 414 - 615  [0.315] 1054
...      White vs Black: 944 - 47 - 1121  [0.712] 2112
Elo difference: 19.6 +/- 10.1, LOS: 100.0 %, DrawRatio: 53.1 %