Gomocup 2016, the 17th tournament (April the 22nd-24th, 2016)

The Gomocup 2016 took place on April the 22nd-24th. This year is 15 year anniversary of Gomocup and it is 10 years since Tomas Kubes took over the managing of the Gomocup from Vojta Havranek and Jiri Fontan, which was originally hosted at Charles University in Prague. After such long time, Tomas decided to pass the organization of the tournament to Kai Sun and Tianyi Hao, and thus the tournament started to be hosted at Cornell University in Ithaca, USA and Tsinghua University in Beijing, China from this year.

There were 6 computers available whose configurations were as follows:

  • Windows 10, x64 Virtual Machine with 4 cores and 8GB RAM (the host was Ubuntu 14.04, x64 with Intel Core i7-4790 (3.6/4Ghz) and 16GB RAM)
  • Windows 10, x64 Intel Core i7-6700K (4/4.2Ghz), 8GB RAM
  • Windows 10, x64 Intel Core i5-4210M (2.6/3.2Ghz), 8GB RAM
  • Windows 10, x64 Intel Core i7-3630QM (2.4/3.4Ghz), 8GB RAM
  • Windows 7, x64 Intel Core i5-2450M (2.5/3.1Ghz), 4GB RAM
  • Windows 7, x64 Intel Core i5-2450M (2.5/3.1Ghz), 4GB RAM

For each game type, only cores with similar speed were used.

In addition to freestyle gomoku and standard gomoku, this year we introduced renju tournament. The tournament manager Piskvork supports the renju rule since version 8.8. The size of board for renju tournament was 15, and the rule used in the renju tournament was based on the international rules of renju with the following modifications:

  • Similar to the freestyle and standard tournaments, the openings for the renju tournament are prepared by renju experts, and are not limited to the 26 patterns;
  • Players (i.e. the AIs) are not allowed to pass;
  • Any game in renju tournament will be an automatic draw after 200 moves.

The openings for Gomocup 2016 were chosen by the following people (sorted alphabetically according to last names):

  • Alexander Bogatirev - Gomoku player, manager of Team Russia online, and member of Gomoku Committee RIF
  • Aivo Oll - Renju player, 7 dan, former Estonian champion, European champion, and world champion
  • Weifu Qiu - Renju player, 3 dan, ranked 4th in the Renju World Championship via Correspondence 2014
  • Rong Xiao - Gomoku expert who proposed the gomoku opening rule "swap after first move", which is one of the most popular gomoku opening rules in China

Thank you all.

openings_16

Note that since this year we had more cores to run the tournament, we prepared 6 openings instead of 3 or 4 for every rule, and we believed that could help reduce the randomness of tournament results. Moreover, to make the openings for different leagues more balanced, Freestyle and Standard leagues no longer shared the same set of openings this year.

What is new?

  • Updates
    • Goro 2016 - changed a bit.
    • Onix 2010b (2016) - a stable version of Onix 2010.
    • Pela 2016 - merge the modifications of AcaPela into Pela; add support to renju rule.
    • PureRocky 2016 - most of the tree based algorithms are removed from the previous AI PureGM, and the AI has been trained over the last several months to get stronger.
    • SlowRenju 2016 - improvement of algorithm and evaluation; support hybrid x86-64 to speed up; add support to renju rule.
    • XoXo 2016 - completely new.
    • Yixin 2016 - improvement of algorithm; support hybrid x86-64 to speed up; bug fixes.
    • Zetor 2016 - partially rewritten.
  • New AI
    • Carbon 2016 - an open-source Gomoku program written by Michal Czardybon. The AI algorithm was one of the strongest as for 2002 and it's reasonable even now. It is ported to Gomocup interface and further improved by Petr Lastovicka. Carbon supports freestyle, standard, and renju rules.
    • Chis 2016 - an open-source Gomoku program written by Peiming Zheng. Chis supports freestyle rule.
    • Eulring 2016 - a Gomoku program wrriten by Bian JunYi. Eulring supports freestyle rule.
    • Ignitor 2016 - a Gomoku program wrriten by Jaechan Lee. Ignitor supports freestyle rule.
    • Stahlfaust 2016 -an open-source Gomoku program written by Marco Kunze and Sebastian Nowozin. It is ported to Gomocup interface by Petr Lastovicka. Stahlfaust supports freestyle rule.
    • XL 2016 - an open-source Renju program written by Chengtao Chen. It is ported to Gomocup interface by Kai Sun and Tianyi Hao. XL supports renju rule.
  • Other News
    • RenjuSolver - Although no update is made to the AI, in addition to freestyle groups, Renjusolver will also take part in standard and renju groups by modifying its configuration file.
    • FastGomoku - FastGomoku 2014 was used instead of FastGomoku 2015 in Gomocup 2016 because it has been shown that version 2014 is significantly stronger than version 2015.
    • Noesis - This AI was removed from Fastgame since Gomocup 2009 for some unknown reasons. After a lot of testing, it was added to Fastgame this year.

In Gomocup 2015 we have faced compatibility problems of DOS programs not supported by Windows 8. This year Windows 10 appears and all new AIs are required to be compatible with Windows 10. Compatibility with Windows XP is not required anymore. Therefore two AIs were removed from this year:

  • HighFive - HighFive crashes as soon as a game is started on Windows 10. The author has tried to fix it but it is still unable to work.
  • Amoeba - The memory usage of Amoeba exceeds the quota too often on Windows 10.

Additionally, since the modifications of AcaPela have been merged into Pela 2016 by Petr Lastovicka, AcaPela was also removed.

There were 3 Freestyle groups, 1 Fastgame group, 1 Standard group, and 1 Renju group this year. AIs were divided into different freestyle groups according to the placement in the last tournament. For Freestyle 2 and 3, the top 4 AIs were moved up to the next group. If the top k (k>4) places were all taken by new (or updated) AIs in a group, then all these k AIs would advance to the next group.

Ranking 2015

  1. YIXIN
  2. RENJUSOLVER
  3. TITO
  4. GORO
  5. SWINE
  6. HGARDEN
  7. HEWER
  8. ACAPELA
  9. GMOTOR
  10. SLOWRENJU
  11. PELA
  12. HIGHFIVE
  13. KANEC
  14. AMOEBA
  15. ZETOR
  16. JUDE
  17. PECUCHET
  18. QMENTAT
  19. NOESIS
  20. PISQ
  21. NABAMOKU
  22. PUSKVOREC
  23. IMRO
  24. PUREGM
  25. VALKYRIE
  26. BENJAMIN
  27. LICHT
  28. XOXO
  29. CRUSHER
  30. FASTGOMOKU
  31. PROLOG
  32. FIVEROW
  33. MUSHROOM

In accordance with the last tournament, memory limit/time per move/per match were determined to be same as Gomocup 2015:

TOURNAMENT TIME LIMIT PER MOVE [S] TIME LIMIT PER MATCH [S] MEMORY LIMIT [MB] BOARD SIZE RULE FOR WIN
Freestyle 1 league 300 1000 350 20 five or more stones
Freestyle 2 league 30 180 350 20 five or more stones
Freestyle 3 league 30 180 350 20 five or more stones
Fastgame 5 120 350 20 five or more stones
Standard 300 1000 350 15 exactly five stones
Renju 300 1000 350 15 renju rule

There were difficulties like every year. One problem was caused by the conflict between Piskvork and PsqUploader (the tool for uploading PSQ files from Piskvork client to the server during the tournament to present the online board in Gomocup online). Sometimes PsqUploader reads the temporary file generated by Piskvork, while at the same time Piskvork is writing the same file. In such scenario, Piskvork would pop a message box saying it cannot create the temporary file, and the game would be stuck at that point, which leads to a false positive timeout for the side to move. We solved the problem by adding a temporary patch to Piskvork. Because of that, Freestyle 3, Freestyle 2, as well as Fastgame was restarted. Another problem was caused by the AI Nabamoku, which was an AI written in Java using multiple threads for thinking. Although we restrict every AI to run on one core by setting the CPU affinity in Piskvork, we found that computers were still quite stuck when running Nabamoku. Therefore, to ensure that Nabamoku would not influence the other AIs' thinking, Freestyle 2 was restarted again without Nabamoku, and the games of Nabamoku was played independently at the end of Freestyle 2.

This year the Freestyle leagues were larger than last year. Since there were many new brains this year, we had modified the rule such that for Freestyle 2 and 3 at least 4 brains would advance to the higher league.

Results of Gomocup 3. Freestyle gomoku league

In Freestyle 3, Carbon (an open source engine) went to the 1st place quickly, followed by Onix (which used to be in Freestyle 1 before 2012). Some other new engines (such as Chis and Eulring) were also following. At last, since the first 7 places were all taken by new or updated AIs, all these 7 AIs went to Freestyle 2, which were Carbon, Onix, Chis, Eulring, XoXo, Ignitor, and FastGomoku.

Results of Gomocup 2. Freestyle gomoku league

When Freestyle 2 started, Carbon and Onix went to the top two places at the beginning, which was similar to Freestyle 3. The 3rd place was competed between Chis and Kanec, and Chis won it at last by a higher ratio. Carbon, Onix, Chis, and Kanec went to Freestyle 1 at last.

Results of Gomocup 1. Freestyle gomoku league

Freestyle 1 was very interesting this year. At first, although Carbon had won more games than Onix in Freestyle 2 and Freestyle 3, Onix seemed to have a better performance in Freestyle 1. While Yixin soon took control of Freestyle 1, the competition between the 2nd and the 7th places were very tight. There was a time that the differences between the points of these places were no more than 1. Although RenjuSolver didn't perform well in the first two openings such that it had once fallen to the 8th place, it played quite well in the following openings and soon caught up with the other AIs. What was a surprise to us, RenjuSolver won all the 26 games in the last opening.

The 2nd to the 5th places were not fixed until the very end several games, while RenjuSolver, Tito, SlowRenju, and Goro were competing for them. At first Goro lost 2 games to Hewer and stayed at the 5th place. The 2nd place was finally determined between RenjuSolver and Tito by a double-win of RenjuSolver, which made the final result to be 7:5 between them. The 3rd place was not fixed until the last game between Tito and Yixin, since the points of SlowRenju and Tito were the same, and the numbers of winning games were only differenced by 1. At last Tito lost to Yixin and stayed at the 4th place, while SlowRenju went to the 3rd.

The winner of Freestyle was Yixin. The second place was taken by RenjuSolver, and the third by SlowRenju.

In the Fastgame, while Yixin and Goro quickly outperformed the other AIs, the games between Slowrenju, Hewer, and Tito were very tight and those three engines competed for the 3rd up to the very end. The Fastgame was won by Yixin. The second was Goro, and after a rigorous competition, Slowrenju became the 3rd. Nabamoku was removed from Fastgame because it had too many timeouts (~22%) this year.

Although the scale of the standard league was the same as last year, the AI list in this league changed a bit. Valkyrie was in this league in previous Gomocups, however, we noticed that Valkyrie crashed many times this year. After checking it we found that actually, Valkyrie did not support standard gomoku on 15x15 board. Therefore, Valkyrie was removed from the final result. Similar to 2015, Yixin won the standard game with a big advantage. The second was Renjusolver which was its first time to take part in this group. The third was Tito.

It was the first year that we started a renju league. In renju, there are forbidden moved for the black, so there would be some more complicated cases for the AIs to consider. In this year, there were 4 centering openings and 2 uncentering ones. Since there were only 6 engines in this year, the rankings of these AIs became quite clear soon after the tournament started. The only uncertainty was between Pela and XL. While Pela had kept ahead of XL in the first 4 openings, XL won all the last 4 games between them and went to the 5th place. The winner of Renju was Yixin. The second was RenjuSolver, and the third was SlowRenju.

At last, the results of Gomocup 2016 were merged into the previous results to compute the Elo ratings of all the AIs. You can see the Elo rating list.

You can download complete results and openings here.