The Libratus poker app made heads turn in the poker world during 2017 when it had success against poker pros in a one-on-one setting. This time around, a group of researchers at Carnegie Melon University (CMU) have developed a new poker-playing machine, but this time, the bot was able to beat human players in a multi-player setting.
The research team, led by Professor Tuomas Sandholm and Ph.D student Noam Brown, created “Pluribus” in a joint project with Facebook AI. The poker robot played against some of the world’s finest poker players in a six-handed no-limit hold’em game, and was able to defeat them more efficiently than any other previous machines on record. It marks the first time that an AI program has topped elite human players at a game with multiple participants.
Monster Bluffer
Pluribus was tested out in a 12-day session playing over 10,000 hands against 15 top players, including Nick Petrangelo, Greg Merson, Daniel McAulay, Sean Ruane, Jason Les, Linus Loeliger, Anthony Gregg, Seth Davies, Michael Gagliano, Jimmy Chou, Jacob Toole, Trevor Savage and Dong Kim.
Each participant’s real identities were not divulged to the players at the time the experiment was conducted, though they were allowed to use an alias for them to track other player tendencies for the duration of the trial. Should the players win, they would share $50,000 amongst each other, distributed according to individual performance.
The researchers conducted two separate experiments with Pluribus. The first one involved a five-human, one AI format (5H+1AI), while the other used a one-human, five-AI format (1H+5AI) where five copies of Pluribus competed against each other plus one human player. In the second experiment, the bots were unable to engage in collusion or communicate.
In both experiments, Pluribus was the dominant player, having a significantly reliable win rate over the human players. According to Brown’s Facebook AI blog post, the machine was able to beat the top pros to the tune of around $5 per hand, and almost $1,000 hour.
Players Impressed By Pluribus
Les who had a first-hand experience of how good Pluribus is said that the bot is an absolute monster bluffer and played much more efficiently than most human players. The outcome of the experiment will have relevant implications on AI and research on incomplete information.
According to Brown, some of their peers did not believe in the possibility of an AI program defeating multiple players. Brown, Sandholm, and the rest of the team created Pluribus by updating Libratus. The latest poker bot requires much less computing power to play matches. Other AIs coming before Pluribus, such as those bots playing DeepMind’s Go, have shown that the machines rule in two-player games, with one winner and one loser.
While the game theory offers a well-defined strategy, it is less helpful in a multi-player setting where several players compete against each other with no clear win-lose indications. These scenarios reflect most real-life challenges.
CFR Technology
Pluribus was able to come up with the best strategy through self-play. Instead of deriving its strategy based on input from hands played by human players or other AIs, the machine developed its main strategy by playing copies of itself. Pluribus starts from scratch and gradually improves by playing randomly, creating better decisions along the way.
The strategy is based on a form of counterfactual regret minimization (CFR) which has been adopted in several one-on-one competitive games. Pluribus works in accordance with “Monte Carlo CFR” where it was able to explore different actions in a given situation and then make a comparison on which hypothetical options would yield better results according to assumed strategies for each of the other players.
As Pluribus was able to successfully deal with multiplayer poker, the experiment will lay the foundation for future AIs to solve complex problems of this kind.  This will lead to more applications like automated negotiations, self-driving cars, and enhanced fraud detection.
Poker Applications
In their report, the CMU team highlighted two key poker applications from the experiment: Pluribus’ strategy confirms that limping from any position other than the small blind is insignificant. The second application is a strategy called donk betting, which Pluribus was able to execute to its advantage. The machine also utilized several other unconventional strategies involving large bets, in value-betting and semi-bluffing, as well as effective range-merging and trap tactics.
Commenting on the latest development, six-time WSOP winner Chris Ferguson said Pluribus is a very tough opponent to compete against. We will have to wait for some time to see how the applications from research turn out!