from the world's big
How AI Learned to Bluff and Beat Humans at Poker
AI has not only beat chess, Go, and Jeopardy champions, but now it has defeated some of the world's best poker players. And unlike chess or Go, playing poker involves unknown information like bluffing.
How about a nice game of chess?
The list of recent defeats where humans were overmatched by machines are well-known: chess champion Garry Kasparov losing against IBM's Deep Blue, Jeopardy wiz Ken Jennings being soundly defeated by IBM's Watson, and Go champion Lee Sodol losing to Google's AlphaGo.
We may also be able to add poker to the list of AI superiority.
Professional poker player Jason Les playing against Libratus, an AI program.
A recent twenty-day competition between poker champions (heads-up no-limit Texas hold'em, 120,000 total hands) and Libratus, an AI program created by Carnegie Mellow University professors Tuomas Sandholm and Noam Brown, had the AI coming out on top. This is particularly surprising because unlike games like chess and Go, where the information is upfront and know ("Perfect Information Games"), poker involves a great deal of hidden information ("Imperfect Information Games") and the seemingly-human characteristic of bluffing. It turns out that AI can learn the art of bluffing.
This year, Libratus became the first AI to defeat poker champions in heads-up no-limit Texas hold'em poker.
"It wasn't just a matter of figuring out a strategy versus a static opponent, it ended up changing its strategy as time went on."-Jason Les, professional poker player
Why is Poker So Difficult for AI to Master?
AI benefits from figuring out a strategy based on rules and known information, and poker included a great deal of hidden information. Unlike a chessboard displaying your opponent's chess pieces, your opponent's hand in poker is hidden. Poker has a near-infinite amount of possible situations--10 to the 160th power to be exact. That's greater than the number of atoms in the universe.
Libratus has a great deal of computer power running it, connected to the Pittsburgh Supercomputer Center. Instead of being taught the best way to play poker--which would be relevant for a Perfect Information Game like chess, checkers, or Go--Libratus was taught the rules of poker and then learned through its interactions with the human players. The AI was given a reward function to win as much money as possible and then instructed to optimize the reward function. (The co-creator of Libratus, Professor Noam Brown of Carnegie Mellon, explains how the AI was programmed in a Software Engineering Daily podcast).
Libratus was constructed by first solving an abstraction of the game via a new variant of Monte Carlo CFR that samples negative-regret actions less frequently. Libratus applied nested subgame solving upon reaching the third betting round, and in response to every subsequent opponent bet thereafter. This allowed Libratus to avoid information abstraction during play, and leverage nested subgame solving’s far lower exploitability in response to opponent off-tree actions.-Safe and Nested Subgame Solving for Imperfect-Information Games, Noam Brown and Tuomas Sandholm
In other words, Libratus learned the subtle flaws in the poker champions' play and began capitalizing on it. While the humans-versus-Libratus event was billed as Brains Versus Artificial Intelligence, it may be better to think it as Human Brains versus AI Brains.
AI Can Beat Poker Champions. So What?
Unlike mastering a set of rules--what IBM's Deep Blue did for chess and Google's AlphaGo did for Go--the success of Libratus may indicate a potential future where AI assists humans in tasks involving negotiation and other situations where the available facts are incomplete.
“It is a really critical milestone in developing AIs that can solve real world problems with incomplete information, which are the ones we need to solve to advance society--not just poker.”-Nick Nystrom, Senior Director of Research at the Pittsburgh Supercomputer Center (speaking to Engadget)
Similar to how IBM's Watson went from an expensive parlor trick on Jeopardy to assisting business decisions, today's poker champion can be tomorrow's business engine.
Andy Samberg and Cristin Milioti get stuck in an infinite wedding time loop.
- Two wedding guests discover they're trapped in an infinite time loop, waking up in Palm Springs over and over and over.
- As the reality of their situation sets in, Nyles and Sarah decide to enjoy the repetitive awakenings.
- The film is perfectly timed for a world sheltering at home during a pandemic.
China moves to Russia and India takes over Canada. The Swiss get Bangladesh, the Bangladeshi India. And the U.S.? It stays where it is.
What if the world were rearranged so that the inhabitants of the country with the largest population would move to the country with the largest area? And the second-largest population would migrate to the second-largest country, and so on?
A recent analysis of a 76-million-year-old Centrosaurus apertus fibula confirmed that dinosaurs suffered from cancer, too.
- The fibula was originally discovered in 1989, though at the time scientists believed the damaged bone had been fractured.
- After reanalyzing the bone, and comparing it with fibulas from a human and another dinosaur, a team of scientists confirmed that the dinosaur suffered from the bone cancer osteosarcoma.
- The study shows how modern techniques can help scientists learn about the ancient origins of diseases.
Centrosaurus apertus fibula
Royal Ontario Museum<p>In the recent study, the team used a combination of techniques to analyze the fibula, including taking CT scans, casting the bone and studying thin slices of it under a microscope. The analysis suggested that the dinosaur likely suffered from osteosarcoma, a type of bone cancer that affects modern humans, typically young adults.</p><p>For further evidence, the team compared the damaged fibula to a healthy fibula from a dinosaur of the same species, and also to a fibula that belonged to a 19-year-old human who suffered from osteosarcoma. Both comparisons supported the osteosarcoma diagnosis.</p>
Evans et al.<p style="margin-left: 20px;">"The shin bone shows aggressive cancer at an advanced stage," Evans said in a <a href="https://www.rom.on.ca/en/about-us/newsroom/press-releases/rare-malignant-cancer-diagnosed-in-a-dinosaur" target="_blank">press release</a>. "The cancer would have had crippling effects on the individual and made it very vulnerable to the formidable tyrannosaur predators of the time."</p><p style="margin-left: 20px;">"The fact that this plant-eating dinosaur lived in a large, protective herd may have allowed it to survive longer than it normally would have with such a devastating disease."</p><p>The fossilized fibula was originally unearthed in a bonebed alongside the remains of dozens of other <em>Centrosaurus </em><em>apertus</em>, suggesting the dinosaur didn't die from cancer, but from a flood that swept it away with its herd.</p>
Dinosaur fibula; the tumor mass is depicted in yellow.
Royal Ontario Museum/McMaster University<p>The new study highlights how modern techniques can help scientists learn more about the evolutionary origins of modern diseases, like cancer. It also shows that dinosaurs suffered through some of the same terrestrial afflictions humans face today.</p><p style="margin-left: 20px;">"Dinosaurs can seem like mythical creatures, but they were living, breathing animals that suffered through horrible injuries and diseases," Evans said, "and this discovery certainly makes them more real and helps bring them to life in that respect."</p>
Join the lauded author of Range in conversation with best-selling author and poker pro Maria Konnikova!
UPDATE: Unfortunately, Malcolm Gladwell was not able to make the live stream due to scheduling issues. Fortunately, David Epstein was able to jump in at a moment's notice. We hope you enjoy this great yet unexpected episode of Big Think Live. Our thanks to David and Maria for helping us deliver a show, it is much appreciated.