Once a week.
Subscribe to our weekly newsletter.
How AI Learned to Bluff and Beat Humans at Poker
AI has not only beat chess, Go, and Jeopardy champions, but now it has defeated some of the world's best poker players. And unlike chess or Go, playing poker involves unknown information like bluffing.
How about a nice game of chess?
The list of recent defeats where humans were overmatched by machines are well-known: chess champion Garry Kasparov losing against IBM's Deep Blue, Jeopardy wiz Ken Jennings being soundly defeated by IBM's Watson, and Go champion Lee Sodol losing to Google's AlphaGo.
We may also be able to add poker to the list of AI superiority.
Professional poker player Jason Les playing against Libratus, an AI program.
A recent twenty-day competition between poker champions (heads-up no-limit Texas hold'em, 120,000 total hands) and Libratus, an AI program created by Carnegie Mellow University professors Tuomas Sandholm and Noam Brown, had the AI coming out on top. This is particularly surprising because unlike games like chess and Go, where the information is upfront and know ("Perfect Information Games"), poker involves a great deal of hidden information ("Imperfect Information Games") and the seemingly-human characteristic of bluffing. It turns out that AI can learn the art of bluffing.
This year, Libratus became the first AI to defeat poker champions in heads-up no-limit Texas hold'em poker.
"It wasn't just a matter of figuring out a strategy versus a static opponent, it ended up changing its strategy as time went on."-Jason Les, professional poker player
Why is Poker So Difficult for AI to Master?
AI benefits from figuring out a strategy based on rules and known information, and poker included a great deal of hidden information. Unlike a chessboard displaying your opponent's chess pieces, your opponent's hand in poker is hidden. Poker has a near-infinite amount of possible situations--10 to the 160th power to be exact. That's greater than the number of atoms in the universe.
Libratus has a great deal of computer power running it, connected to the Pittsburgh Supercomputer Center. Instead of being taught the best way to play poker--which would be relevant for a Perfect Information Game like chess, checkers, or Go--Libratus was taught the rules of poker and then learned through its interactions with the human players. The AI was given a reward function to win as much money as possible and then instructed to optimize the reward function. (The co-creator of Libratus, Professor Noam Brown of Carnegie Mellon, explains how the AI was programmed in a Software Engineering Daily podcast).
Libratus was constructed by first solving an abstraction of the game via a new variant of Monte Carlo CFR that samples negative-regret actions less frequently. Libratus applied nested subgame solving upon reaching the third betting round, and in response to every subsequent opponent bet thereafter. This allowed Libratus to avoid information abstraction during play, and leverage nested subgame solving’s far lower exploitability in response to opponent off-tree actions.-Safe and Nested Subgame Solving for Imperfect-Information Games, Noam Brown and Tuomas Sandholm
In other words, Libratus learned the subtle flaws in the poker champions' play and began capitalizing on it. While the humans-versus-Libratus event was billed as Brains Versus Artificial Intelligence, it may be better to think it as Human Brains versus AI Brains.
AI Can Beat Poker Champions. So What?
Unlike mastering a set of rules--what IBM's Deep Blue did for chess and Google's AlphaGo did for Go--the success of Libratus may indicate a potential future where AI assists humans in tasks involving negotiation and other situations where the available facts are incomplete.
“It is a really critical milestone in developing AIs that can solve real world problems with incomplete information, which are the ones we need to solve to advance society--not just poker.”-Nick Nystrom, Senior Director of Research at the Pittsburgh Supercomputer Center (speaking to Engadget)
Similar to how IBM's Watson went from an expensive parlor trick on Jeopardy to assisting business decisions, today's poker champion can be tomorrow's business engine.
A team of archaeologists has discovered 3,200-year-old cheese after analyzing artifacts found in an ancient Egyptian tomb. It could be the oldest known cheese sample in the world.
A team of archaeologists has discovered 3,200-year-old cheese after analyzing artifacts found in an ancient Egyptian tomb. It could be the oldest known cheese sample in the world.
The tomb that held the cheese lies in the desert sands south of Cairo. It was first discovered in the 19th century by treasure hunters, who eventually lost the knowledge of its location, leaving the Saharan sands to once again conceal the tomb.
“Since 1885 the tomb has been covered in sand and no-one knew about it,” Professor Ola el-Aguizy of Cairo University told the BBC. “It is important because this tomb was the lost tomb.”
In 2010, a team of archaeologists rediscovered the tomb, which belonged to Ptahmes, a mayor and military chief of staff of the Egyptian city of Memphis in the 13th century B.C. In the tomb, the team found a jar containing a “solidified whitish mass,” among other artifacts.
“The archaeologists suspected [the mass] was food, according to the conservation method and the position of the finding inside the tomb, but we discovered it was cheese after the first tests,” Enrico Greco, the lead author of the paper and a research assistant at Peking University in Beijing, told the The New York Times.
To find out what the substance was, the team had to develop a novel way to analyze the proteins and identify the peptide markers in the samples. They first dissolved parts of the substance and then used mass spectrometry and chromatography to analyze its proteins.
Despite more than 3,000 years spent in the desert, the researchers were able to identify hundreds of peptides (chains of amino acids) in the sample. They found some that were associated with milk from goat, sheep and, interestingly, the African buffalo, a species not usually kept as a domestic animal in modern Africa, as Gizmodo reports.
Those results suggested that the substance was cheese, specifically one that was probably similar in consistency to chevre but with a “really, really acidy” taste, as Dr. Paul Kindstedt, a professor at the University of Vermont who studies the chemistry and history of cheese, told the The New York Times.
“It would be high in moisture; it would be spreadable,” he said. “It would not last long; it would spoil very quickly.”
The researchers also found traces of the bacterium Brucella melitensis, which causes brucellosis, a debilitating disease that can cause endocarditis, arthritis, chronic fatigue, malaise, muscle pain and other conditions. It’s a disease usually contracted by consuming raw dairy products.
“The most common way to be infected [with Brucella melitensis] is by eating or drinking unpasteurized/raw dairy products. When sheep, goats, cows, or camels are infected, their milk becomes contaminated with the bacteria,” the U.S. Centers for Disease Control wrote on its website. “If the milk from infected animals is not pasteurized, the infection will be transmitted to people who consume the milk and/or cheese products.”
Dr. Kindstedt said one reason the study is significant is for its novel use of proteomic analysis, which is the systematic identification and quantification of the complete complement of proteins (the proteome) of a biological system.
“As I say to my students every year when I get to Egypt, someone has to go ahead and analyze these residues with modern capabilities,” he told the The New York Times. “This is a logical next step and I think you’re going to see a lot more of this.”
'The Great Pyramid of Chee-za'. An artist's interpretation of a very ripe, slightly deadly Egyptian tomb cheese. (Credit: Creative commons/Big Think)
However, Dr. Kindstedt did offer a bit of caution on the conclusions the researchers drew from the findings.
“The authors of this new study did some nice work,” he told Gizmodo in a statement. “But in my view, on multiple grounds (I suspect in their zeal to be “the first”), they inferred considerably beyond what their data is capable of supporting within reasonable certainty, and almost certainly they are not the first to have found solid cheese residues in Egyptian tombs, just the first to apply proteomic analyses (which is worthy achievement on its own).”
As bad as this sounds, a new essay suggests that we live in a surprisingly egalitarian age.
- A new essay depicts 700 years of economic inequality in Europe.
- The only stretch of time more egalitarian than today was the period between 1350 to approximately the year 1700.
- Data suggest that, without intervention, inequality does not decrease on its own.
Economic inequality is a constant topic. No matter the cycle — boom or bust — somebody is making a lot of money, and the question of fairness is never far behind.
A recently published essay in the Journal of Economic Literature by Professor Guido Alfani adds an intriguing perspective to the discussion by showing the evolution of income inequality in Europe over the last several hundred years. As it turns out, we currently live in a comparatively egalitarian epoch.
Seven centuries of economic history
Figure 8 from Guido Alfani, Journal of Economic Literature, 2021.
This graph shows the amount of wealth controlled by the top ten percent in certain parts of Europe over the last seven hundred years. Archival documentation similar to — and often of a similar quality as — modern economic data allows researchers to get a glimpse of what economic conditions were like centuries ago. Sources like property tax records and documents listing the rental value of homes can be used to determine how much a person's estate was worth. (While these methods leave out those without property, the data is not particularly distorted.)
The first part of the line, shown in black, represents work by Prof. Alfani and represents the average inequality level of the Sabaudian State in Northern Italy, The Florentine State, The Kingdom of Naples, and the Republic of Venice. The latter part, in gray, is based on the work of French economist Thomas Piketty and represents an average of inequality in France, the United Kingdom, and Sweden during that time period.
Despite the shift in location, the level of inequality and rate of increase are very similar between the two data sets.
Apocalyptic events cause decreases in inequality
Note that there are two substantial declines in inequality. Both are tied to truly apocalyptic events. The first is the Black Death, the common name for the bubonic plague pandemic in the 14th century, which killed off anywhere between 30 and 50 percent of Europe. The second, at the dawn of the 20th century, was the result of World War I and the many major events in its aftermath.
The 20th century as a whole was a time of tremendous economic change, and the periods not featuring major wars are notable for having large experiments in distributive economic policies, particularly in the countries Piketty considers.
The slight stall in the rise of inequality during the 17th century is the result of the Thirty Years' War, a terrible religious conflict that ravaged Europe and left eight million people dead, and of major plagues that affected South Europe. However, the recurrent outbreaks of the plague after the Black Death no longer had much effect on inequality. This was due to a number of factors, not the least of which was the adaptation of European institutions to handle pandemics without causing such a shift in wealth.
In 2010, the last year covered by the essay, inequality levels were similar to those of 1340, with 66 percent of the wealth of society being held by the top ten percent. Also, inequality levels were continuing to rise, and the trends have not ended since. As Prof. Alfani explained in an email to BigThink:
"During the decade preceding the Covid pandemic, economic inequality has shown a slow tendency towards further inequality growth. The Great Recession that began in 2008 possibly contributed to slow down inequality growth, especially in Europe, but it did not stop it. However, the expectation is that Covid-19 will tend to increase inequality and poverty. This, because it tends to create a relatively greater economic damage to those having unstable occupations, or who need physical strength to work (think of the effects of the so-called "long-Covid," which can prove physically invalidating for a long time). Additionally, and thankfully, Covid is not lethal enough to force major leveling dynamics upon society."
Can only disasters change inequality?
That is the subject of some debate. While inequality can occur in any economy, even one that doesn't grow all that much, some things appear to make it more likely to rise or fall.
Thomas Piketty suggested that the cause of changes in inequality levels is the difference in the rate of return on capital and the overall growth rate of the economy. Since the return on capital is typically higher than the overall growth rate, this means that those who have capital to invest tend to get richer faster than everybody else.
While this does explain a great deal of the graph after 1800, his model fails to explain why inequality fell after the Black Death. Indeed, since the plague destroyed human capital and left material goods alone, we would expect the ratio of wealth over income to increase and for inequality to rise. His model can provide explanations for the decline in inequality in the decades after the pandemic, however- it is possible that the abundance of capital could have lowered returns over a longer time span.
The catastrophe theory put forth by Walter Scheidel suggests that the only force strong enough to wrest economic power from those who have it is a world-shattering event like the Black Death, the fall of the Roman Empire, or World War I. While each event changed the world in a different way, they all had a tremendous leveling effect on society.
But not even this explains everything in the above graph. Pandemics subsequent to the Black Death had little effect on inequality, and inequality continued to fall for decades after World War II ended. Prof. Alfani suggests that we remember the importance of human agency through institutional change. He attributes much of the post-WWII decline in inequality to "the redistributive policies and the development of the welfare states from the 1950s to the early 1970s."
What does this mean for us now?
As Professor Alfani put it in his email:
"[H]istory does not necessarily teach us whether we should consider the current trend toward growth in economic inequality as an undesirable outcome or a problem per se (although I personally believe that there is some ground to argue for that). Nor does it teach us that high inequality is destiny. What it does teach us, is that if we do not act, we have no reason whatsoever to expect that inequality will, one day, decline on its own. History also offers abundant evidence that past trends in inequality have been deeply influenced by our collective decisions, as they shaped the institutional framework across time. So, it is really up to us to decide whether we want to live in a more, or a less unequal society."
Our love-hate relationship with browser tabs drives all of us crazy. There is a solution.
- A new study suggests that tabs can cause people to be flustered as they try to keep track of every website.
- The reason is that tabs are unable to properly organize information.
- The researchers are plugging a browser extension that aims to fix the problem.
A lot of ideas that people had about the internet in the 1990s have fallen by the wayside as technology and our usage patterns evolved. Long gone are things like GeoCities, BowieNet, and the belief that letting anybody post whatever they are thinking whenever they want is a fundamentally good idea with no societal repercussions.
While these ideas have been abandoned and the tools that made them possible often replaced by new and improved ones, not every outdated part of our internet experience is gone. A new study by a team at Carnegie Mellon makes the case that the use of tabs in a web browser is one of these outdated concepts that we would do well to get rid of.
How many tabs do you have open right now?
We didn't always have tabs. Introduced in the early 2000s, tabs are now included on all major web browsers, and most users have had access to them for a little over a decade. They've been pretty much the same since they came out, despite the ever changing nature of the internet. So, in this new study, researchers interviewed and surveyed 113 people on their use of — and feelings toward — the ubiquitous tabs.
Most people use tabs for the short-term storage of information, particularly if it's information that is needed again soon. Some keep tabs that they know they'll never get around to reading. Others used them as a sort of external memory bank. One participant described this action to the researchers:
"It's like a manifestation of everything that's on my mind right now. Or the things that should be on my mind right now... So right now, in this browser window, I have a web project that I'm working on. I don't have time to work on it right now, but I know I need to work on it. So it's sitting there reminding me that I need to work on it."
You suffer from tab overload
Unfortunately, trying to use tabs this way can cause a number of problems. A quarter of the interview subjects reported having caused a computer or browser to crash because they had too many tabs open. Others reported feeling flustered by having so many tabs open — a situation called "tab overload" — or feeling ashamed that they appeared disorganized by having so many tabs up at once. More than half of participants reported having problems like this at least two or three times a week.
However, people can become emotionally invested in the tabs. One participant explained, "[E]ven when I'm not using those tabs, I don't want to close them. Maybe it's because it took efforts [sic] to open those tabs and organize them in that way."
So, we have a tool that inefficiently saves web pages that we might visit again while simultaneously reducing our productivity, increasing our anxiety, and crashing our machines. And yet we feel oddly attached to them.
Either the system is crazy or we are.
Skeema: The anti-tab revolution
The researchers concluded that at least part of the problem is caused by tabs not being an ideal way of organizing the work we now do online. They propose a new model that better compartmentalizes tabs by task and subtask, reflects users' mental models, and helps manage the users' attention on what is important right now rather than what might be important later.
To that end, the team also created Skeema, an extension for Google Chrome, that treats tabs as tasks and offers a variety of ways to organize them. Users of an early version reported having fewer tabs and windows open at one time and were better able to manage the information they contained.
Tabs were an improvement over having multiple windows open at the same time, but they may have outlived their usefulness. While it might take a paradigm shift to fully replace the concept, the study suggests that taking a different approach to tabs might be worth trying.
And now, excuse me, while I close some of the 87 tabs I currently have open.