Once a week.
Subscribe to our weekly newsletter.
Google's AI Learns Betrayal and "Aggressive" Actions Pay Off
Google's DeepMind artificial intelligence learns what it takes to win, making human-like choices in competitive situations.
As the development of artificial intelligence continues at breakneck speed, questions about whether we understand what we are getting ourselves into persist. One fear is that increasingly intelligent robots will take all our jobs. Another fear is that we will create a world where a superintelligence will one day decide that it has no need for humans. This fear is well-explored in popular culture, through books and films like the Terminator series.
Another possibility is maybe the one that makes the most sense - since humans are the ones creating them, the machines and machine intelligences are likely to behave just like humans. For better or worse. DeepMind, Google’s cutting-edge AI company, has shown just that.
Recently, the DeepMind team ran a series of tests to investigate how the AI would respond when faced with certain social dilemmas. In particular, they wanted to find out whether the AI is more likely to cooperate or compete.
One of the tests involved 40 million instances of playing the computer game Gathering, during which DeepMind showed how far it’s willing to go to get what it wants. The game was chosen because it encapsulates aspects of the classic “Prisoner’s Dilemma” from game theory.
Pitting AI-controlled characters (called “agents”) against each other, DeepMind had them compete to gather the most virtual apples. Once the amount of available apples got low, the AI agents started to display "highly aggressive" tactics, employing laser beams to knock each other out. They would also steal the opponent’s apples.
Here’s how one of those games played out:
The DeepMind AI agents are in blue and red. The apples are green, while the laser beams are yellow.
The DeepMind team described their test in a blog post this way:
“We let the agents play this game many thousands of times and let them learn how to behave rationally using deep multi-agent reinforcement learning. Rather naturally, when there are enough apples in the environment, the agents learn to peacefully coexist and collect as many apples as they can. However, as the number of apples is reduced, the agents learn that it may be better for them to tag the other agent to give themselves time on their own to collect the scarce apples.”
Interestingly, what appears to have happened is that the AI systems began to develop some forms of human behavior.
“This model... shows that some aspects of human-like behaviour emerge as a product of the environment and learning. Less aggressive policies emerge from learning in relatively abundant environments with less possibility for costly action. The greed motivation reflects the temptation to take out a rival and collect all the apples oneself,” said Joel Z. Leibo from the DeepMind team to Wired.
Besides the fruit gathering, the AI was also tested via a Wolfpack hunting game. In it, two AI characters in the form of wolves chased a third AI agent - the prey. Here the researchers wanted to see if the AI characters would choose to cooperate to get the prey because they were rewarded for appearing near the prey together when it was being captured.
"The idea is that the prey is dangerous - a lone wolf can overcome it, but is at risk of losing the carcass to scavengers. However, when the two wolves capture the prey together, they can better protect the carcass from scavengers, and hence receive a higher reward,” wrote the researchers in their paper.
Indeed, the incentivized cooperation strategy won out in this instance, with the AI choosing to work together.
This is how that test panned out:
The wolves are red, chasing the blue dot (prey), while avoiding grey obstacles.
If you are thinking “Skynet is here”, perhaps the silver lining is that the second test shows how AI’s self-interest can include cooperation rather than the all-out competitiveness of the first test. Unless, of course, its cooperation to hunt down humans.
Here's a chart showing the results of the game tests that shows a clear increase in aggression during "Gathering":
Movies aside, the researchers are working to figure out how AI can eventually “control complex multi-agent systems such as the economy, traffic systems, or the ecological health of our planet – all of which depend on our continued cooperation”.
One nearby AI implementation where this could be relevant - self-driving cars which will have to choose safest routes, while keeping the objectives of all the parties involved under consideration.
The warning from the tests is that if the objectives are not balanced out in the programming, the AI might act selfishly, probably not for everyone’s benefit.
What’s next for the DeepMind team? Joel Leibo wants the AI to go deeper into the motivations behind decision-making:
“Going forward it would be interesting to equip agents with the ability to reason about other agent’s beliefs and goals,” said Leibo to Bloomberg.
All this from a wad of gum?
- Researchers recently uncovered a piece of chewed-on birch pitch in an archaeological dig in Denmark.
- Conducting a genetic analysis of the material left in the birch pitch offered a plethora of insights into the individual who last chewed it.
- The gum-chewer has been dubbed Lola. She lived 5,700 years ago; and she had dark skin, dark hair, and blue eyes.
Five thousand and seven hundred years ago, "Lola" — a blue-eyed woman with dark skin and hair — was chewing on a piece of pitch derived from heating birch bark. Then, this women spit her chewing gum out into the mud on an island in Denmark that we call Syltholm today, where it was unearthed by archaeologists thousands of years later. A genetic analysis of the chewing gum has provided us with a wealth of information on this nearly six-thousand-year-old Violet Beauregarde.
This represents the first time that the human genome has been extracted from material such as this. "It is amazing to have gotten a complete ancient human genome from anything other than bone," said lead researcher Hannes Schroeder in a statement.
"What is more," he added, "we also retrieved DNA from oral microbes and several important human pathogens, which makes this a very valuable source of ancient DNA, especially for time periods where we have no human remains."
In the pitch, researchers identified the DNA of the Epstein-Barr virus, which infects about 90 percent of adults. They also found DNA belonging to hazelnuts and mallards, which were likely the most recent meal that Lola had eaten before spitting out her chewing gum.
Insights into ancient peoples
The birch pitch was found on the island of Lolland (the inspiration for Lola's name) at a site called Syltholm. "Syltholm is completely unique," said Theis Jensen, who worked on the study for his PhD. "Almost everything is sealed in mud, which means that the preservation of organic remains is absolutely phenomenal.
"It is the biggest Stone Age site in Denmark and the archaeological finds suggest that the people who occupied the site were heavily exploiting wild resources well into the Neolithic, which is the period when farming and domesticated animals were first introduced into southern Scandinavia."
Since Lola's genome doesn't show any of the markers associated with the agricultural populations that had begun to appear in this region around her time, she provides evidence for a growing idea that hunter-gatherers persisted alongside agricultural communities in northern Europe longer than previously thought.
Her genome supports additional theories on northern European peoples. For example, her dark skin bolsters the idea that northern populations only recently acquired their light-skinned adaptation to the low sunlight in the winter months. She was also lactose intolerant, which researchers believe was the norm for most humans prior to the agricultural revolution. Most mammals lose their tolerance for lactose once they've weaned off of their mother's milk, but once humans began keeping cows, goats, and other dairy animals, their tolerance for lactose persisted into adulthood. As a descendent of hunter-gatherers, Lola wouldn't have needed this adaptation.
A hardworking piece of gum
A photo of the birch pitch used as chewing gum.
These findings are encouraging for researchers focusing on ancient peoples from this part of the world. Before this study, ancient genomes were really only ever recovered from human remains, but now, scientists have another tool in their kit. Birch pitch is commonly found in archaeological sites, often with tooth imprints.
Ancient peoples used and chewed on birch pitch for a variety of reasons. It was commonly heated up to make it pliable, enabling it to be molded as an adhesive or hafting agent before it settled. Chewing the pitch may have kept it pliable as it cooled down. It also contains a natural antiseptic, and so chewing birch pitch may have been a folk medicine for dental issues. And, considering that we chew gum today for no other reason than to pass the time, it may be that ancient peoples chewed pitch for fun.
Whatever their reasons, chewed and discarded pieces of birch pitch offer us the mind-boggling option of learning what someone several thousands of years ago ate for lunch, or what the color of their hair was, their health, where their ancestors came from, and more. It's an unlikely treasure trove of information to be found in a mere piece of gum.
The non-contact technique could someday be used to lift much heavier objects — maybe even humans.
- Since the 1980s, researchers have been using sound waves to move matter through a technique called acoustic trapping.
- Acoustic trapping devices move bits of matter by emitting strategically designed sound waves, which interact in such a way that the matter becomes "trapped" in areas of particular velocity and pressure.
- Acoustic and optical trapping devices are already used in various fields, including medicine, nanotechnology, and biological research.
Sound can have powerful effects on matter. After all, sound strikes our world in waves — vibrations of air molecules that bounce off of, get absorbed by, or pass through matter around us. Sound waves from a trained opera singer can shatter a wine glass. From a jet, they can collapse a stone wall. But sound can also be harnessed for delicate interactions with matter.
Since the 1980s, researchers have been using sound to move matter through a phenomenon called acoustic trapping. The method is based on the fact that sound waves produce an acoustic radiation force.
"When an acoustic wave interacts with a particle, it exerts both an oscillatory force and a much smaller steady-state 'radiation' force," wrote the American Physical Society. "This latter force is the one used for trapping and manipulation. Radiation forces are generated by the scattering of a traveling sound wave, or by energy gradients within the sound field."
When tiny particles encounter this radiation, they tend to be drawn toward regions of certain pressure and velocity within the sound field. Researchers can exploit this tendency by engineering sound waves that "trap" — or suspend — tiny particles in the air. Devices that do this are often called "acoustic tweezers."
Building a better tweezer
A study recently published in the Japanese Journal of Applied Physics describes how researchers created a new type of acoustic tweezer that was able to lift a small polystyrene ball into the air.
Tweezers of Sound: Acoustic Manipulation off a Reflective Surface youtu.be
It is not the first example of a successful "acoustic tweezer" device, but the new method is likely the first to overcome a common problem in acoustic trapping: sound waves bouncing off reflective surfaces, which disrupts acoustic traps.
To minimize the problems of reflectivity, the team behind the recent study configured ultrasonic transducers such that the sound waves that they produce overlap in a strategic way that is able to lift a small bit of polystyrene from a reflective surface. By changing how the transducers emit sound waves, the team can move the acoustic trap through space, which moves the bit of matter.
Move, but don't touch
So far, the device is only able to move millimeter-sized pieces of matter with varying degrees of success. "When we move a particle, it sometimes scatters away," the team noted. Still, improved acoustic trapping and other no-contact lifting technologies — like optical tweezers, commonly used in medicine — could prove useful in many future applications, including cell separation, nanotechnologies, and biological research.
Could future acoustic-trapping devices lift large and heavy objects, maybe even humans? It seems possible. In 2018, researchers from the University of Bristol managed to acoustically trap particles whose diameters were larger than the sound wavelength, which was a breakthrough because it surpassed "the classical Rayleigh scattering limit that has previously restricted stable acoustic particle trapping," the researchers wrote in their study.
In other words, the technique — which involved suspending matter in tornado-like acoustic traps — showed that it is possible to scale up acoustic trapping.
"Acoustic tractor beams have huge potential in many applications," Bruce Drinkwater, co-author of the 2018 study, said in a statement. "I'm particularly excited by the idea of contactless production lines where delicate objects are assembled without touching them."
Australian parrots have worked out how to open trash bins, and the trick is spreading across Sydney.
- If sharing learned knowledge is a form of culture, Australian cockatoos are one cultured bunch of birds.
- A cockatoo trick for opening trash bins to get at food has been spreading rapidly through Sydney's neighborhoods.
- But not all cockatoos open the bins; some just stay close to those that do.
Dumpster-diving trash parrots
In a study about these smart birds just published in Science, researchers define animal culture as "population-specific behaviors acquired via social learning from knowledgeable individuals."
Co-lead author of the study Barbara Klump of the Max Planck Institute of Animal Behavior in Konstanz, Germany says, "[C]ompared to humans, there are few known examples of animals learning from each other. Demonstrating that food scavenging behavior is not due to genetics is a challenge."
An opportunity presented itself in a video that co-author Richard Major of the Australian Museum shared with Klump and the other co-authors. In the video, a sulphur-crested cockatoo used its beak to pull up the handle of a closed garbage bin — using its foot as a wedge — and then walked back the lid sufficiently to flip it open, exposing the bin's edible contents.
Major has been studying Cacatua galerita for 20 years and says, "Like many Australian birds, sulphur-crested cockatoos are loud and aggressive." The study describes them as a "large-brained, long-lived, and highly social parrot." Says Major, "They are also incredibly smart, persistent, and have adapted brilliantly to living with humans."(Research regarding some of the ways in which wild animals adapt to the presence of humans has already produced some fascinating results and is ongoing.)
Clever cockie opens bin - 01 youtu.be
The researchers became curious about how widespread this behavior might be and saw a research opportunity. After all, says John Martin, a researcher at Taronga Conservation Society, "Australian garbage bins have a uniform design across the country, and sulphur-crested cockatoos are common across the entire east coast."
Martin continues, "In 2018, we launched an online survey in various areas across Sydney and Australia with questions such as, 'What area are you from, have you seen this behavior before, and if so, when?'"
Word gets around
Credit: magspace/Adobe Stock
Although the cockatoos' maneuver was reported in only three suburbs before 2018, by the end of 2019, people in 44 areas reported observing the behavior. Clearly, more and more cockatoos were learning how to successfully dumpster dive.
As further proof, says Klump, "We observed that the birds do not open the garbage bins in the same way, but rather used different opening techniques in different suburbs, suggesting that the behavior is learned by observing others." One individual bird in north Sydney invented its own method, and the scientists saw it grow in popularity throughout the local population.
To track individual birds, the researchers marked 500 cockatoos with small red dots. Subsequent observations revealed that not all cockatoos are bin-openers. Only about 10 percent of them are, and they are mostly males. The other cockatoos apparently restrict their education to a different lesson: hang around with a bin-opener, and you will get supper.
Thanks to the surveys, the researchers consider the entire project to be a valuable citizen-science experiment. "By studying this behavior with the help of local residents, we are uncovering the unique and complex cultures of their neighborhood birds."
The few seconds of nuclear explosion opening shots in Godzilla alone required more than 6.5 times the entire budget of the monster movie they ended up in.