- President Donald Trump wants social media companies to develop A.I. that can flag potential mass shooters.
- Experts agree that artificial intelligence is not advanced enough, nor are current moderating systems up to the task.
- A majority of Americans support stricter gun laws, but such policies have yet to make headway.
On August 3, a man in El Paso, Texas, shot and killed 22 people and injured 24 others. Hours later, another man in Dayton, Ohio, shot and killed nine people, including his own sister. Even in a country left numb by countless mass shootings, the news was distressing and painful.
President Donald Trump soon addressed the nation to outline how his administration planned to tackle this uniquely American problem. Listeners hoping the tragedies might finally spur motivation for stricter gun control laws, such as universal background checks or restrictions on high-capacity magazines, were left disappointed.
Trump's plan was a ragbag of typical Republican talking points: red flag laws, mental health concerns, and regulation on violent video games. Tucked among them was an idea straight out of a Philip K. Dick novel.
"We must recognize that the internet has provided a dangerous avenue to radicalize disturbed minds and perform demented acts," Trump said. "First, we must do a better job of identifying and acting on early warning signs. I am directing the Department of Justice to work in partnership with local, state and federal agencies as well as well as social media companies to develop tools that can detect mass shooters before they strike."
Basically, Trump wants digital precogs. But has artificial intelligence reached such grand, and potentially terrifying, heights?
A digitized state of mind
It's worth noting that A.I. has made impressive strides at reading and quantifying the human mind. Social media is a vast repository of data on how people feel and think. If we can suss out the internal from the performative, we could improve mental health care in the U.S. and abroad.
For example, a study from 2017 found that A.I. could read the predictive markers for depression in Instagram photos. Researchers tasked machine learning tools with analyzing data from 166 individuals, some of whom had been previously diagnosed with depression. The algorithms looked at filter choice, facial expressions, metadata tags, etc., in more than 43,950 photos.
The results? The A.I. outperformed human practitioners at diagnosing depression. These results held even when analyzing images from before the patients' diagnoses. (Of course, Instagram is also the social media platform most likely to make you depressed and anxious, but that's another study.)
Talking with Big Think, Eric Topol, a professor in the Department of Molecular Medicine at Scripps, called this the ability to "digitize our state of mind." In addition to the Instagram study, he pointed out that patients will share more with a self-chosen avatar than a human psychiatrist.
"So when you take this ability to digitize a state of mind and also have a support through an avatar, this could turn out to be a really great way to deal with the problem we have today, which is a lack of mental health professionals with a very extensive burden of depression and other mental health conditions," Topol said.
Detecting mass shooters?
....mentally ill or deranged people. I am the biggest Second Amendment person there is, but we all must work togeth… https://t.co/T9OthUAsXe— Donald J. Trump (@Donald J. Trump)1565352202.0
However, it's not as simple as turning the A.I. dial from "depression" to "mass shooter." Machine learning tools have gotten excellent at analyzing images, but they lag behind the mind's ability to read language, intonation, and social cues.
As Facebook CEO Mark Zuckerberg said: "One of the pieces of criticism we get that I think is fair is that we're much better able to enforce our nudity policies, for example, than we are hate speech. The reason for that is it's much easier to make an A.I. system that can detect a nipple than it is to determine what is linguistically hate speech."
Trump should know this. During a House Homeland Security subcommittee hearing earlier this year, experts testified that A.I. was not a panacea for curing online extremism. Alex Stamos, Facebook's former chief security officer, likened the world's best A.I. to "a crowd of millions of preschoolers" and the task to demanding those preschoolers "get together to build the Taj Mahal."
None of this is to say that the problem is impossible, but it's certainly intractable.
Yes, we can create an A.I. that plays Go or analyzes stock performance better than any human. That's because we have a lot of data on these activities and they follow predictable input-output patterns. Yet even these "simple" algorithms require some of the brightest minds to develop.
Mass shooters, though far too common in the United States, are still rare. We've played more games of Go, analyzed more stocks, and diagnosed more people with depression, which millions of Americans struggle with. This gives machine learning software more data points on these activities in order to create accurate, responsible predictions — that still don't perform flawlessly.
Add to this that hate, extremism, and violence don't follow reliable input-output patterns, and you can see why experts are leery of Trump's direction to employ A.I. in the battle against terrorism.
"As we psychological scientists have said repeatedly, the overwhelming majority of people with mental illness are not violent. And there is no single personality profile that can reliably predict who will resort to gun violence," Arthur C. Evans, CEO of the American Psychological Association, said in a release. "Based on the research, we know only that a history of violence is the single best predictor of who will commit future violence. And access to more guns, and deadlier guns, means more lives lost."
Social media can't protect us from ourselves
First Lady Melania Trump visits with the victims of the El Paso, Texas, shooting. Image source: Andrea Hanks / Flickr
One may wonder if we can utilize current capabilities more aggressively? Unfortunately, social media moderating systems are a hodgepodge, built piecemeal over the last decade. They rely on a mixture of A.I., paid moderators, and community policing. The outcome is an inconsistent system.
For example, the New York Times reported in 2017 that YouTube had removed thousands of videos using machine learning systems. The videos showed atrocities from the Syrian War, such as executions and people spouting Islamic State propaganda. The algorithm flagged and removed them as coming from extremist groups.
In truth, the videos came from humanitarian organizations to document human rights violations. The machine couldn't tell the difference. YouTube reinstated some of the videos after users reported the issue, but mistakes at such a scale do not give one hope that today's moderating systems could accurately identify would-be mass shooters.
That's the conclusion reached in a report from the Partnership on A.I. (PAI). It argued there were "serious shortcomings" in using A.I. as a risk-assessment tool in U.S. criminal justice. Its writers cite three overarching concerns: accuracy and bias; questions of transparency and accountability; and issues with the interface between tools and people.
"Although the use of these tools is in part motivated by the desire to mitigate existing human fallibility in the criminal justice system, it is a serious misunderstanding to view tools as objective or neutral simply because they are based on data," the report states. "While formulas and statistical models provide some degree of consistency and replicability, they still share or amplify many weaknesses of human decision-making."
In addition to the above, there are practical barriers. The technical capabilities of law enforcement vary between locations. Social media platforms deal in massive amounts of traffic and data. And even when the red flags are self-evident — such as when shooters publish manifestos — they offer a narrow window in which to act.
The tools to reduce mass shootings
Protesters at March for Our Lives 2018 in San Francisco. Image source: Gregory Varnum / Wikimedia Commons
Artificial intelligence offers many advantages today and will offer more in the future. But as an answer to extremism and mass shootings, experts agree it's simply the wrong tool. That's the bad news. The good news is we have the tools we need already, and they can be implemented with readily available tech.
"Based on the psychological science, we know some of the steps we need to take. We need to limit civilians' access to assault weapons and high-capacity magazines. We need to institute universal background checks. And we should institute red flag laws that remove guns from people who are at high risk of committing violent acts," Evans wrote.
Evans isn't alone. Experts agree that the policies he suggests, and a few others, will reduce the likelihood of mass shootings. And six in 10 Americans already support these measures.
We don't need advanced A.I. to figure this out. There's only one developed country in the world where someone can legally and easily acquire an armory of guns, and it's the only developed country that suffers mass shootings with such regularity. It's a simple arithmetic.
- From AI to Mass Shootings, Neuroscience Is the Future of Problem ... ›
- Can A.I. create more diverse workplaces? - Big Think ›
Airspeeder, the world's newest motorsport, is set to debut its first race in 2021.
What can you expect to see? Something like a mix between Red Bull's air racing and the pod-racing scenes from "Star Wars: The Phantom Menace" — manned electric cars flying close together in the desert at 120 mph, nose-diving off cliffs, and racing over lakes, all while hopefully avoiding collisions.
Airspeeder calls its vehicles flying electric cars, but it's probably easier to think of the wheelless multicopters as car-sized drones. Powered by electric batteries, the carbon-fiber craft use eight propellers to fly, and the tiltable motors are designed to allow pilots to navigate through the course's pylons at high speeds.
Credit: Airspeeder
To prevent crashes, Airspeeder is working with the companies Acronis and Teknov8 to develop "high-speed collision avoidance" systems for its Speeders.
"As they compete, Speeders will utilise cutting-edge LiDAR and Machine Vision technology to ensure close but safe racing, with defined and digitally governed no-fly areas surrounding spectators and officials," Airspeeder wrote in a blog post.
Credit: Airspeeder
Beyond motorsports, Airspeeder hopes to help advance the electric vertical take-off and landing (eVTOL) sector. This sector is where companies like Uber, Hyundai, and Airbus are working to develop air taxis, which could someday take the ridesharing industry into the skies. By 2040, the autonomous urban aircraft industry could be worth $1.5 trillion, according to a 2019 report from Morgan Stanley.
Still, many technical and regulatory hurdles remain. Matt Pearson, Airspeeder's founder and CEO, thinks the futuristic motorsport will help to not only speed up that process, but also pave the way for self-driving cars.
"Even with autonomous vehicles on the ground, it's a difficult thing to get right because computers have to make decisions very fast," Airspeeder's founder and CEO, Matt Pearson, told GQ." But in a racing environment, you have a pretty controlled course and you have the ability to make all the vehicles cooperate with each other. You have a whole load of vehicles talking to each other, so if there's an incident or a pilot slows down or there's a traffic jam on the course they're all aware of each other. This is something we think will revolutionise autonomous vehicles on the ground. It's technology that will make flying cars a reality in our cities in the future."
Airspeeder has yet to announce a date for the first race, but Pearson said he hopes to put on three races over the first season. The company is developing two courses: one in California's Mojave Desert, and one near Coober Pedy in South Australia.
Space debris orbiting Earth
In 1957, the Soviet Union launched a human-made object into orbit for the first time. It marked the dawn of the Space Age. But when Sputnik 1's batteries died and the aluminum satellite began lifelessly orbiting the planet, it marked the end of another era: the billions of years during which space was pristine.
Today, the space above Earth is the world's "largest garbage dump," according to NASA. It's littered with 8,000 tons of human-made junk, called space debris, left by space agencies over the past six decades.
The U.S. now tracks more than 25,000 pieces of space junk. And that's only the debris that ground-based radar technologies can track. The U.S. Space Surveillance Network estimates there could be more than 170 million pieces of space debris currently orbiting Earth, with the majority being tiny fragments smaller than 1 mm.
Space debris: Trashing a planet
Space debris includes all human-made objects, big and small, that are orbiting Earth but no longer serve a useful function. A brief inventory of known space junk includes: a spatula, a glove, a mirror, a bag filled with astronaut tools, spent rocket stages, stray bolts, paint chips, defunct spacecraft, and about 3,000 dead satellites — all of which are orbiting Earth at speeds of roughly 18,000 m.p.h.
By allowing space debris to accumulate unchecked, we could be building a prison that keeps us stranded on Earth for centuries.
Most space junk is floating in low Earth orbit (LEO), the region of space within an altitude of about 100 to 1,200 miles. LEO is also where most of the world's 3,000 satellites operate, powering our telecommunications, GPS technologies, and military operations.
"Millions of pieces of orbital debris exist in low Earth orbit (LEO) — at least 26,000 the size of a softball or larger that could destroy a satellite on impact; over 500,000 the size of a marble big enough to cause damage to spacecraft or satellites; and over 100 million the size of a grain of salt that could puncture a spacesuit," wrote NASA's Office of Inspector General Office of Audits.
If LEO becomes polluted with too much space junk, it could become treacherous for spacecraft, threatening not only our modern technological infrastructure, but also humanity's ability to venture into space at all.
By allowing space debris to accumulate unchecked, we could be building a prison that keeps us stranded on Earth for centuries.
An outsized problem
Space debris of any size poses grave threats to spacecraft. But tiny, untrackable micro-debris presents an especially dreadful problem: A paint fragment chipped off a spacecraft might not seem dangerous, but it careens through space at nearly 10 times the speed of a bullet, packing enough energy to puncture an astronaut's suit, crack a window of the International Space Station, and potentially destroy satellites.
Impacts with space debris are common. During the Space Shuttle era, NASA replaced an average of one to two shuttle windows per mission "due to hypervelocity impacts (HVIs) from space debris." To be sure, some space debris are natural micrometeoroids. But much of it is human-made, like the fragment that struck the starboard payload bay radiator of the STS-115 flight in 2006.
"The debris penetrated both walls of the honeycomb structure, and the shock wave from the penetration created a crack in the rear surface of the radiator 6.8 mm long," NASA wrote. "Scanning electron microscopy and energy dispersive X-ray detection analysis of residual material around the hole and in the interior of the radiator shows that the impactor was a small fragment of circuit board material."
The European Space Agency notes that any fragment of space debris larger than a centimeter could shatter a spacecraft into pieces.
Impact chip on the ISSESA
To dodge space junk, the International Space Station (ISS) has to conduct "avoidance maneuvers" a couple times every year. In 2014, for example, flight controllers decided to raise the ISS's altitude by half a mile to avoid collision with part of an old European rocket in its orbital path.
NASA has strict guidelines for how it decides to perform these maneuvers.
"Debris avoidance maneuvers are planned when the probability of collision from a conjunction reaches limits set in the space shuttle and space station flight rules," NASA wrote. "If the probability of collision is greater than 1 in 100,000, a maneuver will be conducted if it will not result in significant impact to mission objectives. If it is greater than 1 in 10,000, a maneuver will be conducted unless it will result in additional risk to the crew."
These precautionary measures are becoming increasingly necessary. In 2020, the ISS had to move three times to avoid potential collisions. One of the latest close-calls came with such little warning that astronauts were instructed to take shelter in the Russian segment of the space station, in order to be closer to their Soyuz MS-16 spacecraft, which serves as an escape pod in case of an emergency.
The Kessler syndrome
The hazards of space debris grow exponentially over time. That's because of a problem that NASA scientist Donald J. Kessler outlined in 1978. The so-called Kessler syndrome states that as space becomes increasingly packed with spacecraft and debris, collisions become more likely. And because each collision would create more debris, it could trigger a chain reaction of collisions — potentially to the point where near-Earth space becomes a shrapnel field through which safe travel is impossible.
A paint fragment chipped off a spacecraft might not seem dangerous, but it careens through space at nearly 10 times the speed of a bullet, packing enough energy to puncture an astronaut's suit, crack a window of the International Space Station, and potentially destroy satellites.
The Kessler syndrome may already be playing out. Perhaps it began with the first known case of a spacecraft being severely damaged by artificial space debris, which occurred in 1996 when the French spy satellite Cerise was struck by a piece of an old European Ariane rocket. The collision tore off a 13-foot segment of the satellite.
The next major space debris incident occurred in 2007 when China conducted an anti-satellite missile test in which the nation destroyed one of its own weather satellites, triggering international criticism and creating more than 3,000 pieces of trackable space debris, most of which was still in orbit ten years after the explosion.
Then, in 2009, an unexpected collision between communications satellites — the active Iridium 33 and the defunct Russian Cosmos-2251 — produced at least 2,000 large fragments of space debris and as many as 200,000 smaller pieces, according to NASA. About half of all space debris currently orbiting Earth came from the Iridium-Cosmos collision and China's missile test.
There's more. Russia's BLITS satellite was spun out of its orbital path in 2013 after being struck by a piece of space debris suspected to have come from China's 2007 missile test; the European Space Agency's Copernicus Sentinel-1A satellite was struck by a tiny particle in 2016; and a window of the ISS was hit by a small fragment that same year.
As nations and private companies plan to send more satellites into orbit, collisions and impacts could soon become more common.
The promise and peril of satellite mega-constellations
Space organizations have recently begun launching satellites into low Earth orbit at an unprecedented pace. The goal is to create "mega-constellations" of satellites that provide high-quality internet access to virtually all parts of the planet.
Internet-providing satellites have existed for years, but they're typically expensive and provide slower service than land-based internet infrastructure. That's mainly because it can take a relatively long time for a signal to travel from the satellite to the user due to the high altitudes at which many of these satellites float above us in geostationary orbit.
China and companies like SpaceX, OneWeb, and Amazon aim to solve this problem by launching thousands of satellites into lower orbits in order to reduce signal latency, or the time it takes for the signal to travel to and from the satellite. But some space experts worry satellite mega-constellations could create more space debris.
"We face entirely new challenges as hundreds of satellites are launched every month now — more than we used to launch in a year," Thomas Schildknecht of the International Astronomical Union said at a European Space Agency conference in April. "The mega-constellations are producing huge risks of collisions. We need more stringent rules for traffic management in space and international mechanisms to ensure enforcement of the rules."
A 2017 study funded by the European Space Agency found that the deployment of satellite mega-constellations into low Earth orbit could increase the number of catastrophic collisions by 50 percent. Still, it remains unclear whether sending more satellites into space will necessarily cause more collisions.
SpaceX, for example, claims that Starlink satellites aren't at significant risk of collision because they're equipped with automated collision-avoidance propulsion systems. However, this system seemed to fail in 2019 when a Starlink satellite had a close call with a European science satellite named Aeolus. The company later said it had fixed the bug.
A batch of 60 Starlink test satellites stacked atop a Falcon 9 rocket.SpaceX
Currently, there are no strict international rules governing the deployment and management of satellite mega-constellations. But there are some international efforts to curb space debris risks.
The most concerted effort is the Inter-Agency Space Debris Coordination Committee (IADC), a forum that comprises 13 of the world's space agencies, including those of the U.S., Russia, China, and Japan. The committee aims "to exchange information on space debris research activities between member space agencies, to facilitate opportunities for cooperation in space debris research, to review the progress of ongoing cooperative activities, and to identify debris mitigation options."
The IADC's Space Debris Mitigation Guidelines list three broad goals:
1. Preventing on-orbit break-ups
2. Removing spacecraft from the densely populated orbit regions when they reach the end of their mission
3. Limiting the objects released during normal operations
But even though the world's space agencies recognize the gravity of the space debris problem, they're reluctant to act because of an incentives-based dilemma.
Space debris: A classic tragedy of the commons
Space debris is everyone's problem, but no one entity is obligated to solve it. It's a tragedy of the commons — an economic scenario in which individuals with access to a shared and scarce resource (space) act in their own best interest (spend the least amount of money). Left unchecked, the shared resource is vulnerable to depletion or corruption.
For example, the U.S. by itself could develop a novel method for removing space debris, which, if successful, would benefit all organizations with assets in space. But the odds of this happening are slim because of a game-theoretical dilemma.
A 2018 study published in Frontiers in Robotics and AI explains:
"[In space debris removal] each stakeholder has an incentive to delay its actions and wait for others to respond. This makes the space debris removal setting an interesting strategic dilemma. As all actors share the same environment, actions by one have a potential immediate and future impact on all others. This gives rise to a social dilemma in which the benefits of individual investment are shared by all while the costs are not. This encourages free-riders, who reap the benefits without paying the costs. However, if all involved parties reason this way, the resulting inaction may prove to be far worse for all involved. This is known in the game theory literature as the tragedy of the commons."
Similar to trying to curb climate change, there's no clear answer on how to best incentivize nations to mitigate space debris. (For what it's worth, the game theoretical model in the 2018 study found that a centralized solution — e.g., one where a single actor makes decisions on mitigating space debris, perhaps on behalf of a multinational coalition — is less costly than a decentralized solution.)
Although space organizations have been slow to act, many have been exploring ways to remove space junk from orbit and prevent new debris from forming.
Cleaning up space debris
Space organizations have proposed and experimented with many ways to remove debris from space. Although the techniques vary, most agree on strategy: get rid of the big stuff first.
That's because collisions involving large objects would create lots of new debris. So, removing big debris first would simultaneously clean up low Earth orbit and slow down the phenomenon of cascading collisions described by the Kessler syndrome.
To clean up low Earth orbit, space organizations have proposed using:
- Electrodynamic tethers: In 2017, the Japanese Aerospace Exploration Agency attempted to remove space debris by outfitting a cargo ship with an electrodynamic tether — essentially a fishing net made of stainless steel and aluminium. The craft then tried to "catch" space debris with the aim of dragging it into lower orbit, where it would eventually crash to Earth. The experiment failed.
- Ultra-thin nets: NASA's Innovative Advanced Concepts program has funded research for a project that would deploy extremely thin nets designed to wrap around space debris and drag them down to Earth's atmosphere.
- "Laser brooms": Since the 1990s, space researchers have proposed using ground-based lasers to strategically heat one side of a piece of space debris, which would change its orbit so that it re-enters Earth's atmosphere sooner. Because the laser systems would be based on Earth, this strategy could prove to be relatively affordable.
- Drag sails: As a relatively passive way to accelerate the de-orbit of space junk, NASA and other space organizations have been exploring the viability of attaching sails to space junk that would help guide debris back to Earth. These sails could either be packed within new satellites, to be deployed once the satellites are no longer useful, or attached to existing space junk.
Illustration of Brane Craft Phase II, which would use thin nets to capture space debris.Siegfried Janson via NASA
ClearSpace-1
But perhaps one of the most promising solutions for space debris is the ESA-funded ClearSpace-1 mission. Set to launch in 2025, ClearSpace-1 intends to be the first mission that successfully removes space debris from orbit. The goal is to launch a satellite into orbit and rendezvous with the upper stage of Europe's Vega launcher, which was left in space after a 2013 flight.
ClearSpace-1 satellite using its robotic arm to capture space debrisClearSpace-1
Once the satellite meets up with the debris, it will try to capture the junk with a robotic arm and then perform a controlled atmospheric reentry. The task will be challenging, in part because space junk tumbles as it flies above Earth, meaning the satellite will have to match its movements in order to safely capture it.
Freethink recently spoke to the ClearSpace-1 team to get a better understanding of the mission and its challenges.
Catching the Most Dangerous Thing in Space Freethink via youtube.com
But not all space debris removal strategies center on technology. A 2020 paper published in PNAS argued that imposing taxes on each satellite in orbit would be the most effective way to clean up space. Called "orbital use fees," the plan would charge space organizations an annual fee of roughly $235,000 per each satellite that's in orbit. The fee would, in theory, incentivize nations and companies to declutter space over time.
The main hurdle of orbital-use fees is getting all of the world's space organizations to agree to such a plan. If they do, it could help eliminate the tragedy of the commons aspect of space debris and potentially quadruple the value of the space industry by 2040.
"The costly buildup of debris and satellites in low-Earth orbit is fundamentally a problem of incentives — satellite operators currently lack the incentives to factor into their launch decisions the collision risks their satellites impose on other operators," the researchers wrote. "Our analysis suggests that correcting these incentives, via an OUF, could have substantial economic benefits to the satellite industry, and failing to do so could have substantial and escalating economic costs."
No matter the solution, cleaning up space debris will be a complex and expensive challenge that requires a coordinated, international effort. If the global community wants to maintain modern technological infrastructure and venture deeper into space, conducting business as usual isn't an option.
"Imagine how dangerous sailing the high seas would be if all the ships ever lost in history were still drifting on top of the water," Jan Wörner, European Space Agency (ESA) director general, said in a statement. "That is the current situation in orbit, and it cannot be allowed to continue."
"Researchers have been giving robots human-like perception," says MIT Associate Professor Fadel Adib. In a new paper, Adib's team is pushing the technology a step further. "We're trying to give robots superhuman perception," he says.
The researchers have developed a robot that uses radio waves, which can pass through walls, to sense occluded objects. The robot, called RF-Grasp, combines this powerful sensing with more traditional computer vision to locate and grasp items that might otherwise be blocked from view. The advance could one day streamline e-commerce fulfillment in warehouses or help a machine pluck a screwdriver from a jumbled toolkit.
The research will be presented in May at the IEEE International Conference on Robotics and Automation. The paper's lead author is Tara Boroushaki, a research assistant in the Signal Kinetics Group at the MIT Media Lab. Her MIT co-authors include Adib, who is the director of the Signal Kinetics Group; and Alberto Rodriguez, the Class of 1957 Associate Professor in the Department of Mechanical Engineering. Other co-authors include Junshan Leng, a research engineer at Harvard University, and Ian Clester, a PhD student at Georgia Tech.
Play videoAs e-commerce continues to grow, warehouse work is still usually the domain of humans, not robots, despite sometimes-dangerous working conditions. That's in part because robots struggle to locate and grasp objects in such a crowded environment. "Perception and picking are two roadblocks in the industry today," says Rodriguez. Using optical vision alone, robots can't perceive the presence of an item packed away in a box or hidden behind another object on the shelf — visible light waves, of course, don't pass through walls.
But radio waves can.
For decades, radio frequency (RF) identification has been used to track everything from library books to pets. RF identification systems have two main components: a reader and a tag. The tag is a tiny computer chip that gets attached to — or, in the case of pets, implanted in — the item to be tracked. The reader then emits an RF signal, which gets modulated by the tag and reflected back to the reader.
The reflected signal provides information about the location and identity of the tagged item. The technology has gained popularity in retail supply chains — Japan aims to use RF tracking for nearly all retail purchases in a matter of years. The researchers realized this profusion of RF could be a boon for robots, giving them another mode of perception.
"RF is such a different sensing modality than vision," says Rodriguez. "It would be a mistake not to explore what RF can do."
RF Grasp uses both a camera and an RF reader to find and grab tagged objects, even when they're fully blocked from the camera's view. It consists of a robotic arm attached to a grasping hand. The camera sits on the robot's wrist. The RF reader stands independent of the robot and relays tracking information to the robot's control algorithm. So, the robot is constantly collecting both RF tracking data and a visual picture of its surroundings. Integrating these two data streams into the robot's decision making was one of the biggest challenges the researchers faced.
"The robot has to decide, at each point in time, which of these streams is more important to think about," says Boroushaki. "It's not just eye-hand coordination, it's RF-eye-hand coordination. So, the problem gets very complicated."
The robot initiates the seek-and-pluck process by pinging the target object's RF tag for a sense of its whereabouts. "It starts by using RF to focus the attention of vision," says Adib. "Then you use vision to navigate fine maneuvers." The sequence is akin to hearing a siren from behind, then turning to look and get a clearer picture of the siren's source.
With its two complementary senses, RF Grasp zeroes in on the target object. As it gets closer and even starts manipulating the item, vision, which provides much finer detail than RF, dominates the robot's decision making.
RF Grasp proved its efficiency in a battery of tests. Compared to a similar robot equipped with only a camera, RF Grasp was able to pinpoint and grab its target object with about half as much total movement. Plus, RF Grasp displayed the unique ability to "declutter" its environment — removing packing materials and other obstacles in its way in order to access the target. Rodriguez says this demonstrates RF Grasp's "unfair advantage" over robots without penetrative RF sensing. "It has this guidance that other systems simply don't have."
RF Grasp could one day perform fulfilment in packed e-commerce warehouses. Its RF sensing could even instantly verify an item's identity without the need to manipulate the item, expose its barcode, then scan it. "RF has the potential to improve some of those limitations in industry, especially in perception and localization," says Rodriguez.
Adib also envisions potential home applications for the robot, like locating the right Allen wrench to assemble your Ikea chair. "Or you could imagine the robot finding lost items. It's like a super-Roomba that goes and retrieves my keys, wherever the heck I put them."
The research is sponsored by the National Science Foundation, NTT DATA, Toppan, Toppan Forms, and the Abdul Latif Jameel Water and Food Systems Lab (J-WAFS).
Reprinted with permission of MIT News. Read the original article.
