Teaching Robots the Most Powerful Word in the English Language

“I'm sorry Dave; I'm afraid I can't do that.”

We're bearing witness to an interesting time in technological development: the rise of robot servants. They'll soon be in our cars (if they haven't been uploaded to your vehicle already) and delivering our Amazon packages. Many dream of one day having a robot butler that will service their every need. But what if a human gives a command that could do harm? Or one that may put the robot at risk?

Researchers Gordon Briggs and Matthias Scheutz, from Tufts University, are working on a mechanism to teach robots to say, "No," to their human overlords. The system is one that allows a robot to not only understand the language of a command, but the larger context — whether the robot is actually capable of executing it.

A series of conditions, known as felicity conditions, must hold in order for the robot to accept the proposed action. The researchers write:

1. Knowledge: Do I know how to do X?
2. Capacity: Am I physically able to do X now? Am I normally physically able to do X?
3. Goal priority and timing: Am I able to do X right now?
4. Social role and obligation: Am I obligated based on my social role to do X?
5. Normative permissibility: Does it violate any normative principle to do X?

In the video below, the researchers demonstrate how a robot might process a command to walk forward when an obstacle is in the way, reasoning that a command to walk forward is wrong.

The ethical debates surrounding robots in mainstream society is still a heated one. It's an area Jerry Kaplan talks about quite a bit, questioning how our laws will adapt to punish wrong-doing robots. He says humans are “going to need new kinds of laws that deal with the consequences of well-intentioned autonomous actions that robots take.”

But what about the well-intentioned robots with ill-intentioned owners?


Natalie has been writing professionally for about 6 years. After graduating from Ithaca College with a degree in Feature Writing, she snagged a job at PCMag.com where she had the opportunity to review all the latest consumer gadgets. Since then she has become a writer for hire, freelancing for various websites. In her spare time, you may find her riding her motorcycle, reading YA novels, hiking, or playing video games. Follow her on Twitter: @nat_schumaker

Photo Credit: Getty Images / Staff

Big Think
Sponsored by Lumina Foundation

Upvote/downvote each of the videos below!

As you vote, keep in mind that we are looking for a winner with the most engaging social venture pitch - an idea you would want to invest in.

Keep reading Show less

7 fascinating UNESCO World Heritage Sites

Here are 7 often-overlooked World Heritage Sites, each with its own history.

Photo by Raunaq Patel on Unsplash
Culture & Religion
  • UNESCO World Heritage Sites are locations of high value to humanity, either for their cultural, historical, or natural significance.
  • Some are even designated as World Heritage Sites because humans don't go there at all, while others have felt the effects of too much human influence.
  • These 7 UNESCO World Heritage Sites each represent an overlooked or at-risk facet of humanity's collective cultural heritage.
Keep reading Show less

Scientists create a "lifelike" material that has metabolism and can self-reproduce

An innovation may lead to lifelike evolving machines.

Shogo Hamada/Cornell University
Surprising Science
  • Scientists at Cornell University devise a material with 3 key traits of life.
  • The goal for the researchers is not to create life but lifelike machines.
  • The researchers were able to program metabolism into the material's DNA.
Keep reading Show less

Scientists discover how to trap mysterious dark matter

A new method promises to capture an elusive dark world particle.

Surprising Science
  • Scientists working on the Large Hadron Collider (LHC) devised a method for trapping dark matter particles.
  • Dark matter is estimated to take up 26.8% of all matter in the Universe.
  • The researchers will be able to try their approach in 2021, when the LHC goes back online.
Keep reading Show less