Is your child’s smart toy actually a creepy surveillance tool?

Imagine a child born this year who will be surrounded by technology at every phase of her childhood. When she is three years old, Sophie’s parents buy her a smart doll that uses facial recognition and artificial intelligence (AI) to watch, listen to, and learn from her.

Like many children, Sophie will come to love this toy. And like previous generations of children with a favorite doll or teddy bear, she will carry it around with her, talk with it, and sleep with it beside her for many years.

If the smart doll is designed responsibly, this toy could be her best friend; if not, it will be a surveillance tool that records her every move and word spoken in its presence by her, her friends, and even her parents.

Smart toys use AI to learn about the child user and personalize the play or learning experience for them. They can learn a child’s favourite colour, song, and learn to recognize that child and other familiar people in that child’s life. While this may sound futuristic, there are many smart toys that already provide these capabilities. The market for these toys is rapidly expanding and will grow to $18 billion by 2023.

To address this urgent use of AI, the World Economic Forum recently launched the Smart Toy Awards to recognize ethically and responsibly designed AI-powered toys that create an innovative and healthy play experience for children.

Smart toys provide enormous promise for children. They can customize learning based on data they gather about children; they can teach computer programming skills to children; and they can help children with disabilities develop cognitive, motor, and social skills.

But at the same time, smart toys provide large potential risks if they are not designed to protect children’s data, safety, and cybersecurity.

A cautionary tale

The example of Sophie’s smart doll is not far-fetched. In 2017, My Friend Cayla – an early smart toy that used facial and voice recognition – was declared an illegal surveillance tool in many countries.

If the Cayla doll was connected to a phone, data was sent to the manufacturer and a third-party company for processing and storage. And anyone with the My Friend Cayla app on their phone within 30 feet of a toy could access the toy and listen to the child user.

Germany issued a “kill order” for the doll and required parents to destroy it “with a hammer.” Today, the only surviving Cayla dolls in Germany reside in the Spy Museum in Berlin.

The risks posed by smart toys

Sophie applies to college when she is 18 years old. If her smart doll had collected data on her from the age of 3 to 9, the company who built the toy could know her better than her parents. Without adequate data protections, the company could also sell this data to the colleges to which she is applying or other third parties.

After college, Sophie applies to a job. If the employer bought data gathered on Sophie as a child, they could learn about her strengths and weaknesses. What if Sophie bullied her younger sister, yelled at her parents, or refused to do her homework as a child? All these actions conducted in the privacy of the family’s home could be known by the company and sold to third parties who could use this information to discriminate against Sophie. The family’s life is no longer private.

Today, data is gold but gathering data on children is inherently problematic. As a company gathers data about children through Sophie’s doll, they may have a responsibility to act or intervene. Imagine that Sophie tells her doll about suicidal thoughts and self-harm. Should the company be required to alert the parents and call 911?

The more data that a smart toy gathers the more complex scenarios smart toy companies will likely face. Every company designing a smart toy with the capabilities to gather this information must consider these worst-case scenarios as they develop toys to protect the safety of the child user and those around them.

Developing responsible and ethical smart toys

Despite these significant risks, ethical and responsible smart toys are being developed. The Smart Toy Awards have developed four key governance criteria for companies developing AI-powered toys: data privacy and cybersecurity; accessibility and transparency; age appropriateness; and healthy play.

Sophie’s smart doll illustrates the importance of strong data privacy and clearly communicating to adults buying the toy what the smart doll does and how it operates. This must be communicated in the Terms of Service in language understandable by non-technologically literate audiences. At minimum, Smart Toys should meet COPPA requirements in the US and GDPR in the EU.

Parents and guardians should understand with whom children’s data is being shared and for what purpose. Companies should empower parents, guardians, and children to make their own decisions about how children’s data is being used. And companies should not sell children’s data to third parties.

Data privacy is a foundation for ethical and responsible smart toys, but they must also be designed to be accessible, transparent, age appropriate, and promote healthy play and children’s mental health.

The future of childhood

Sophie’s doll doesn’t necessarily pose concern for her and her parents, and data collected on her won’t hinder her future if the data is carefully protected. In the EU, GDPR provides the right to be forgotten, and a similar policy could allow children like Sophie to request that all data collected on them as children by their smart toys be deleted when they turn 18 years old, so they would have a fresh start as they begin adulthood.

Sophie and all children should have a fair shot at childhood, education, careers, and life. The data collected on them as children should not be used to discriminate against them in the future.

Smart toys like Sophie’s doll can play a pivotal role in childhoods, catalyzing creativity and critical thinking skills. Many companies are developing smart toys with careful consideration for ethics and responsibility. We urge companies to adopt our governance criteria as they’re designing and developing smart toys.

Childhood is a sacred time and parents will do everything they can to protect their children’s experiences. This won’t be possible unless stakeholders work together across the private, public, and nonprofit sectors to develop ethical, responsible, and innovative smart toys that protect and foster the essence of childhood.

Reprinted with permission of the World Economic Forum. Read the original article.