AI and brain interfaces may be about to change how we make music

Computer control in the form of AI and brain-computer interfaces is being introduced to the art of composing.

Moriyama and piano
(Yamaha Corporation)

When Yamaha demonstrated their AI allowing a dancer to play the piano with movement in a Tokyo concert hall in November 2017, it was the latest example of the ways in which computers are increasingly getting involved in music-making. We’re not talking about the synthesizers and other CPU-based instruments in contemporary music. We’re talking about computers’ potential as composers’ tools, or as composers themselves.


Yamaha’s use of AI is quite different. In the recent performance, world-renowned dancer Kaiji Moriyama was outfitted with electrodes on his back, wrists, and ankles, and set free to express himself as AI algorithms converted his movements into musical phrases for transmission to Yamaha’s Disklavier piano via MIDI messages. (MIDI is a computer language through which musical instruments can be controlled.)

(Yamaha Corporation)

Yamaha’s AI, which they’re still developing, worked with a database of linked musical phrases from which it selected and drew melodies for sending to the instrument based on Moriyama’s motions. 

Moriyama was accompanied onstage by the Berlin Philharmonic Orchestra Scharoun Ensemble.

(Yamaha Corporation)

We’ve written previously about other musical AI, the web-based AI platform called Amper that uses AI to compose passages based on descriptors of style, instrumentation, and mood. Singer/songwriter Taryn Southern was using Amper as her primary collaborator in writing an album. 

Another fascinating avenue being explored is the use of brain-to-computer (BCI) interfaces that allow wearers to think music into existence. It’s a fascinating way for anyone to play music, but it’s especially promising for people whose physical limitations make the creation of music difficult or even impossible otherwise.

Certain electroencephalogram signals correspond to known brain activities such as the P300 ERP (for “event-related potential”) that signifies a person’s reaction to some stimulus. It’s previously been by brain-computer interface (BCI) applications in spelling, operating local environmental controls, operating web browsers, and for painting. In September 2017, researchers led by BCI expert Gernot Müller-Putz from TU Graz's Institute of Neural Engineering published research in PLOS/One describing their “Brain Composer” project that leveraged P300 to bring musical ideas directly from composers’ mind to notated sheets of music. They work in collaboration with MoreGrasp and "Feel Your Reach”.

The researchers’ first step was training the BCI application to recognize alphabetical letters before moving on to note length and pitch, as well as notation values such as rests, ties, and such. In this video, the researchers demonstrate the successful outcome of just 90-minutes’ work for a motor-impaired subject.

(i-KNOW Conference)

This is all exciting stuff — and a heaven-send for musical souls with physical limitations — even if the results are a little odd, as in Yamaha’s case. (The BCI example sounds remarkably like the theme from Fringe.)

We’ve been augmenting our natural musical capabilities with technology ever since we picked up our first rock — and certainly by the time we honked steam-punk-looking saxophones. We should have no issue with adding AI and BCIs to our toolbox.

If AI comes up with music we might not, that’s fine. The workings of music remain pretty mysterious, in any event, so here’s an intriguing thought. Though many of us basically prefer music that’s catchy and sonorous, the stuff that really gets us in the gut tends to have something of the unexpected to it, a surprising dissonance or rhythm, and odd “hair” out of place, that makes the moment leap from our speakers or the stage and into our lives as more of an experience than a piece of art, leaving us a little startled and even moved. So if AI can beat a flesh-and-blood chess player by not thinking like a human, imagine what we’re about to hear.

 

What does kindness look like? It wears a mask.

Northwell Health CEO Michael Dowling has an important favor to ask of the American people.

Sponsored by Northwell Health
  • Michael Dowling is president and CEO of Northwell Health, the largest health care system in New York state. In this PSA, speaking as someone whose company has seen more COVID-19 patients than any other in the country, Dowling implores Americans to wear masks—not only for their own health, but for the health of those around them.
  • The CDC reports that there have been close to 7.9 million cases of coronavirus reported in the United States since January. Around 216,000 people have died from the virus so far with hundreds more added to the tally every day. Several labs around the world are working on solutions, but there is currently no vaccine for COVID-19.
  • The most basic thing that everyone can do to help slow the spread is to practice social distancing, wash your hands, and to wear a mask. The CDC recommends that everyone ages two and up wear a mask that is two or more layers of material and that covers the nose, mouth, and chin. Gaiters and face shields have been shown to be less effective at blocking droplets. Homemade face coverings are acceptable, but wearers should make sure they are constructed out of the proper materials and that they are washed between uses. Wearing a mask is the most important thing you can do to save lives in your community.
Keep reading Show less

Science confirms: Earth has more than one 'moon'

Two massive clouds of dust in orbit around the Earth have been discussed for years and finally proven to exist.

J. Sliz-Balogh, A. Barta and G. Horvath
Surprising Science
  • Hungarian astronomers have proven the existence of two "pseudo-satellites" in orbit around the earth.
  • These dust clouds were first discovered in the sixties, but are so difficult to spot that scientists have debated their existence since then.
  • The findings may be used to decide where to put satellites in the future and will have to be considered when interplanetary space missions are undertaken.
Keep reading Show less

How do pandemics end? History suggests diseases fade but are almost never truly gone

Instead of looking forward, we should be consulting the past.

Andrew Redington/Getty Images
Coronavirus

When will the pandemic end? All these months in, with over 37 million COVID-19 cases and more than 1 million deaths globally, you may be wondering, with increasing exasperation, how long this will continue.

Keep reading Show less

Scientists stumble across new organs in the human head

New cancer-scanning technology reveals a previously unknown detail of human anatomy.

Credit: Valstar et al., Netherlands Cancer Institute
Surprising Science
  • Scientists using new scanning technology and hunting for prostate tumors get a surprise.
  • Behind the nasopharynx is a set of salivary glands that no one knew about.
  • Finding the glands may allow for more complication-free radiation therapies.
Keep reading Show less
Personal Growth

Millennials reconsidering finances and future under COVID-19

A new survey found that 27 percent of millennials are saving more money due to the pandemic, but most can't stay within their budgets.

Scroll down to load more…
Quantcast