Synthetic Intelligence Powered Prosthetic Arm Allows Drummer To Proceed Making Music

Jason Barnes liked drumming when he was a little boy – he hit anything he could get his hands on as soon as he started listening to music! In 2012, when he was 22 years old, his right hand was amputated due to an industrial accident. In theory, this meant Jason couldn’t play the drums as well. Although it was a setback, Jason was determined to find a solution that would help him continue his passion for drumming well into the future.

Jason initially drummed with what is known as a body-powered prosthesis. There were no electronics or robot controls so he had to use a lot of shoulder and elbow to get the hits he wanted. There were two problems with this prosthesis, however – it was exhausting and it limited how fast he could play. To remedy these problems, he came up with the idea of ​​developing a robotic drum prosthesis. Since 2013, Jason has worked with Gil Weinberg, a renowned robotics scientist and founding director of Georgia Tech for Music Technology, to use artificial intelligence to create what is possibly the most advanced robotic drum arm in the world.

As soon as Jason reached out, Gil knew what he wanted – he wanted to feel more of his muscles and control the ricochet of the drumstick. In short, play the drums as humanely as possible with a prosthetic arm. When they started working on this project, they found that there were many complex steps to take care of. One of my biggest concerns was latency – the delay that occurred between Jason’s hit and the actual detection of the prosthesis. There was even a 50 millisecond delay. In addition, how the drumstick was grabbed and ricocheted by a drum was also important – these were items that were difficult to calibrate.

The team then decided to use EMG or electromyography. This method consists of sensors in the prosthesis that pick up electrical signals from Jason’s stump. With the help of these signals, the team was able to identify complex machine learning patterns and then assign them to specific movements. TensorFlow, Google’s open source AI platform, does this in real time – no more latency! When Jason flexes his muscle, he tightens his grip on the stick and loosens it when he stretches his muscle, just like a normal hand.

Close up of Jason's prosthetic robotic arm that he uses for drumming.

This revolutionary robotic drum arm has opened many doors to the exploration of accessibility, especially in music, and has enabled many disabled musicians to continue making music with ease without encountering many limitations.

Check out the video below to hear Jason’s story and how this drum arm came to be a reality!

Jason is on Instagram as Cybernetx and his music can be found on Spotify, Apple Music and Soundcloud.

Source: Google, Google

Comments are closed.