Augmented Reality Meets Music: Learn to play the piano with AR technology

16

September

2024

5/5 (1)

Have you ever dreamed of playing the piano but never really had the time to go for lessons or couldn’t have access to an instrument to play? Now, these problems can easily be solved with the help of the new emerging technology of Augmented Reality (AR).

The logo of the PianoVision app

PianoVision [1] is a new application that uses AR to teach its users how to play the piano, regardless of the user’s access to an actual real piano or a “virtual” one. The app is available on MetaQuest and the gamers can use their headsets to learn notes or hand positions, by either using a fully “virtual” piano or in addition to a real piano [2]. PianoVision is billed as “the next evolution of the world’s most popular instrument” [3].

Using the AR technology, users can see their hands and the real/virtual piano keyboards in front of them while the app overlays interactive animations through the headsets, showing how to play the chosen song on the instrument. What gives an exciting experience is that the app calibrates easily with your physical piano by touching the two end notes from the piano. Once you do this, you can see on your real/virtual piano the octaves and the type of notes from all the piano. The indication of which finger you should use to hit a note is another great feature, together with the app stopping the song when you hit the wrong note and waiting for you until the correct note is pressed. For those interested in learning the theory behind playing the piano, PianoVision also offers the possibility of learning the notes by showing a floating piano sheet above the piano while playing the song on your instrument.

Video showing a sample of a user playing the piano with the help of PianoVision

The founder of the revolutionary application, Zac Reid, launched the app in February 2022 and claimed that he created the app intending to help wannabe musicians learn the piano in the same style as Guitar Hero was intended to teach users how to play the guitar [3]. Guitar Hero is a rhythm game where players use a guitar-shaped controller to simulate playing the guitar [4].

So, are you excited to learn how to play an instrument with the help of the emerging technology of AR? Hop into this adventure that perfectly blends the most representative classical music instrument with the new possibilities offered by augmented reality technology!

References:
[1] – https://www.pianovision.com/
[2] – https://www.classicfm.com/discover-music/instruments/piano/metaverse-virtual-augmented-reality/
[3] – https://www.musicradar.com/news/pianovision-ar-learning-app
[4] – https://en.wikipedia.org/wiki/Guitar_Hero

Please rate this

How is AI transforming the music industry? – 5 trends under examination

9

October

2019

No ratings yet. Artificial Intelligence (AI) is one of the most promising technologies of the future, bringing many opportunities and threats to diverse industries. But with entering the music industry AI has the potential to transform an industry that is (so far) dominated and shaped by contemporary art, talents and individualism throughout the human history. Already today, many AI-based programs and applications are used for discovering, creating and teaching music. It helps to test new ideas, find emotional context and integrates fun in the learning process. Let’s examine some already existing trends:

 

#1 Recommending music (Spotify)
Probably the best-known example of AI in the music industry it the recommendation system of Spotify. As there are thousands of songs uploaded on Spotify every day, it is difficult to only find and propose songs that are categorized based on genre. Rather, Spotify uses an AI algorithm to sponsor your “Discover weekly” playlist. The algorithm is predicting the preference a user would give to an item, based on previous behavior (ratings, length of listening, type of songs the user often listen to etc.). With that information the system proposes other items the user might enjoy.

#2 Creating music (AIVA)
AI that composes music is already in practice for a while. Most AI-powered systems work by using deep learning networks and analyze large amount of input data (in form of existing music pieces). The software then analyzes patterns in rhythm, chords, length and notes in this source material. Through reinforcement learning, the algorithm builds up on characteristics and writes its own melodies that resembles a certain genre. One example is AIVA is that can compose emotional music soundtracks for films, video games or commercials. The company believes that “AI-generated music will enable new use cases of music, and empower the next generation of composers and content creators.”

#3 Inspiring song texts (IBM Watson)
AI is not only capable of composing music, the technology can also act as an inspiring muse for song writers. In 2016 IBM Watson wrote a song with musician Alex Da Kid, that was even successful in the Top 40 charts of Billboard Charts. The AI-powered program is able to create lyrics that correspond to a predefined emotional theme. The program uses natural-language processing to analyze a massive amount of unstructured data like articles, blogs and data from social media in order to depict and illustrate the most influential and current topics of the time. With that Watson served as the perfect partner for Alex Da Kid by giving inspiration for a meaningful and contemporary song text. The ability to identify patterns across data and making connections across current topics, is a one of AI’s greatest benefits. You can listen to the song here.

#4 Learning to play an instrument (Musiah)
Many people don’t want or can afford private music lessons to learn an instrument. In fact, already now people value the flexibility and freedom that e-learning of mobile applications bring with them. There are already some applications on the market, but Musiah claims to be the best online piano teaching app that provides learners with human-like piano instructions. Through AI the piano app can follow every move of your fingers and can respond to your playing in real time – making you feel your piano teacher is sitting right next to you, listening on your playing and giving instructions, support and immediate feedback. And the best thing (next to independency and flexibility): the application is affordable for most as it only costs a fraction of a traditional piano teacher.

#5 Playing with virtual-musicians (Continuator)
Don’t want to play on your own? Continuator makes it possible to interactively play with a virtual musician in your own style! The continuator is a musical instrument that learns and interactively plays with you in your own individual music style. The algorithm, designed by François Pachet in Sony Computer Science Laboratory, can learn and interactively play with a live-musician by real-time learning and composing a piece of music from the point the live-musician stopped. For instance, in an experiment the continuator played with a professional pianist, and the audience could not differentiate whether the pianist still played or the computer already took over. Click here for some examples.

 

Well, there are already many great and mind-blowing ways that AI is impacting the way we like, create and make music. While some advances like Spotify’s recommendation system or the piano learning app Musiah can be great extensions to the current industry, artificial music is still unfamiliar for most. For many, it seems questionable by how far artificial music will be able to evoke feelings we usually have after hearing a heart-breaking song of our favorite real artist. In the end, we are emphasizing with the artist and believe we have similar experiences, feelings or problems.

So, whether a computer can outset musicians entirely is rather a fundamental question to humanity and hard to find an answer. What do you think? Do you think the AI can and will fundamentally change the music industry?

 

Bibliography

Dehal, B., (2018).  How AI-generated music is changing the way hits are made. [online] theverge.com. Available at: https://www.theverge.com/2018/8/31/17777008/artificial-intelligence-taryn-southern-amper-music [Accessed 08.10.2019]

Kharkovya, O., (2019). Artificial Intelligence and Music: What to Expect?. [online] towardsdatascience.com. Available at  https://towardsdatascience.com/artificial-intelligence-and-music-what-to-expect-f5125cfc934f [Accessed 08.10.2019]

Marr, B., (2019). The amazing ways Artificial Intelligence is transforming the music industry. [online] Forbes.com. Available at: https://www.forbes.com/sites/bernardmarr/2019/07/05/the-amazing-ways-artificial-intelligence-is-transforming-the-music-industry/#736817965072 [Accessed 08.10.2019]

Velardo, V., (2019). Spotify’s Discover Weekly explained – Breaking from your music bubble or, maybe not? [online] medium.com. Available at: https://medium.com/the-sound-of-ai/spotifys-discover-weekly-explained-breaking-from-your-music-bubble-or-maybe-not-b506da144123 [Accessed 08.10.2019]

Please rate this