Can Hidden Markov Models Accurately Match Prerecorded and Live Guitar Data?

  • Thread starter Thread starter ADDA
  • Start date Start date
  • Tags Tags
    Lost Model
Click For Summary

Discussion Overview

The discussion revolves around the challenge of matching prerecorded guitar data with live data using Hidden Markov Models (HMMs) and neural networks. Participants explore various approaches to transcribing guitar sounds into tablature, addressing the complexities involved in accurately identifying notes and strings played in real-time.

Discussion Character

  • Exploratory
  • Technical explanation
  • Conceptual clarification
  • Debate/contested

Main Points Raised

  • One participant suggests using the Baum Welch process for estimating the matching of two sequences, considering scaling and rounding data to integers as states.
  • Another participant questions the clarity of the original problem and the specifics of the data being used.
  • A different participant proposes recording sounds for each fret on each string and training a neural network with this data to improve accuracy in identifying notes.
  • Concerns are raised about the complexity of guitar sounds, noting that multiple notes can be produced in various ways, complicating the identification process.
  • One participant mentions having some success in identifying root tones through signal processing but expresses uncertainty about error calculation in training a network.
  • Another participant emphasizes the importance of practical limitations in guitar playing, such as the number of strings that can be played simultaneously and the physical constraints of hand positioning.
  • Discussion includes the challenges of overtones and the difficulty in distinguishing between plucked tones and overtones, complicating the training of neural networks.
  • One participant highlights the subjective nature of transcribing music, noting that different musicians may arrive at different interpretations of the same piece.

Areas of Agreement / Disagreement

Participants express a range of views on the feasibility and methods for matching guitar sounds, with no consensus reached on the best approach. The discussion reflects differing opinions on the complexity of the task and the effectiveness of proposed solutions.

Contextual Notes

Participants acknowledge various limitations, such as the ambiguity in identifying notes due to overtones, the subjective nature of musical transcription, and the physical constraints of guitar playing that may affect the feasibility of certain approaches.

Who May Find This Useful

This discussion may be of interest to those involved in music technology, machine learning applications in audio processing, and musicians seeking to understand the complexities of sound transcription and analysis.

ADDA
Messages
67
Reaction score
2
Hi, I'm attempting to match prerecorded data from a guitar, to data recorded, as a user plays, as shown in the following video:



The red lines are the prerecorded data, and the cyan lines are the live data. In the above example, each string is plucked open, so if you look to the lower left first, the low E string, the above that the A string and so forth.

From this pdf (no pun intended) http://www.indiana.edu/~iulg/moss/hmmcalculations.pdf posted on wikipedia, I can infer that the Baum Welch process gives an estimate of matching two sequences. My initial thoughts, were to scale the data since it is in the [0.0-1.0) range, round to an integer and then set that as a state, yet I think that there is another way of doing this. Any suggestions? I'm using the jahmm library for now.
 
Technology news on Phys.org
I have no idea what the question is you're trying to solve. Reconstruct a melody from guitar souds?
I also have no idea what data you're using.
 
willem2, I'm attempting to write down music to tabalture for musicians. Here is an example:

 
So you are attempting to listen to a sound, and determine exactly what string was plucked at which fret? If you play the guitar, I'm sure that you know that many notes can be made in many different ways. For example: middle C is both the fifth fret on the G string and the first fret of the B string. They are the same note, but do not sound the same because they have different undertones. If you play two strings at the same time, it gets even harder. Here is how I would approach this problem:

Record myself playing each fret on each string, record the sound, then map it to the correct tab. I'd then build a neural network and train it using the data that I have produced. That should give you a decent start, and you can expand the training material to include things like multiple strings being played together.

I'm not sure what the second video is. If those squares are supposed to represent frets, they do not match the sound that I hear.
 
  • Like
Likes   Reactions: stoomart
newjerseyrunner... thanks for your reply... I've been trying to train a network with the above data to no resolve, yet however, I've somewhat succeeded in find the root tone through signal processing alone. Here is my research:



What do you think? Some omissions and a few mistakes, yet I'll be ready to move forward, maybe after some more testing. The above video (in this post) does not include any AI or recording of tones. I've been thinking of training a network as a user might play yet am still unsure of proper error calculation. Anyrate, the above video (in the last post) is A Am B Bm C D Dm E/Em F G
 
The calculations for neural networks are done through back propagation, they are not to be put in manually.

It's looking good. I'd think maybe the next step is to start adding limitations to how far apart the green boxes can be. Most of the tabs displayed that I saw aren't actually playable with one hand. You can only play 4 strings at a time (plus the open strings) and your hand is only so big, so it doesn't make sense to have one string on the 12th fret and the other at the first.
 
I watch the following video on neural nets:



which change the way that I write them. I had thought that the activation function was the hyperbolic tangent. My efforts to find the root tone have been difficult because most chords involve an octave overtone. So, the root tone may have 1, 2, or 3 overtones. Each corresponding to 12, 19, or 24 halftones. I am unsure of how to train a network because, in my experimentation, there is no absolute way to know for certain whether a tone is plucked or an overtone, and I've just had another idea, of which I've forgotten.

Regarding decision tree like limitations, I can think of exceptions: bar chords (only 4 strings), and capos (only the open strings). I realize that you are offering help, and I am thankful.
 
Do you know how to play the guitar?
What you are trying to do is VERY difficult if you want a general solution. This is one of those problems that even the human brain struggles with since there is always multiple play something on a guitar and the "best" way to play something will often not be determined by what sounds best but what is possible'/ easier to play (which will also depend on the tempo of the song; e.g. switching from a bar chord to an open chord takes more time then to just use another bar chord).

Hence, if you put two experienced musicians in a room and ask them to transcribe a piece of music it is very possible that they would come up with very different solutions (nowadays this is less of a problems since it is often possible to find a video and actually look at the hand of the guitar player when transcribing a song) .
Add the possibility of different turnings (common) and the use of a capo (also very common) into the mix and you have a very tricky problem.