# Lost in Hidden Markov Model

1. Jul 9, 2017

Hi, I'm attempting to match prerecorded data from a guitar, to data recorded, as a user plays, as shown in the following video:

The red lines are the prerecorded data, and the cyan lines are the live data. In the above example, each string is plucked open, so if you look to the lower left first, the low E string, the above that the A string and so forth.

From this pdf (no pun intended) http://www.indiana.edu/~iulg/moss/hmmcalculations.pdf posted on wikipedia, I can infer that the Baum Welch process gives an estimate of matching two sequences. My initial thoughts, were to scale the data since it is in the [0.0-1.0) range, round to an integer and then set that as a state, yet I think that there is another way of doing this. Any suggestions? I'm using the jahmm library for now.

2. Jul 10, 2017

### willem2

I have no idea what the question is you're trying to solve. Reconstruct a melody from guitar souds?
I also have no idea what data you're using.

3. Jul 12, 2017

willem2, I'm attempting to write down music to tabalture for musicians. Here is an example:

4. Jul 13, 2017

### newjerseyrunner

So you are attempting to listen to a sound, and determine exactly what string was plucked at which fret? If you play the guitar, I'm sure that you know that many notes can be made in many different ways. For example: middle C is both the fifth fret on the G string and the first fret of the B string. They are the same note, but do not sound the same because they have different undertones. If you play two strings at the same time, it gets even harder. Here is how I would approach this problem:

Record myself playing each fret on each string, record the sound, then map it to the correct tab. I'd then build a neural network and train it using the data that I have produced. That should give you a decent start, and you can expand the training material to include things like multiple strings being played together.

I'm not sure what the second video is. If those squares are supposed to represent frets, they do not match the sound that I hear.

5. Jul 31, 2017

newjerseyrunner... thanks for your reply... I've been trying to train a network with the above data to no resolve, yet however, I've somewhat succeeded in find the root tone through signal processing alone. Here is my research:

What do you think? Some omissions and a few mistakes, yet I'll be ready to move forward, maybe after some more testing. The above video (in this post) does not include any AI or recording of tones. I've been thinking of training a network as a user might play yet am still unsure of proper error calculation. Anyrate, the above video (in the last post) is A Am B Bm C D Dm E/Em F G

6. Aug 1, 2017

### newjerseyrunner

The calculations for neural networks are done through back propagation, they are not to be put in manually.

It's looking good. I'd think maybe the next step is to start adding limitations to how far apart the green boxes can be. Most of the tabs displayed that I saw aren't actually playable with one hand. You can only play 4 strings at a time (plus the open strings) and your hand is only so big, so it doesn't make sense to have one string on the 12th fret and the other at the first.

7. Aug 2, 2017

I watch the following video on neural nets:

which change the way that I write them. I had thought that the activation function was the hyperbolic tangent. My efforts to find the root tone have been difficult because most chords involve an octave overtone. So, the root tone may have 1, 2, or 3 overtones. Each corresponding to 12, 19, or 24 halftones. I am unsure of how to train a network because, in my experimentation, there is no absolute way to know for certain whether a tone is plucked or an overtone, and I've just had another idea, of which I've forgotten.

Regarding decision tree like limitations, I can think of exceptions: bar chords (only 4 strings), and capos (only the open strings). I realize that you are offering help, and I am thankful.

8. Aug 3, 2017

### f95toli

Do you know how to play the guitar?
What you are trying to do is VERY difficult if you want a general solution. This is one of those problems that even the human brain struggles with since there is always multiple play something on a guitar and the "best" way to play something will often not be determined by what sounds best but what is possible'/ easier to play (which will also depend on the tempo of the song; e.g. switching from a bar chord to an open chord takes more time then to just use another bar chord).

Hence, if you put two experienced musicians in a room and ask them to transcribe a piece of music it is very possible that they would come up with very different solutions (nowadays this is less of a problems since it is often possible to find a video and actually look at the hand of the guitar player when transcribing a song) .
Add the possibility of different turnings (common) and the use of a capo (also very common) into the mix and you have a very tricky problem.