I need tips and advice for eventually building a synesthetic device

[seeingmusic]

Disclaimer: I have zero electrical engineering/physics/math experience. In fact, I am a painting major. (Laugh it up)

That said, I've been dreaming for quite some time of a very exacting music visualizer which would put all other music visualizers to shame, in that it would have a different visual signal for each and every musical pitch, ranging from low A to high C, as on a piano.

This visualizer isn't software based, though, its tuner-based. I want to modify basic guitar tuners so that their display, which triggers the little LED's which tell you which note you are playing and whether it is in tune, whenever you play a certain pitch to the tuner, would instead trigger a colored light to be turned on, each time a different one, depending on which pitch has been sounded.

I would have 88 lights (for each pitch between low A and high C) arranged from lowest to highest, and 88 modified guitar tuners, each attached to one of those lights. Each guitar tuner is only listening for the note which corresponds to the particular light to which it is attached, because I will have modified the tuner to only trigger the light when pitches of a very small range are input. Ex. there will be a tuner and a light for middle A, 440hz, and the tuner will have been modified to only trigger the display (the light) whenever a pitch in the very close vicinity of 440hz has been detected. But this will also exist for every other note as well.

Does anybody understand this dream of mine? I don't know any of the technical words, phrases, jargon, but I know exactly what I am trying to do. I just need help with learning how to modify the circuit boards of guitar tuners, (or perhaps building my own) and whether these are digital, analog, and how to connect them with LED lights, in order to bring about this interaction. I'm basically clueless about everything except for this vision. Just imagine a concert pianist who is playing Rachmaninoff or Brahms or Beethoven, or ANYTHING, and this elaborate device is positioned above him as he plays, so that the structure of the song is instantaneously revealed to your ears and eyes simultaneously, and in dazzling colors. This device could pick up any songs, whether they emanate from a speaker system or from a live band or from a lone singer, and reveal the structure of the music as it occurs.

 

[berkeman]

Disclaimer: I have zero electrical engineering/physics/math experience. In fact, I am a painting major. (Laugh it up)

That said, I've been dreaming for quite some time of a very exacting music visualizer which would put all other music visualizers to shame, in that it would have a different visual signal for each and every musical pitch, ranging from low A to high C, as on a piano.

This visualizer isn't software based, though, its tuner-based. I want to modify basic guitar tuners so that their display, which triggers the little LED's which tell you which note you are playing and whether it is in tune, whenever you play a certain pitch to the tuner, would instead trigger a colored light to be turned on, each time a different one, depending on which pitch has been sounded.

I would have 88 lights (for each pitch between low A and high C) arranged from lowest to highest, and 88 modified guitar tuners, each attached to one of those lights. Each guitar tuner is only listening for the note which corresponds to the particular light to which it is attached, because I will have modified the tuner to only trigger the light when pitches of a very small range are input. Ex. there will be a tuner and a light for middle A, 440hz, and the tuner will have been modified to only trigger the display (the light) whenever a pitch in the very close vicinity of 440hz has been detected. But this will also exist for every other note as well.

Does anybody understand this dream of mine? I don't know any of the technical words, phrases, jargon, but I know exactly what I am trying to do. I just need help with learning how to modify the circuit boards of guitar tuners, (or perhaps building my own) and whether these are digital, analog, and how to connect them with LED lights, in order to bring about this interaction. I'm basically clueless about everything except for this vision. Just imagine a concert pianist who is playing Rachmaninoff or Brahms or Beethoven, or ANYTHING, and this elaborate device is positioned above him as he plays, so that the structure of the song is instantaneously revealed to your ears and eyes simultaneously, and in dazzling colors. This device could pick up any songs, whether they emanate from a speaker system or from a live band or from a lone singer, and reveal the structure of the music as it occurs.

Sounds like a fun idea, and worth pursuing.

I'd suggest that you check out the EE department at your school, to see if they have some kind of hobbyist group or something similar. Or post a flyer asking if there are any creative EE types who might like to collaborate on a new music-related device. You really need someone there that you can bounce some of your ideas off of.

The person you link up with should have built some similar hobby stuff before (ask to see some of their work), and should be versed in the use of microcontrollers (like PICs), and in the use of digital signal processing (DSP) to process analog audio signals.

Even Steve Jobs needed Woz to help with the technical stuff when they were starting off...

 

[seeingmusic]

Thanks berkeman. That's good advice. I'll be working on a catchy flyer to post at the University of Houston engineering schools.

Despite my extreme ineptitude I am convinced that I am supposed to create this device.

 

[bob1182006]

So this device is going to be able to decompose music into the fundamental pitches and display it in real time, sort of how a trained ear can listen to music and determine which notes to play on an instrument?
Seems like an interesting product and great project.

You might also want to speak with the EE department and see if the students do a final year project, they usually welcome project ideas.
I'd be really interested in doing this as my senior project but I'm not at the University of Houston.

 

[Attheveryend]

SO I've been having similar machinations, although I see taking this as far as an implant that regulates sensory functions, but that's a bit of a stretch.

Unfortunately, guitar tuners or really any tuner is not able to distinguish between multiple pitches. It simply reports the average pitch, but only if the average remains constant for long enough to establish a trend. Usually those devices report nonsense if there is any ambiguity to the sound. IN summary, you can not use a guitar tuner to pick apart the individual pitches of a song. It hears them all at once and averages them.

Theoretically you could wire a guitar tuner into a computer and use it as a microphone, but this is not better than using a microphone, or simply having the computer play it and analyse the waveform digitally from start to finish.

You will probably want to start with a piece of software that applies a fast Fourier transform to the entire audio channel so that it can pick apart all of the constituent pitches. From there you may assign a functional relationship between diatonic pitches, colors, positions, shapes, 3d structures, etc. that produces visuals to your liking.

If you wish, you can take this idea in another direction and try to superimpose the pitch data from one instant of the song with the instant just before or after it and then port the combined data to the visual engine in order to create a kind of interferometric visual.

Whatever you decide, you must recognize that this endeavor is almost entirely a mathematical pursuit. There is a lot to computer graphics generation, even in only two dimensions. In order to make this device a success you will need physics, math, and computer programming knowledge.
Specifically, you will need to know the physics of wavelike motion, sound, differential calculus, Fourier analysis, audio/visual programming, and some understanding of maxwell's equations in order to come away from this project with a working prototype.

Or you can just convince people who know the above things to do it for you.

 

[dlgoff]

Way back in the '70s, one of the EE students project was designing and building what he called a "dollyscope". If I remember correctly, the design was entirely analog; filters and amplifiers built from discrete components. And for the chalk-board size display, he used an array of fairly large lenses to spread the light from small but very bright incandescent lamps behind each. In front of the lenses was a white translucent screen. I'm thinking there were at least 88 channels to which when demonstrating with http://www.youtube.com/watch?v=_4IRMYuE1hI", blew the crowd away.

 

[seeingmusic]

@Attheveryend -- My understanding was that most software with fast Fourier transforms was similarly limited in their efficiency at distinguishing and picking apart various constituent pitches. Most audio signals from a microphone would be rather noisy. Then again, like I said my understanding is very limited. In addition I would prefer to create a physical, analog device which interprets live signals rather than digital bytes.

@digoff -- That's remarkable and sounds very akin to what I am trying to accomplish. It is my understanding that there are very exacting "tuners" that exist, admittedly not standard guitar tuners, whose microphones are much more sensitive, but that you would still be able to limit, to the point that they would only trigger displays when a certain note has been detected.

 

[jim hardy]

interesting thoughts...

what you are doing is analagous to how we hear - the cochlea has an array of frequency sensitive nerve endings that do just what you describe.

electronic filters are capable of that but the difference between adjacent notes is so small it takes exotic filters to do the job.

there's a National Semiconductor application note AN779 that's an introduction to filters.

but with the marvels of synchronous detection
and the IC's that are available nowadays for electronic musical instruments

sounds like fun...

anybody who's lived with a piano has heard its strings go into sympathetic vibration with sounds in the room. same principle as the cochlea...
http://hyperphysics.phy-astr.gsu.edu/hbase/sound/cochlea.html
http://hyperphysics.phy-astr.gsu.edu/hbase/sound/corti.html#c1
you are i think toying with the idea of an electronic "organ of the corti" ...

 

[dlgoff]

interesting thoughts...

... it takes exotic filters to do the job.

I believe it was 1968 when I had the pleasure to attend a small EE seminar at the University of Kansas where the guest lecturer was http://en.wikipedia.org/wiki/Robert_Moog" sounded.

Later he ended up http://www.google.com/patents?id=IYxjAAAAEBAJ&pg=PA1&source=gbs_selected_pages&cad=2#v=onepage&q&f=false" his low-pass and high-pass filters.

This looks exactly like how I remember it.