Final year University Project, Music Visualisation question

In summary, the individual is a computer science student who is starting their final year in September. They are looking to do an independent final year project related to their interests, particularly in generating an image from a piece of music. They are unsure of the metrics to use, but are considering rhythm, lyrics, velocity, and harmonics. The project must have success criteria and could involve surveying participants. They have some experience in programming in Java, Haskell, and C, but are unsure if this project would be too difficult for them in the given time frame. They are open to simplifying the project if necessary and are considering doing it in real time. They are seeking advice on languages and software to use, with a suggestion to try processing.
  • #1
ellipsoid
2
0
TL;DR Summary
Would generating an image from a given piece of music be too difficult and too much work for a final year project and what languages and software would be best for it?
Hi there. I start my final year of my computer science degree this September and, alongside other modules, will do an independent final year project. I have not had very much CS experience outside my degree so I am a bit unsure what would be feasible for me while not being too trivial such that I receive a poor grade.

I was looking to do something related to my interests and one of my initial ideas that my tutor seemed to like was generating an image from a given piece of music. I am not yet fully sure about what metrics I would use to do this, but I was thinking a combination of rhythm, lyrics, velocity and harmonics.

For a general example, an aggressive sounding piece of music would result in an aggressive feeling image, a calming song, a calm feeling image and so on. The project must have some success criteria that can be evaluated and I was thinking I could survey participants and ask whether the generated image fits with the song.

In the past two years in my degree, I have programmed a fair bit in Java, Haskell and a tiny bit in C. Would you say that this idea for a project would be two difficult to do in the roughly 6 months of time given considering my perhaps more limited experience? Also could you tell what language(s) you would recommend something like this be done in as well as existing packages and/or software that could be useful for this?

I was thinking that with a given audio file, I could do some web retrieval of existing images and combine it with some abstract procedurally generated images however if you think doing both would be too much or if one of these would be too difficult then I can simplify?

Also, my initial plan was for this to be done in real time with visual elements coming in as a piece is playing but once again, if this would be too difficult to achieve good results with, I can rethink that? I guess I have a few questions here but I would appreciate any thoughts on any of this and am happy to provide more info where I can. Thank you very much.
 
Technology news on Phys.org
  • #2
Don't tackle any kind of "live" image.

So minimally, you need to read the audio file and extract your basic input: rhythm, velocity, harmonics. Don't try to do anything with lyrics from the audio source.

I would suggest doing this very, very incrementally. Your main requirement it to get something done and demonstrable in 6 months.

I don't know what development tools are available to you. If you have C, you know how to compile, build, and execute - that's probably the best.

So <with C notes>...
Step 1: Read a wave file <fopen fread> one 1024-sample frame at a time are report out the average amplitude each frame <printf>. No GUI. No stereo. If your favorite music is MP3, convert it to wav. Simple text output, one text output line per frame.

Step 2: If you want rhythm and velocity, you're going to have to play with the data and look for it. Perhaps isolating the base with a simple low pass filter and then looking for amplitude variations will give you the beat. Hopefully, you'll find that fun.

Step 3: If you want harmonics, you need FFT. As an example, if your 1024-sample frames are numbered 1 through 25, then I would try taking them in overlapping two's (Frames 1 and 2, frames 2 and 3, etc) and then doing an FFT on each 4096-sample block and find the peaks - which will correspond to the audio frequencies.

After each step above, if you find yourself with less than 3 months left to go, stop and start the image part.

Since I have no idea where your talents lie, I have no idea if this is doable for you - or if it is too simple.
 
  • Like
Likes Vanadium 50
  • #3
Last edited:
  • Like
Likes robphy
  • #4
ellipsoid said:
Would generating an image from a given piece of music be too difficult
Not intrinsically.

ellipsoid said:
and what languages and software would be best for it?

Now I am more worried. If you are asking this basic a question, I think another project might be wiser.
 
  • Like
Likes berkeman
  • #5
You should use whatever language you are most familiar with.

Basically you are trying to assign values to rhythm, loudness, frequency and changes in these to generate a pattern of dots, possibly a line at a time based on a time interval. Perhaps with the commonest frequencies shown as wave forms. The choice of programming language is almost irrelevant. Unless one language has an established way to extract data from an audio file.

And as stated above, do little things one at a time then combine them, then expand to more than a few seconds.
 
  • #6
Welcome to PF. :smile:

ellipsoid said:
I was looking to do something related to my interests and one of my initial ideas that my tutor seemed to like was generating an image from a given piece of music. I am not yet fully sure about what metrics I would use to do this, but I was thinking a combination of rhythm, lyrics, velocity and harmonics.
Is there a possibility that you can team up with an art major in a combined project? Ideally if he/she needs to do a senior project as well, this could be a good project to collaborate on. For example, in your initial brainstorming with your collaborator, you could ask them to sketch what they feeling as they listen to some parts of some musical pieces, and together you may be able to start to put some rules/themes in place. Without the help of an artistic person on this project, I have a feeling that it will not go well for you.

ellipsoid said:
I was thinking that with a given audio file, I could do some web retrieval of existing images
As an extension of the above, once you start to get some agreement on the resulting inspired skeches/drawings from the content of the audio files, you could start to do some machine learning where you program associations between the music and the strength of the matches in what you retrieve from the web. At first you could use simple Google Images searches for images that contain "fast" or " intense" or "rhythmic", etc., and then you may be able to use your collaboration to get more sophisticated combined searches that give better image hits.

ellipsoid said:
Also, my initial plan was for this to be done in real time with visual elements coming in as a piece is playing
So that is an important thing to decide going in, IMO. It seems more natural for the images that you display to be updated every 10-15 seconds or so, based on the current mood of the music. Most pieces move around a bit over the course of a complete piece, I think, so just distilling one image for the whole piece seems unsatisfying for me.

OTOH, you may need to have the ability to detect sudden changes in the music and adjust quickly to a new image. That would also imply that this is not done in real time, but is done more like a compiler processes the input program. Think of the piece "In the air tonight" by Phil Collins at 2:38 and of course at 3:15. :smile:

 
  • #7
Also, there may be some value in getting the associated lyrics somehow (subtitles? Are synchronized lyrics routinely available for musical pieces?) and making those words and phrases also Google search terms to find images that illustrate those words and phrases. So when Phil sings "I can hear it coming in the air tonight...", that could pull up images of forests at night, or similar... :smile:
 
  • Like
Likes hmmm27
  • #8
Sorry, just one more post for now. It reminds me of the time back in high school when I was listening to a lot of Santana Abraxas on an old reel-to-reel tape player, and my best friend Andy came over. I put the headphones on him and played "I hope you're feeling better" at high volume, and told him to watch when I opened a motocross magazine (to a picture of MX champion Joel Robert on a high speed MX jump) at 0:09 in the song... He was pleased. :smile:



1658798824738.png

https://en.wikipedia.org/wiki/Joël_Robert
 
Last edited:
  • #9
berkeman said:
Santana Abraxas
The third album was better. :wink:

Everybody's Everything is spectacular (trumpets Mic Gillette and Gref Adams) as is Everything's Coming Our Way (especially the organ solo - Gregg Rolie?)
 
  • Like
Likes berkeman
  • #10
berkeman said:
Also, there may be some value in getting the associated lyrics somehow (subtitles? Are synchronized lyrics routinely available for musical pieces?) and making those words and phrases also Google search terms to find images that illustrate those words and phrases. So when Phil sings "I can hear it coming in the air tonight...", that could pull up images of forests at night, or similar... :smile:
So I just googled rhythm and frequency and switched to images. Lots of interesting ideas there, especially under the stock image collections' websites. You could even try generating an individual image for rhythm, loudness, frequency, changes in each of these for each piece of music. That gives you options in your project description to have, say, two stated goals (say rhythm and frequency) and then follow-up goals (say loudness vs frequency or rhythm vs changes in rhythm).

It's always a good idea to avoid too wide a range of stated goals in a student project and end up leaving several with no evidence of attempts at getting them to work. But extra features are better, as if time runs out, you just don't worry. And if you get the first goals completed to a reasonable standard, you can afford to switch to an optional goal even if you do not complete it, but can still include it in the report.

IE Must have, Should have, Could have, Wont have (google moscow rules for programming)
Having follow-up goals also shows you don't have limited ideas, just limited time (the usual problem for all student projects ;) )
 

Related to Final year University Project, Music Visualisation question

1. What is a Final Year University Project?

A Final Year University Project is a project undertaken by a student in their final year of university as a requirement for graduation. It is a culmination of their knowledge and skills acquired throughout their degree program.

2. What is Music Visualisation?

Music Visualisation is the process of creating visual representations of music, often through the use of technology. It involves interpreting musical elements such as tempo, pitch, and timbre to create visual effects that enhance the listening experience.

3. How is Music Visualisation used in a Final Year University Project?

Music Visualisation can be used in a Final Year University Project as a tool to showcase the student's understanding and application of concepts taught in their degree program. It can also be used as a creative outlet to explore the relationship between music and visual art.

4. What are some examples of Music Visualisation in a Final Year University Project?

Examples of Music Visualisation in a Final Year University Project can include creating a computer program that generates visual effects based on live music input, designing a music video that incorporates animated visualisations, or creating a virtual reality experience that immerses the user in a visual representation of music.

5. What skills are required to create a successful Music Visualisation for a Final Year University Project?

To create a successful Music Visualisation for a Final Year University Project, one would need a combination of technical skills such as programming and software proficiency, as well as creative skills in graphic design, animation, and music production. It also requires a strong understanding of music theory and the ability to translate musical elements into visual representations.

Similar threads

  • Programming and Computer Science
Replies
3
Views
1K
  • Programming and Computer Science
Replies
10
Views
1K
  • Programming and Computer Science
Replies
1
Views
633
  • Art, Music, History, and Linguistics
2
Replies
63
Views
3K
Replies
1
Views
718
  • STEM Academic Advising
Replies
4
Views
882
Replies
5
Views
1K
  • Other Physics Topics
Replies
3
Views
1K
  • STEM Academic Advising
Replies
3
Views
265
  • Programming and Computer Science
Replies
4
Views
2K
Back
Top