BrainModel: Audio Analysis/Synthesis Software, FFT/Spectograms, Mic/Spkr

[neurocomp2003]

Not sure if this belongs in this forum, its subforums or engineering.
Thanks for reading, the post is long. Any suggestions would be grateful.
-------------------
Application Field: Brain Modelling
Topic: Audio Analysis then Audio Synthesis
OS: Microsoft Windows,
IDE: MSVC (unless the package is for cygwin that is workable in MSVC)
Language: C/C++ [HPC/cluster reasons]
-------------------
QUESTIONS:
[Q1]: What packages to use to make a spectrogram? And am i on the right track?
[Q2]: is spectrogram or scaleogram better for the task below?
[Q3]: is onboard soundcard capable of the task below? [card: sigmatel HD audio codec]
[Q4]: what format is the input sound signal to FFT algorithm?
[Q5]: if the FFT algorithm outputs (Re,Im) pairs, does a spectrogram use the I am values.
-------------------
Desire: [Phase 1] I need to embed a spectrogram[t v Hz v Amp?] from realtime sound(microphone) or loaded file(wav or mp3) into a C/C++ GUI application(currently wx) to perform audio analysis whereby the spectrogram or some arrangement of the data will be later fed into a neural network to mimic hair cells (which includes vision). [Phase 2] Once phase 1 is setup, I then need to perform audio synthesis via phonemes and basic music oral sounds to mimic the vocal tract or some higher level area.

OR perhaps i should reverse the tasks.
-------------------
What I know:
-my sound card is sigma tel high definition audio codec
-the spectrogram process: get sound buffer--> feed to FFT/wavelet --> feed to GUI
-how to load/play/save wav/mp3 sounds via packages (openal, directx/midi, RT, portaudio,SDL)

-------------------
What I've Tried:
- [still working through] marsyas library , sndobj library but both give fuzzy output signal via my soundcard, and neither seems to access the hardware simultaneously for both input(mic) and output(spk)
- I've looked at CLAM++ but sadly i require a MSVC compatible library because i rely on MSVC IDE
-------------------
What i'd prefer not to use:
-speech specific software like SAPI or the one from CMU, though i probably need too.
-synthesis specific software like STK.
-------------------
What I'm looking for in the packages
-open source?
- I'd rather use a package capable of both analysis and synthesis which includes some FFT and midi capability. Though presumably one can get the FFT elsewhere but it seems its included in linux APIs. Hopefully the API is easy to use with pseudo code looking like

[] GetMic(...)/LoadSoundFile(...)
[] GetSoundBuffer(...)
[] CallingFFT(...)

- also being able to insert frequencies with duration and timbre would be useful if the package doesn't have midi.

-pointing me to a newsgroup/mailgroup would also help
-------------------
thank you again for reading,

 

[Evalesco]

It sounds like you have done your research and have a good understanding of what you need to accomplish. Based on the information you provided, it looks like you may benefit from using a library like CLAM++, which is compatible with MSVC and has both FFT and MIDI capabilities. Additionally, you may want to look into other libraries that offer audio analysis tools, such as LibROSA or LibSndFile. Finally, there are helpful discussion forums for audio programming related topics, such as the Sound Programming Forum or the Audio Programming Forum. I hope this helps!