Fear of public speaking is rarely addressed in academia. And yet it's vitually impossible to pursue a professorship without speaking at department colloquia, conferences, giving lectures, and of course - the dreaded oral exams.
Personally I find speaking at conferences the most intimidating. You have to stand in front of sometimes hundreds of people - most of whom are experts in your field, some of whom are working on the exact same problem, and a few of whom will challenge you with unexpected questions.
There are things you can do to help you with your fear. I might recommend taking a course in public speaking, or failing that some kind of dramatic arts. You could also join or start a Toastmasters chapter (although I personally found the only Toastmasters meeting I attended rather annoying). Either way, the more you practice, the easier it gets. Start with small, familiar groups and expand from there.
If it helps, I consider myself a confident public speaker, but I still get very nervous before giving a talk.