A lot of confused talk in this thread.
First, here is the standard brain to body ratio table...
http://books.google.co.nz/books?id=..._t3aAw&sa=X&oi=book_result&ct=result&resnum=8
You will see humans are indeed an outlier. Dolphins, our nearest competition, are pygmy whales and so have larger brains proportionately for that reason (or so it can be argued).
You also need to bear in mind that brains are scaled according to a powerlaw across the primitive-to-advanced regions. So prefrontal cortex in particular is massively larger proportionately in humans. And prefrontal is important to the kind of fluid intelligence we get excited about.
What is intelligence? Well it is not really abstract problem solving. Biologically speaking, it is anticipation - the ability to predict and thus control your world. Humans like any other animal are optimised by evolution for this, rather than doing maths or even raven's matrices.
Of course, human society now values new kinds of mental activities, quite specialised ones. So some very mal-adapted people (mathematicians I have known who can barely function in the real world) may be very good at some particular tasks.
Does this make them "intelligent" in the proper sense?
To be fair, the real successes in the field of abstract problem solving tend to be actually competent across the board. They certainly need high social intelligence to win in the academic game. The other kind tend to be smart, yet not creative. They can apply rules but not invent rules.
So what is actually different about intelligent brains? There is no simple answer - because the brain is a complex device. Like a racing car, there are many factors to tune and most cut across each other. Pushing the envelope here screws something else over there, so the overall performance is worse.
I think the best answers are likely to come from the study of network dynamics - so for example, the study of scalefree nets. A tuning of connectivity to an "edge of chaos" balance that optimises both the stability and plasticity of the system. The capacity for the routine and also the creative.
This is where you start to get down to some meaningful "raw" metrics to relate brain organisation to brain performance.
There are other aspects to this as well. So for instance, the ability to suppress connections is as crucial as making connections when thinking. A smart mind is a high contrast mind that is sifting wheat from chaff efficiently.
There is a weak correlation of brain size to IQ scores. A somewhat strong correlation of IQ and reaction times. And of the network dynamics approach is correct, then there should be a good correlation between IQ and scalefree neural connectivity - but that is not something I've yet seen any story on how to measure.