No. For an obvious example, consider a vector whose magnitude but not direction is increasing as a function of time: ##\vec{F}(t+\Delta{t})-\vec{F}(t)## points in the same direction as ##\vec{F}(t)##. You're thinking of the case in which the magnitude of the vector is constant over time, in which case the derivative must indeed be perpendicular (as in acceleration in the case of uniform circular motion).
[Edit: Chet got there first but I used more Latex so I still win ]