First of all, I have no idea what you mean by point being near a function! I assume you mean near the graph of the function but it is interesting that just defining "near" in this case requires that you know the answer to your question! Fortunately, it is not necessary to assume "near". It is easy to show that the shortest distanced from a point to a line is along the perpendicular from that point to that line. If a line from a point to a curve gives a 'local' minimum, then it must be perpendicular to the tangent line at the point where it meets the curve and so "perpendicular to the curve" since that is defined as "perpendicular to the tangent line".
For the global problem we have to be a little more careful! For example, suppose f(x) is defined by f(x)= x2- 4 if -2\le x\le 2 and not defined for other values of x (the graph is the parabola with vertex at (0,-4) and x-intercepts (-2,0) and (0,2) but not extending beyond x=-2 or x= 2). If p= (0, 2), then the nearest points on the graph to p are (-2, 0) and (0, 2) but the lines from p to those are not perpendicular to the graph. If we were to define f(x)= x2- 4 only for -2< x< 2, there wouldn't even be a "shortest distance"!
If you are talking about a graph without endpoints, which I suspect is what you intend, then all "global minima" are "local minima" and so are along lines perpendicular to the graph.