Numerical computation of the derivative

[valenumr]

I think it is important to see the proposed formula more in light of classical "macroscopics", i.e. not trying to get at the highly accurate microscopic detail. I personally do not know whether the formula is of much use or not. We see the same "classical" problem of "loss of scale" that can haunt Newton's method for finding zeros. I only meant to point out that the proposal has some appeal in a "big picture" way. It is worth bearing in mind that classical Maxwell theory had to give way to quantum mechanics as ways of addressing microscopic behaviors became more relevant and accessible. But I don't think that implies "lack of utility" as there are instances where we want this sort of picture, and as we well know, Newton's method works pretty nicely in lots of cases despite grievous deficiencies.

Well, for sure you can have functions that just don't work well with basic numerical approaches. If it is something high frequency, for example, and you try to compute a derivative that should be at a maxima or minima, it won't work well with a standard approach, but might be better with a +/- dh. But I think that really is a more fundamental problem of scale and numerical accuracy in the simulation. I think it's reasonable (I haven't gotten deep on the math too much) that the extra error term (say h²) in a quadratic gets erased doing it this way, but in a lot of cases, this term will be much smaller than f(x) to the point that it is outside of machine precision. Obviously not always. But if it does become problem I don't think the method of taking the derivative is the issue. It would be better to scale the function appropriately if at all possible to get more accurate results.

 

[pbuk]

I haven't gotten deep on the math too much

<sigh> then what is the point in commenting? Numerical analysis without the analysis is just guessing.

the extra error term (say h²) in a quadratic gets erased doing it this way

No, I repeat, the point in a two-sided calculation is that the $f''(x)$ term gets erased is eliminated. Where $f''(x) \gg f'(x)$, which is of course true at an extremum, this leads to better accuracy. However where this is not true the two-sided calculation can be less accurate, because you are doubling the effect of $\epsilon$.

 

[mgeorge001]

Well, for sure you can have functions that just don't work well with basic numerical approaches. If it is something high frequency, for example, and you try to compute a derivative that should be at a maxima or minima, it won't work well with a standard approach, but might be better with a +/- dh. But I think that really is a more fundamental problem of scale and numerical accuracy in the simulation. I think it's reasonable (I haven't gotten deep on the math too much) that the extra error term (say h²) in a quadratic gets erased doing it this way, but in a lot of cases, this term will be much smaller than f(x) to the point that it is outside of machine precision. Obviously not always. But if it does become problem I don't think the method of taking the derivative is the issue. It would be better to scale the function appropriately if at all possible to get more accurate results.

I think your point about scale is significant. Thanks. You have a mature perspective on this, that's obvious. A lot of this kind of work boils down to assumptions about small/small: That's a tricky 0/0 issue, but with a lot of work, like being able to reliably neglect say an h^2, I think numerical experts can address the difficulties meaningfully. The 0/0 problem vis a vis logs is equivalent to the infinity - infinity problem. Both enter the picture sometimes.

 

[valenumr]

<sigh> then what is the point in commenting? Numerical analysis without the analysis is just guessing.No, I repeat, the point in a two-sided calculation is that the $f''(x)$ term gets erased is eliminated. Where $f''(x) \gg f'(x)$, which is of course true at an extremum, this leads to better accuracy. However where this is not true the two-sided calculation can be less accurate, because you are doubling the effect of $\epsilon$.

Perhaps I misread or misinterpreted that as saying the term is eliminated when $f''(x)$ is zero, which over an interval is just a straight line, and isn't that interesting. So I meant to say, and didn't express it well, that I'm pretty sure if $f''(x)$ is some constant, then the answer should be exact (within machine rounding precision).

For example, you can think of the symmetrical derivation function in terms of the mean value theorem. It is essentially computing a secant line between two points on f. MVT says there is an x' between those two points whose tangent line is parallel to the secant. It doesn't necessarily mean that x' == x, but in the case of a parabola, you can show that for any choice of h, x' will always be equal to x, or in other words, the two endpoints of the secant line will always be equidistant from the value x whose tangent line is parallel.

So when I say I didn't go into the math more deeply, I just meant that I didn't make any an attempt to prove the statement: "if $f''(x)$ is some constant over the evaluation interval, then the answer is exact" for all functions, but I think it is probably true.