How Do We Understand Differentiability and Gradients in Multivariable Calculus?

[sponsoredwalk]

For a function ƒ defined on an open set U having the point X:(x₁,x₂,...,x_n)
and the point ||H|| such that the point X + H lies in the set we try to
define the meaning of the derivative.

$$\frac{f(X \ + \ H) \ - \ f(X)}{H}$$ is an undefined quantity, what does it mean
to divide by a vector?

The idea is to use

$$f'(x) \ = \ \lim_{h \to 0} \frac{f(x \ + \ h) \ - \ f(x)}{h}$$

and define

$$g(h) \ = \ \lim_{h \to 0} \frac{f(x \ + \ h) \ - \ f(x)}{h} \ - \ f'(x)$$

for h ≠ 0 but notice that $$\lim_{h \to 0} \ g(h) \ = \ 0$$.

If we then write

$$g(h) \ + \ f'(x) \ = \ \frac{f(x \ + \ h) \ - \ f(x)}{h}$$

$$g(h) \cdot h \ + \ f'(x) \cdot h \ = \ f(x \ + \ h) \ - \ f(x)$$

this holds true as long as h ≠ 0 because if h = 0 the derivative term, f'(x) makes no sense.

If the above relation holds for a function whether we've chosen h to be either positive or
negative we can then claim the function differentiable seeing as the limit exists
& the relation holds true. Therefore h can be rewritten as |h| and the above as

$$g(h) \cdot |h| \ + \ f'(x) \cdot h \ = \ f(x \ + \ h) \ - \ f(x)$$

(My book only gives the h indicated above with |h| & I'm not sure why Lang doesn't give
the h beside the f'(x) absolute value signs one too? I think it's because he divides in a
minute by h & wants to get rid of the h beside the f'(x) & leave the g(h)•h one but it seems
to me like picking & choosing things as you please...)

So, if the function is differentiable there exists a function g such that

$$g(h) \cdot |h| \ + \ f'(x) \cdot h \ = \ f(x \ + \ h) \ - \ f(x) \ , \ \lim_{h \to 0} \ g(h) \ = \ 0$$

so we can define the following

$$g(h) \cdot \frac{|h|}{h} \ + \ f'(x) \ = \ \frac{f(x \ + \ h) \ - \ f(x)}{h}$$

and in the limit we find equality above as the g(h) term makes the fraction of |h|/h
disappear.

Is all of this correct? What is with the absolute value situation? Why are we picking &
choosing our definitions with |h| when it's convenient? (Or, more accurately - what am I not getting? )

Thanks for taking the time to read that!

 

[Hurkyl]

This exposition is very confusing; I think they're trying to give an "intuitive" reason why we would choose the definition of derivative in a particular way, rather than a more rigorous argument.


IMO, differential approximation is one of the cleanest ways to deal with these things. We want:

$$f(\vec{x} + \vec{y}) \approx f(\vec{x}) + f'(\vec{x}) \vec{y}$$​

And so we insist that the error term vanish as $\vec{y} \mapsto 0$ more rapidly than $\vec{y}$ itself. So:

$$\lim_{\vec{y} \mapsto 0} \frac{f(\vec{x} + \vec{y}) - f(\vec{x}) - f'(\vec{x}) \vec{y}}{|\vec{y}|} = 0$$​

or equivalently

$$\lim_{|\vec{y}| \mapsto 0} \frac{|f(\vec{x} + \vec{y}) - f(\vec{x}) - f'(\vec{x}) \vec{y}|}{|\vec{y}|} = 0$$​

($f'(\vec{x})$ should be a "covector" -- something that maps vectors to numbers in a special way -- but any such thing can be written instead as a dot product with a vector, which is why you usually see vectors and dot products instead)



Now that I looking over again the expression from the book, it's not so bad. Everything before your parenthetical makes sense for real-valued functions of a single variable. His goal is to find a characterization of the derivative that would also make sense for the multi-variate case. Of course, he has the benefit of knowing the "right" answer beforehand. Note that for univariate functions,

$$\lim_{h \rightarrow 0} \frac{g(h)}{h} = 0$$​

is true if and only if

$$\lim_{h \rightarrow 0} \frac{g(h)}{|h|} = 0$$​

is true. Also, these are true if and only if

$$\lim_{h \rightarrow 0} \frac{|g(h)|}{|h|} = 0$$​

is true. So the three formulations of derivative are equivalent.

The only other reasonable guess I see would be to define the error term g(h) as a vector, and have g(h).h as a dot product. But if you think about it, you'll see this is unsatisfactory because this says we could have massive error in our differential approximation, as long as it's perpendicular to h.

 

[sponsoredwalk]

I think I get what you're talking about, I'm reading Lang's Multivariable calculus book & it's a
bit unconventional in that vectors are written as X instead of $$\overline{x}$$ and
other little things I've noticed. I'll read a more modern book later

I take it that $$\overline{y}$$ in your post corresponds with H in mine & that you're
method is a vector version of single variable differential approximation.

I like what Lang did here, I've read less rigorous expositions on multivariable derivatives & they
confused a little but I'm liking this, he's extended the concept of the single variable
derivative for vectors by doing all of what I posted in my first post then redefined things.

When I take the limit in the last latex frame of my first post this is just Lang giving a
justification that it does exist in parallel to the single variable method, but then he says

"Therefore, the existence of a number a=f'(x) and a function g satisfying
f(x + h) - f(x) = f(x)h + |h|g(h) with lim h-->0 g(h) = 0
could have been used as the definition of differentiability in the case
of functions of one variable. The great advantage of this is that no h
appears in the denominator. It is this relation which will suggest to us
how to define differentiability for functions of several variables, and how
to prove the chain rule for them. "

Lang then redefines everything in terms of vectors
f(X + H) - f(X) = ∇f(X)•H + |H|g(H)

So the idea is that the |H|g(H) dissappears in the limit is it? This would answer your
worry about the dot product masking the huge error in the differential approximation.
I mean he hasn't explicitly said it yet but it seems to be the case.

 

[HallsofIvy]

For a function ƒ defined on an open set U having the point X:(x₁,x₂,...,x_n)
and the point ||H|| such that the point X + H lies in the set we try to
define the meaning of the derivative.

$$\frac{f(X \ + \ H) \ - \ f(X)}{H}$$ is an undefined quantity, what does it mean
to divide by a vector?

It doesn't mean anything to divide by a vector. You should be dividing by the length of the vector:
$$\frac{f(x+ H)- f(x)}{||H||}$$

The idea is to use

$$f'(x) \ = \ \lim_{h \to 0} \frac{f(x \ + \ h) \ - \ f(x)}{h}$$

and define

$$g(h) \ = \ \lim_{h \to 0} \frac{f(x \ + \ h) \ - \ f(x)}{h} \ - \ f'(x)$$

for h ≠ 0 but notice that $$\lim_{h \to 0} \ g(h) \ = \ 0$$.

If we then write

$$g(h) \ + \ f'(x) \ = \ \frac{f(x \ + \ h) \ - \ f(x)}{h}$$

$$g(h) \cdot h \ + \ f'(x) \cdot h \ = \ f(x \ + \ h) \ - \ f(x)$$

this holds true as long as h ≠ 0 because if h = 0 the derivative term, f'(x) makes no sense.

yes, the derivative is defined as the limit as h goes to 0, not the value when h= 0.

If the above relation holds for a function whether we've chosen h to be either positive or
negative we can then claim the function differentiable seeing as the limit exists
& the relation holds true. Therefore h can be rewritten as |h| and the above as

$$g(h) \cdot |h| \ + \ f'(x) \cdot h \ = \ f(x \ + \ h) \ - \ f(x)$$

(My book only gives the h indicated above with |h| & I'm not sure why Lang doesn't give
the h beside the f'(x) absolute value signs one too? I think it's because he divides in a
minute by h & wants to get rid of the h beside the f'(x) & leave the g(h)•h one but it seems
to me like picking & choosing things as you please...)

So, if the function is differentiable there exists a function g such that

$$g(h) \cdot |h| \ + \ f'(x) \cdot h \ = \ f(x \ + \ h) \ - \ f(x) \ , \ \lim_{h \to 0} \ g(h) \ = \ 0$$

so we can define the following

$$g(h) \cdot \frac{|h|}{h} \ + \ f'(x) \ = \ \frac{f(x \ + \ h) \ - \ f(x)}{h}$$

and in the limit we find equality above as the g(h) term makes the fraction of |h|/h
disappear.

Is all of this correct? What is with the absolute value situation? Why are we picking &
choosing our definitions with |h| when it's convenient? (Or, more accurately - what am I not getting? )

Thanks for taking the time to read that!

You have to define "differentiable" before you can define the "derivative" of a
vector valued function.

In general, we say that function f(x), from Rⁿ to R^m is differentiable at x= a if and only if there exist a linear function L from Rⁿ To R^m and a function $\epsilon(x)$ from Rⁿ to R^m such that:
1) $f(x)= f(a)+ L(x- a)+ \epsilon(x- a)$
and
2) $\lim_{x\to a}\frac{\epsilon(x-a)}{|x-a|}= 0$

Then we show that the L defined above is unique and say that it is the derivative of f at x= a.