There may still be some minor inaccuracies in the statement and the proof of the theorem. I will take another look at this later today to see if I find any. Feel free to post a comment if you find a mistake before I do.
Theorem: Let ##x\in\mathbb R## be arbitrary. Let f,g be arbitrary functions. If g is differentiable at x, and f is differentiable at g(x), then ##f\circ g## is differentiable at x, and
\begin{align}
(f\circ g)'(x)=f'(g(x))g'(x).
\end{align}
(Comment: In this context, "function" means "real-valued function with a domain that's a subset of ℝ").
Proof:
Let ε>0 be arbitrary. We're going to show that there exists a δ>0 such that for all h,
\begin{align}
|h|<\delta\ \Rightarrow\ %\left|\frac{f(g(x+h))-f(g(x))}{h}-f'(g(x))g'(x)\right|<\varepsilon
\left|\frac{f\circ g(x+h)-f\circ g(x)}{h}-f'(g(x))g'(x)\right|<\varepsilon
\end{align} This will prove both that ##f\circ g## is differentiable at x, and that we have the right formula for ##(f\circ g)'(x)##. We start by defining a notation. For each real number x and each function u that's differentiable at x, define
\begin{align}
R_{u,x}(h)=u(x+h)-u(x)-hu'(x)
\end{align} for all h such that x+h is in the domain of u. Note that ##R_{u,x}(0)=0## and that
\begin{align}
\frac{R_{u,x}(h)}{h}=\frac{u(x+h)-u(x)}{h}-u'(x)\rightarrow 0\text{ as }h\rightarrow 0.
\end{align} Let h be arbitrary. Define ##k=hg'(x)+R_{g,x}(h)##. Since g is differentiable at x, and f is differentiable at g(x), we have
\begin{align}
f(g(x+h)) &=f\big(g(x)+hg'(x)+R_{g,x}(h)\big)
=f(g(x)+k)\\
&=f(g(x))+kf'(g(x))+R_{f,g(x)}(k).
\end{align}This implies that
\begin{align}
&\frac{f\circ g(x+h)-f\circ g(x)}{h} =\frac{f(g(x+h))-f(g(x))}{h}\\
&=\frac{f(g(x))+kf'(g(x))+R_{f,g(x)}(k)-f(g(x))}{h}\\
&=\frac{\big(hg'(x)+R_{g,x}(h)\big)f'(g(x))+R_{f,g(x)}(k)}{h}\\
&=f'(g(x))g'(x)+f'(g(x))\frac{R_{g,x}(h)}{h}+\frac{R_{f,g(x)}(k)}{h},
\end{align} which implies that
\begin{align}
&\left|\frac{f\circ g(x+h)-f\circ g(x)}{h}-f'(g(x))g'(x)\right|
=\left|f'(g(x))\frac{R_{g,x}(h)}{h}+\frac{R_{f,g(x)}(k)}{h}\right|\\
&\leq |f'(g(x))|\left|\frac{R_{g,x}(h)}{h}\right|+\left|\frac{R_{f,g(x)}(k)}{h}\right|.
\end{align}
We're going to show that there exists a δ>0 such that each of the two terms above is <ε/2 when |h|<δ. The first term presents no difficulties. We just choose ##\delta_1>0## such that
\begin{align}
|h|<\delta_1\ \Rightarrow\ \left|\frac{R_{g,x}(h)}{h}\right| <\frac{\varepsilon}{2|f'(g(x))|}.
\end{align} The second term is much more difficult to deal with. If k=0, we have
$$\left|\frac{R_{f,g(x)}(k)}{h}\right|=0
<\frac{\varepsilon}{2}
$$ for all h. If k≠0, we have
\begin{align}
\left|\frac{R_{f,g(x)}(k)}{h}\right|
&=\left|\frac{R_{f,g(x)}(k)}{k}\right|\left|\frac{k}{h}\right|=
\left|\frac{R_{f,g(x)}(k)}{k}\right|\left|g'(x) +\frac{R_{g,x}(h)}{h}\right|\\
&\leq \left|\frac{R_{f,g(x)}(k)}{k}\right|\bigg(|g'(x)|
+\left|\frac{R_{g,x}(h)}{h}\right|\bigg).
\end{align} Choose ##\delta_2>0## such that
\begin{align}
|h|<\delta_2\ \Rightarrow\ \left|\frac{R_{g,x}(h)}{h}\right|<|g'(x)|.
\end{align} Choose ##\delta_3>0## such that
\begin{align}
|k|<\delta_3\ \Rightarrow\ \left|\frac{R_{f,g(x)}(k)}{k}\right| <\frac{\varepsilon}{4|g'(x)|}.
\end{align}Choose ##\delta_4>0## such that
$$|h|<\delta_3\ \Rightarrow |k|<\delta_3.$$ This is possible because
$$|k|=|hg'(x)+R_{g,x}(h)|=|g(x+h)-g(h)|,$$ and g is continuous at x. These choices ensure that if k≠0, then for all h with ##|h|<\min\{\delta_2,\delta_4\}##, we have
\begin{align}
\left|\frac{R_{f,g(x)}(k)}{h}\right|\leq
\left|\frac{R_{f,g(x)}(k)}{k}\right|\bigg(|g'(x)|
+\left|\frac{R_{g,x}(h)}{h}\right|\bigg) <\frac{\varepsilon}{4|g'(x)|}2|g'(x)|
<\frac{\varepsilon}{2}.
\end{align} If we define ##\delta=\min\{\delta_1,\delta_2,\delta_4\}##, then for all real numbers h such that ##|h|<\delta##,
\begin{align}
&\left|\frac{f\circ g(x+h)-f\circ g(x)}{h}-f'(g(x))g'(x)\right|\\
&\leq |f'(g(x))|\left|\frac{R_{g,x}(h)}{h}\right|+\left|\frac{R_{f,g(x)}(k)}{h}\right|
<\frac{\varepsilon}{2}+\frac{\varepsilon}{2} =\varepsilon.
\end{align}
