SchroedingersLion
- 211
- 56
- TL;DR
- Need some verification or corrections.
Greetings,
suppose we have 3d vectors ##\mathbf{x}_k, \mathbf{y}_k, \mathbf{b}## for ##k=1,...,N## and a 3x3 matrix ##\mathbf{W}## with real elements ##w_{i,j}##.
Are the following two results correct?
$$
\frac{\partial}{\partial \mathbf{b}} \sum_k ||\mathbf{Wx}_k+\mathbf{b}-\mathbf{y}_k||² = \sum_k 2(\mathbf{Wx}_k+b-\mathbf{y}_k)
$$
$$
\frac{\partial}{\partial w_{i,j}} \sum_k ||\mathbf{Wx}_k+\mathbf{b}-\mathbf{y}_k||² = \sum_k 2 (\mathbf{Wx}_k+\mathbf{b}-\mathbf{y}_k)\cdot
\begin{pmatrix}
0 \\
... \\
0 \\
x_{k,j}\\
0\\
...\\
0
\end{pmatrix}
$$
where the nonzero entry in the column vector is in row ##i## and where ##x_{k,j}## is the ##j-th## component of vector ##\mathbf{x}_k##.
Calculating the scalar product gives
$$
\sum_k 2(\sum_{n=1}^{3} w_{i,n}x_{k,n} +b_{i} - y_{k,i})x_{k,j}
$$
suppose we have 3d vectors ##\mathbf{x}_k, \mathbf{y}_k, \mathbf{b}## for ##k=1,...,N## and a 3x3 matrix ##\mathbf{W}## with real elements ##w_{i,j}##.
Are the following two results correct?
$$
\frac{\partial}{\partial \mathbf{b}} \sum_k ||\mathbf{Wx}_k+\mathbf{b}-\mathbf{y}_k||² = \sum_k 2(\mathbf{Wx}_k+b-\mathbf{y}_k)
$$
$$
\frac{\partial}{\partial w_{i,j}} \sum_k ||\mathbf{Wx}_k+\mathbf{b}-\mathbf{y}_k||² = \sum_k 2 (\mathbf{Wx}_k+\mathbf{b}-\mathbf{y}_k)\cdot
\begin{pmatrix}
0 \\
... \\
0 \\
x_{k,j}\\
0\\
...\\
0
\end{pmatrix}
$$
where the nonzero entry in the column vector is in row ##i## and where ##x_{k,j}## is the ##j-th## component of vector ##\mathbf{x}_k##.
Calculating the scalar product gives
$$
\sum_k 2(\sum_{n=1}^{3} w_{i,n}x_{k,n} +b_{i} - y_{k,i})x_{k,j}
$$