How to get the derivative of this convex quadratic

[justin_huang]

$\frac{d}{dx}f(x)=\frac{d}{dx}[ \frac{1}{2}x_{}^{T}Qx-b_{}^{T}x]$

how to get this derivative, what is the answer? is there textbook describe it?

 

[Bacle2]

I think this :

https://www.physicsforums.com/showthread.php?t=535601

may help.

 

[DuncanM]

This equation is analogous to a second degree polynomial, so I think the derivative is simply Qx - b.

 

[HallsofIvy]

Not quite. Because multiplication of matrices is not commutative, taking the derivative you have to "symmetrize" Q- the derivative is $[(Q+Q^T)/2]x- b$.

Look at the simple two dimensional situation:
$$\frac{1}{2}\begin{bmatrix}x & y \end{bmatrix}\begin{bmatrix}a & b \\ c & d \end{bmatrix}\begin{bmatrix}x \\ y\end{bmatrix}- \begin{bmatrix}e & f\end{bmatrix}\begin{bmatrix}x \\ y\end{bmatrix}$$
$$= \frac{1}{2}ax^2+ \frac{b+ c}{2}xy+ \frac{1}{2}dy^2- ex- ey$$

The "derivative with respect to x", since x is a vector, is the gradient:
$$\begin{bmatrix}ax+ \frac{b+c}{2}y- e & dy+ \frac{b+c}{2}x- f\end{bmatrix}$$
$$= \begin{bmatrix}a & \frac{b+c}{2} \\ \frac{b+c}{2} & d\end{bmatrix}\begin{bmatrix}x \\ y\end{bmatrix}- \begin{bmatrix}e & f\end{bmatrix}$$

 

[blue_raver22]

The derivative of a convex quadratic function can be obtained by using the chain rule and the properties of matrix derivatives. The first step is to rewrite the function in matrix form as f(x) = 1/2 x^T Q x - b^T x, where Q is a symmetric matrix and b is a vector.

Next, we can use the properties of matrix derivatives to find the derivative of the first term, 1/2 x^T Q x. This can be written as x^T Q x = x^T (Qx) = (Qx)^T x. Using the product rule, we can find the derivative as follows:

d/dx (1/2 x^T Q x) = (Qx)^T x + x^T (Qx)

= x^T (Q^T + Q) x

= x^T (2Q) x

= 2 x^T Q x

Similarly, the derivative of the second term, b^T x, can be found as b.

Therefore, the derivative of the convex quadratic function f(x) is given by:

d/dx f(x) = 2 x^T Q x - b

This is the general formula for the derivative of a convex quadratic function. It is important to note that this formula can also be derived using the definition of the derivative and the properties of matrix multiplication.

There are many textbooks and online resources that describe the process of finding the derivative of a convex quadratic function. Some examples include "Introduction to Linear Algebra" by Gilbert Strang and "Matrix Differential Calculus with Applications in Statistics and Econometrics" by Jan R. Magnus and Heinz Neudecker. It is also helpful to consult with a mathematics or statistics expert for further clarification and understanding.