Positive Definite Matrices and Their Properties

[Fernando Revilla]

I quote a question from Yahoo! Answers

1. Prove that if X ∈ R^(d×n) then XX^T and X^TX are both positive semidefinite.
6. Prove that if X ∈ R^(d×n) has rank d, then XX^T is positive definite (invertible).

I have given a link to the topic there so the OP can see my response.

 

[Fernando Revilla]

For all $x\in\mathbb{R}^{d\times 1}:$
$$x^T(XX^T)x=(X^Tx)^T(X^Tx)=(y_1,\ldots,y_n) \begin{pmatrix}{y_1}\\{\vdots}\\{y_n}\end{pmatrix}=y_1^2+\ldots+y_n^2\geq 0$$
which implies $XX^T$ is positive semidefinite (or positive definite). Similar arguments for $X^TX$.

If $\text{rank }X=d$, then $\text{rank }(XX^T)=\text{rank }X=d$, which implies $XX^T$ is invertible. This means that $XX^T$ is congruent to a matrix $\text{diag }(\alpha_1,\ldots,\alpha_d)$ with $\alpha_i>0$ for all $i$, as a consequence $XX^T$ is positive definite

 

[MrJava]

When I try to solve the case for X^TX I get stuck at the following:
x^T(X^TX)x = x^TX^TXx = (Xx)^TXx

Please kindly guide me next step.

 

[Fernando Revilla]

When I try to solve the case for X^TX I get stuck at the following: x^T(X^TX)x = x^TX^TXx = (Xx)^TXx

Right. Now, the difference is that $x\in \mathbb{R}^{n\times 1}$ instead of $\mathbb{R}^{d\times 1}.$ So, for all $x\in \mathbb{R}^{n\times 1}$
$$(Xx)^T(Xx)=(w_1,\ldots,w_d) \begin{pmatrix}{w_1}\\{\vdots}\\{w_d}\end{pmatrix} =w_1^2+\ldots+w_d^2\geq 0$$

 

[MrJava]

Right. Now, the difference is that $x\in \mathbb{R}^{n\times 1}$ instead of $\mathbb{R}^{d\times 1}.$ So, for all $x\in \mathbb{R}^{n\times 1}$
$$(Xx)^T(Xx)=(w_1,\ldots,w_d) \begin{pmatrix}{w_1}\\{\vdots}\\{w_d}\end{pmatrix} =w_1^2+\ldots+w_d^2\geq 0$$

Ok I get the point, thank you.