currently I am trying to solve a least squares problem of the following form:

[itex]min_{M}[/itex] ||Y - M*X||[itex]^2[/itex]

where M is a 3x3 matrix and Y and X are 3xN matrices. However, the matrix M is of a special form. It is a rank 1 matrix which satisfies M*M =

**0**[itex]_{3x3}[/itex] and the trace of M is zero, too.

Enforcing that the trace is zero seems rather easy, since it is a linear constraint. But I have no idea how to enforce M*M =

**0**[itex]_{3x3}[/itex], as it includes multiple quadratic constraints.

Does anyone have an idea how to solve this problem?