Design Matrix: Restrictions & Definition - Mike

[mikeph]

Hi,
Are there any restrictions on what a design matrix can be?

I have no background in this area, I'm just wondering from the wikipedia article, is it a mathematical entity or simply a name? Can it be all zeros, diagonal, or anything I want?

Thanks
Mike

 

[statdad]

What type of problem are you considering? Design matrices can have different forms depending on whether you are doing a type of ANOVA or regression, whether the problem is multivariate or univariate.
The only easy answer to give is the one for "can it be all zeros?" No, it can't.

As one simple example: if you want to do a linear regression through the points (2,5), (3,4), (5,12), (7,13),
the "design matrix" that would be used to develop the fit with matrix methods has its first column all 1s and the second column the x-values of these points.

 

[chiro]

Hey MikeyW.

One thing you need to look out for is when you have observation vectors that are nearly linearly dependent. When this occurs you get all kinds of crazy behavior.

Most packages will pick this up and you can use software to find out the vectors that have this property.

Also if you have variables that are largely correlated, then you can use something like Principal Components to un-correlate them and use for a regression.