Silly Matrix Algebra/Regression Question

  • Context: Undergrad 
  • Thread starter Thread starter Weather Freak
  • Start date Start date
  • Tags Tags
    Matrix
Click For Summary
SUMMARY

The discussion centers on performing linear regression using matrix algebra, specifically the equation Y = XA, where Y represents the output matrix, X the input matrix, and A the coefficients. The user struggles to derive X from Y and A, mistakenly applying a scalar approach to matrices. The correct approach involves using the transpose of the equation and considering the condition of the matrix A*A', as well as the numerical methods employed, which can significantly impact results.

PREREQUISITES
  • Understanding of linear regression and its matrix representation
  • Familiarity with matrix operations, including transposition and inversion
  • Knowledge of numerical stability and conditioning of matrices
  • Experience with programming libraries for matrix computations, such as NumPy or MATLAB
NEXT STEPS
  • Study the properties of matrix transposition and its implications in linear equations
  • Learn about matrix conditioning and how it affects numerical solutions
  • Explore the use of libraries like NumPy for efficient matrix operations
  • Investigate alternative methods for solving linear regression, such as QR decomposition or Singular Value Decomposition (SVD)
USEFUL FOR

Data scientists, statisticians, and anyone involved in machine learning or numerical analysis who seeks to deepen their understanding of linear regression and matrix algebra.

Weather Freak
Messages
40
Reaction score
0
So I have what I think is going to be a silly matrix algebra question.

I'm performing a linear regression, using standard matrix algebra:

Y = XA (all are matrices)

I have found my coefficients (A) and now I wish to use that information to get X as a function of Y.

If this were a normal algebraic equation, I'd have X = Y/A, but that doesn't make sense in matrix-land.

I've tried: Y * A' * inv(A * A') = X, but that doesn't give me numbers which make any sense.

Can anyone help me clear this up? I'm sure it's not too difficult... thanks!

EDIT: I suppose it might be useful to give some matrix dimensions, but I think the algebra should be the same no matter what... A = (10x144), X = (1x10), Y = (1x144)
 
Physics news on Phys.org
The approach you are taking (so-called "normal equations") will often work, although in general it isn't good from a numerics perspective, expecially for a large number of unknowns. I think we need more details in order to help.

How do you know that the results make no sense?

What exactly are your basis functions (powers, othogonal polynomials, or ...)? In other words, what does A look like, and what is the condition of A*A' ? It could be that A*A' is poorly conditioned.

Also, how are you doing the numerics? Did you write you own software or are you using some well established library/application? single precision or double?

jason
 
If you take the transpose of the equation, you are left with a linear equation [itex]A^Tx^T = y^T[/itex]which can be solved in numerous ways. Your suggestion is one of them.
 

Similar threads

  • · Replies 1 ·
Replies
1
Views
2K
  • · Replies 9 ·
Replies
9
Views
3K
  • · Replies 1 ·
Replies
1
Views
4K
  • · Replies 19 ·
Replies
19
Views
4K
  • · Replies 14 ·
Replies
14
Views
5K
  • · Replies 33 ·
2
Replies
33
Views
3K
  • · Replies 19 ·
Replies
19
Views
3K
  • · Replies 5 ·
Replies
5
Views
2K
  • · Replies 7 ·
Replies
7
Views
2K
  • · Replies 1 ·
Replies
1
Views
2K