Negative eigenvalues in covariance matrix

Click For Summary
Running the factoran function in MATLAB on a large matrix of daily stock returns has revealed issues with negative eigenvalues in the covariance matrix, which are attributed to floating point errors. The covariance matrix should ideally be positive semi-definite, but the presence of very small negative eigenvalues complicates this. Subtracting the market return does not resolve the issue. A suggested solution involves adding a small constant value (σ2) to the diagonal elements of the covariance matrix, effectively increasing the noise floor and eliminating negative eigenvalues. This technique, known as diagonal loading or Tikhonov regularization, requires careful selection of σ2 to ensure it is large enough to address the numerical errors but small compared to the relevant eigenvalues. There is concern about the lack of built-in options in MATLAB for this adjustment, leading to the possibility of modifying the MATLAB function directly.
BWV
Messages
1,603
Reaction score
1,956
Trying to run the factoran function in MATLAB on a large matrix of daily stock returns. The function requires the data to have a positive definite covariance matrix, but this data has many very small negative eigenvalues (< 10^-17), which I understand to be a floating point issue as 'real' covariance matrices are positive semi-definite. Does not make a difference whether or not I subtract the market return (which reduces the correlation). Any thoughts, I can find options to 'fix' the cov matrix, but nothing to tweak the data prior to calculating the cov matrix
 
Technology news on Phys.org
Add a small constant value σ2 to the diagonal elements of your covariance matrix. This has the effect of increasing the noise floor in your data, eliminating the negative values arising from numerical errors. The smallest eigenvalues will now take the value σ2 (approximately--but that's a topic for another thread). Choose σ2 large compared to 1e-17 but small compared to the eigenvalues of interest. If your eigenvalues of interest are of order >=1, for instance, then σ2=1e-10 will suffice without affecting the validity of your data. This is known in the statistical signal processing literature as diagonal loading and in the mathematics literature as Tikhonov regularization.
 
  • Like
Likes sysprog, FactChecker and BWV
Thanks, afraid of that, will have to hack the MATLAB function to do this, surprised there is not an option on the cov function or another built-in function to do this. Its just something I am playing around with, so not sure if I will invest the time
 
Let us know how it turns out!
 
I tried a web search "the loss of programming ", and found an article saying that all aspects of writing, developing, and testing software programs will one day all be handled through artificial intelligence. One must wonder then, who is responsible. WHO is responsible for any problems, bugs, deficiencies, or whatever malfunctions which the programs make their users endure? Things may work wrong however the "wrong" happens. AI needs to fix the problems for the users. Any way to...

Similar threads

  • · Replies 1 ·
Replies
1
Views
2K
  • · Replies 27 ·
Replies
27
Views
4K
  • · Replies 5 ·
Replies
5
Views
1K
  • · Replies 8 ·
Replies
8
Views
2K
  • · Replies 5 ·
Replies
5
Views
5K
Replies
11
Views
6K
  • · Replies 1 ·
Replies
1
Views
9K
  • · Replies 15 ·
Replies
15
Views
2K
Replies
1
Views
3K
  • · Replies 2 ·
Replies
2
Views
9K