MATLAB MATLAB linear optimization out of memory

Click For Summary
The discussion revolves around resolving an "Out of memory" error encountered in MATLAB while attempting to solve a linear optimization problem using the "fmincon" function. The user is working with a large-dimension matrix, specifically a 24583x1 vector and a product of matrices that includes a 300x403 random matrix and a 403x24583 Gabor basis dictionary. Suggestions include utilizing sparse matrix functions to reduce memory usage and ensuring that unnecessary copies of matrices or temporary arrays are not being created. It is noted that even though the matrix PHI is large, the main issue arises from the system generating multiple copies of arrays during the optimization process. The discussion emphasizes the importance of managing memory effectively when dealing with large datasets in MATLAB.
OhMyMarkov
Messages
81
Reaction score
0
Hello everyone!

I'm trying to solve a linear optimization problem with a large-dimension matrix, MATLAB says memory is full. What to do?

Thanks!
 
Physics news on Phys.org
OhMyMarkov said:
Hello everyone!

I'm trying to solve a linear optimization problem with a large-dimension matrix, MATLAB says memory is full. What to do?

Thanks!

See if you can use sparse matrix functions?

Check that you are not creating unnecessary copies of matrices and or temporary arrays?

How large is the large-dimension matrix?

CB
 
I'm having the Out of memory error whenever I call "fmincon" as follows:x = fmincon("at"ftest, zeros(24583,1), -eye(24583), zeros(24583,1), PHI, S(:,1,1);Now, ftest is the L1 norm, PHI is the product of two matrices, a sampling 300x403 random matrix and a 403x24583 Gabor basis dictionary. Please note that one of the sampling matrix is in "full" representation while the larger matrix is in "sparse" representation. I also tried making PHI explicitly in sparse format to check but this didn't solve the problem?

It seems that the minimizer expands PHI.

Thank you for your time.

OMM
 
OhMyMarkov said:
I'm having the Out of memory error whenever I call "fmincon" as follows:x = fmincon("at"ftest, zeros(24583,1), -eye(24583), zeros(24583,1), PHI, S(:,1,1);Now, ftest is the L1 norm, PHI is the product of two matrices, a sampling 300x403 random matrix and a 403x24583 Gabor basis dictionary. Please note that one of the sampling matrix is in "full" representation while the larger matrix is in "sparse" representation. I also tried making PHI explicitly in sparse format to check but this didn't solve the problem?

It seems that the minimizer expands PHI.

Thank you for your time.

OMM

That would not in itself be a problem (it is an ~79 MB structure), it is the fact that the system will generate multiple copies of arrays of that size in doing its work.

I presume the dimensions of S are not too great ...

CB
 

Similar threads

  • · Replies 9 ·
Replies
9
Views
3K
  • · Replies 5 ·
Replies
5
Views
3K
  • · Replies 32 ·
2
Replies
32
Views
4K
  • · Replies 5 ·
Replies
5
Views
2K
  • · Replies 2 ·
Replies
2
Views
3K
  • · Replies 4 ·
Replies
4
Views
1K
  • · Replies 5 ·
Replies
5
Views
2K
  • · Replies 2 ·
Replies
2
Views
2K
  • · Replies 1 ·
Replies
1
Views
1K
  • · Replies 1 ·
Replies
1
Views
4K