OpenBLAS crashes on Ubuntu 15.04

  • Thread starter Thread starter Pablo Brubeck
  • Start date Start date
  • Tags Tags
    matlab ubuntu
AI Thread Summary
OpenBLAS crashes on Ubuntu 15.04 when attempting to reduce, invert, or factorize large matrices (10000x10000), causing system restarts without error messages. The issue appears linked to memory limitations, as the matrix size is approximately 6.4GB to 8.0GB, which may exceed available RAM. Smaller matrices work without crashing, suggesting a memory management problem. Users are advised to check memory usage with the 'free' command and consider increasing swap space or RAM. Additionally, since Octave does not exhibit the same issue, it may serve as a temporary alternative for users facing this problem.
Pablo Brubeck
Messages
6
Reaction score
0
Whenever I try to reduce, invert, or factorize matrices of size 10000x10000 my whole system suddenly restarts without any error message. This happens on both julia and MATLAB when I run the command A=rand(10000,10000)^-1;

I suspect the problem is due to the linear algebra libraries (I have openblas 0.2.12-1 and lapack 3.5.0-4). I'm running Ubuntu 15.04 on an Intel Core i7-4790K, 2x8GB RAM Kingston Fury, Asus Z97-P motherboard, and NVIDIA 980 GTX gpu.

Help please, and thanks.
 
Physics news on Phys.org
It may very well be the size of your matrix - it is slightly less than 1Gbyte in size. Try with a 100x100 matrix first and see if it still crashes.
 
Svein said:
It may very well be the size of your matrix - it is slightly less than 1Gbyte in size. Try with a 100x100 matrix first and see if it still crashes.
It does not crash with small matrices, it can work well with 1000. My windows laptop with lower specs can work with those large sizes.
 
Pablo Brubeck said:
My windows laptop with lower specs can work with those large sizes.
Let me guess - you are running a 64bit version of Windows on your laptop and 32bit version of Ubuntu...
 
Svein said:
Let me guess - you are running a 64bit version of Windows on your laptop and 32bit version of Ubuntu...
Both are 64bit OS running on 64bit machines. It seems that the problem is not present when using octave.
 
Last edited:
What you have on Ubuntu is called a kernel panic - usually from corruption of
kernel data often due to a programming error in user space piddling in kernel space.

The linux command
Code:
free
will show you how much memory is in use at
any given time. Normally, a single user systems will have about 90% of memory
free - note that those buffers you see come and go dynamically

From the man7.org man page information on the free(1) command
free displays the total amount of free and used physical and swap
memory in the system, as well as the buffers and caches used by the
kernel. The information is gathered by parsing /proc/meminfo. The
displayed columns are:

total Total installed memory (MemTotal and SwapTotal in
/proc/meminfo)

used Used memory (calculated as total - free - buffers - cache)

free Unused memory (MemFree and SwapFree in /proc/meminfo)

shared Memory used (mostly) by tmpfs (Shmem in /proc/meminfo,
available on kernels 2.6.32, displayed as zero if not
available)

buffers
Memory used by kernel buffers (Buffers in /proc/meminfo)

cache Memory used by the page cache and slabs (Cached and Slab in
/proc/meminfo)

buff/cache
Sum of buffers and cache

available
Estimation of how much memory is available for starting new
applications, without swapping. Unlike the data provided by
the cache or free fields, this field takes into account page
cache and also that not all reclaimable memory slabs will be
reclaimed due to items being in use (MemAvailable in
/proc/meminfo, available on kernels 3.14, emulated on kernels
2.6.27+, otherwise the same as free)

Now you have a tool.

Your matrix is a lot larger than mentioned before. 64 (bits in a signed
integer) * 10000 *10000 is ~6.4GB, with 80 bits in a double precision floating
point varable it is ~8.0GB. Math packages tend to use existing numeric formats
unless you are using extended precision - like in GMP.

The free tool will give you available. If you need more memory you will have to add swap space.
Virtual memory = swap (paging) file size plus memory. This is temporary fix.
Add more RAM is another possibility but has a monetary penalty.

Also consider letting the people who support the problem software packages know
of the issue. This is important in the long run.

Since octave does not exhibit the problem use it instead if the above is too
much of a problem. But still consider reporting the problem.
 
jim mcnamara said:
64 (bits in a signed integer) * 10000 *10000 is ~6.4GB, with 80 bits in a double precision floating
point varable it is ~8.0GB.
Umm - 64bits = 8bytes...
 
Thank you - you are absolutely right.
 

Similar threads

Replies
11
Views
16K
Back
Top