OpenBLAS crashes on Ubuntu 15.04

[Pablo Brubeck]

Whenever I try to reduce, invert, or factorize matrices of size 10000x10000 my whole system suddenly restarts without any error message. This happens on both julia and MATLAB when I run the command A=rand(10000,10000)^-1;

I suspect the problem is due to the linear algebra libraries (I have openblas 0.2.12-1 and lapack 3.5.0-4). I'm running Ubuntu 15.04 on an Intel Core i7-4790K, 2x8GB RAM Kingston Fury, Asus Z97-P motherboard, and NVIDIA 980 GTX gpu.

Help please, and thanks.

 

[Svein]

It may very well be the size of your matrix - it is slightly less than 1Gbyte in size. Try with a 100x100 matrix first and see if it still crashes.

 

[Pablo Brubeck]

It may very well be the size of your matrix - it is slightly less than 1Gbyte in size. Try with a 100x100 matrix first and see if it still crashes.

It does not crash with small matrices, it can work well with 1000. My windows laptop with lower specs can work with those large sizes.

 

[Svein]

My windows laptop with lower specs can work with those large sizes.

Let me guess - you are running a 64bit version of Windows on your laptop and 32bit version of Ubuntu...

 

[Pablo Brubeck]

Let me guess - you are running a 64bit version of Windows on your laptop and 32bit version of Ubuntu...

Both are 64bit OS running on 64bit machines. It seems that the problem is not present when using octave.

 

[jim mcnamara]

What you have on Ubuntu is called a kernel panic - usually from corruption of
kernel data often due to a programming error in user space piddling in kernel space.

The linux command

free

will show you how much memory is in use at
any given time. Normally, a single user systems will have about 90% of memory
free - note that those buffers you see come and go dynamically

From the man7.org man page information on the free(1) command

free displays the total amount of free and used physical and swap
memory in the system, as well as the buffers and caches used by the
kernel. The information is gathered by parsing /proc/meminfo. The
displayed columns are:

total Total installed memory (MemTotal and SwapTotal in
/proc/meminfo)

used Used memory (calculated as total - free - buffers - cache)

free Unused memory (MemFree and SwapFree in /proc/meminfo)

shared Memory used (mostly) by tmpfs (Shmem in /proc/meminfo,
available on kernels 2.6.32, displayed as zero if not
available)

buffers
Memory used by kernel buffers (Buffers in /proc/meminfo)

cache Memory used by the page cache and slabs (Cached and Slab in
/proc/meminfo)

buff/cache
Sum of buffers and cache

available
Estimation of how much memory is available for starting new
applications, without swapping. Unlike the data provided by
the cache or free fields, this field takes into account page
cache and also that not all reclaimable memory slabs will be
reclaimed due to items being in use (MemAvailable in
/proc/meminfo, available on kernels 3.14, emulated on kernels
2.6.27+, otherwise the same as free)

Now you have a tool.

Your matrix is a lot larger than mentioned before. 64 (bits in a signed
integer) * 10000 *10000 is ~6.4GB, with 80 bits in a double precision floating
point varable it is ~8.0GB. Math packages tend to use existing numeric formats
unless you are using extended precision - like in GMP.

The free tool will give you available. If you need more memory you will have to add swap space.
Virtual memory = swap (paging) file size plus memory. This is temporary fix.
Add more RAM is another possibility but has a monetary penalty.

Also consider letting the people who support the problem software packages know
of the issue. This is important in the long run.

Since octave does not exhibit the problem use it instead if the above is too
much of a problem. But still consider reporting the problem.

 

[Svein]

64 (bits in a signed integer) * 10000 *10000 is ~6.4GB, with 80 bits in a double precision floating
point varable it is ~8.0GB.

Umm - 64bits = 8bytes...

 

[jim mcnamara]

Thank you - you are absolutely right.