Running a multidimensional problem on a single core

[Telemachus]

Hi. I have written a Fortran program to solve a pde, which has six dimensions. One of these dimensions is time, 3 are for space, and the other are related to velocities.

The program works fine when I run it with $38^3$ points in space (one for each space dimension), and 80 other dimensions related to the velocities. However, I have tried to increase the number of velocity variables from 80 to 288, for which I would expect the program to run roughly 3 to 4 times the time it took with 80 variables.

The total number of unknowns is, in the first case: $38^3\times 80 \times n_{time steps}$. In the second situation is $38^3\times 288 \times n_{time steps}$. For the first run, it took exactly 494 seconds to solve it. However, when I run the second situation, it has been going on for hours, and it haven't finished, so I just quit the run, because I am suspecting that there is something going on. The number of time steps in both situations is 200, but I'm not storing the solutions in time, I don't think that matters.

However, I have enough ram for it, 16gb. It would require $38^3\times 80 \times 8 bytes$ of memory (I am working with double precision real numbers). This gives only 35.12 megabytes of memory required. And 126.43 megabytes for the 288 velocities. I don't see where the problem is. Is there something I am missing or this should be working? If it took 494 seconds in the first case, being the second problem less than 4 times bigger, I would expect this to be solved in roughly 2000 seconds, but it has been like an hour running without finishing.

 

[anorlunda]

You don't give us much to go on. Here are two speculations.

1. Either your program or some library function that you use, needs intermediate storage or a second copy of the arrays to work with.
2. Integer overflow in your code or in a library function.

 

[Telemachus]

Hi. Almost all the code have been written by me, I am only using one subroutine which isn't mine, but didn't have any issue previously. The code works fine with smaller problems, the running time scales as expected until that point. I will try to let it running for more time, but I am suspecting that there is something else. I think I have the same issue if I increase the spatial coordinates, but I haven't tried because it goes like a power of three, so I will run into trouble faster.

I don't think I have any integer overflow neither. Is there any other possibility? I mean, clearly I have more than enough ram memory. But is there any other internal memory in the cpu that could be going out of range at some critical point or this should work for arbitrary large problems while I have enough ram to store it?

 

[anorlunda]

What kinds operations are you (or FORTRAN) doing on that matrix? Inversion? Factorization? anything?

Modern FORTRAN does operations on whole arrays or matrices.
That generates invisible calls to library functions.

Old FORTRAN makes you do everything one element at a time.

Edit: Of course yes the operating system needs memory too. You haven't told us what the OS is.

 

[Telemachus]

I am using Fortran77. The only library I am using is FFTW, but the way I am calling it really shouldn't make any difference when I use 80 or 288 velocities (all go on independently). I am not using any matrix operations, I work with the arrays and many do loops.

 

[Telemachus]

I have just found a bug in the code, I had fixed the memory for a set of a arrays to the situation where I had the 80 velocities. I think that's it. I'll fix it. Thanks.