Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Allocating memory on windows

  1. Mar 25, 2007 #1
    When I declare an array of size 24 mb in c++, I have discovered that the result is a runtime error if the declaration is made inside a function. If the declaration is made outside a function, then there is no problem.

    The problem exists with both VC++ Express and Cygwin GCC (GCC gives a segmentation fault). There is no problem on Suse Linux.

    Does windows have a limitation on the amount of memory that can be allocated using functions or something?

    I have 1 GB Ram of which atleast 350 MB is free at all times and 3.2 GHz 2MB L2 cache Pentium IV.
     
  2. jcsd
  3. Mar 25, 2007 #2

    robphy

    User Avatar
    Science Advisor
    Homework Helper
    Gold Member

    It might help if you show some code.
    Do you deallocate the memory when you exit the function?
     
  4. Mar 25, 2007 #3
    Actually the array declared is a static array. Here's the relevant code which generates error.
    <code>
    void LoadHistory(bool k)
    {
    //Declare and initialize variables
    int matchUser[3000000][2] //The erring variable

    fstream fbinary, fmovieratingcount;
    fbinary.open(BINARY_FILE_PATH, ios::in | ios::binary);
    fmovieratingcount.open(MOVIE_RATING_COUNT, ios::in | ios::binary);

    if(!fbinary && !fmovieratingcount)
    {
    cout<<"FILE CAN'T BE OPENED!\a\n";
    exit(1);
    }

    int tmp, movieId;

    for(int i = 0; i < 3000000; i++)
    {
    matchUser[0] = 0;
    }

    //While loop to read movie ratings
    while(!fmovieratingcount.eof() && !fbinary.eof() && net_movie_no < NO_MOVIES)
    {
    int custId, ratings_no;
    int rating;

    fmovieratingcount.read((char*) &tmp, sizeof(tmp));
    fbinary.read((char*) &movieId, sizeof(movieId));
    }

    }
    </code>

    Moving matchUser declaration outside the function results in no error.

    <code>
    int matchUser[3000000][2] //No error now
    void LoadHistory(bool k)
    {
    //Declare and initialize variables


    fstream fbinary, fmovieratingcount;
    fbinary.open(BINARY_FILE_PATH, ios::in | ios::binary);
    fmovieratingcount.open(MOVIE_RATING_COUNT, ios::in | ios::binary);

    if(!fbinary && !fmovieratingcount)
    {
    cout<<"FILE CAN'T BE OPENED!\a\n";
    exit(1);
    }

    int tmp, movieId;

    for(int i = 0; i < 3000000; i++)
    {
    matchUser[0] = 0;
    }

    //While loop to read movie ratings
    while(!fmovieratingcount.eof() && !fbinary.eof() && net_movie_no < NO_MOVIES)
    {
    int custId, ratings_no;
    int rating;

    fmovieratingcount.read((char*) &tmp, sizeof(tmp));
    fbinary.read((char*) &movieId, sizeof(movieId));
    }

    }
    </code>
     
  5. Mar 25, 2007 #4

    robphy

    User Avatar
    Science Advisor
    Homework Helper
    Gold Member

  6. Mar 25, 2007 #5
    Looks like you are trying to allocate a big array on the stack.
    Stacks have finite sizes in C, usually you can increase the size but it really does not make any sense to create stacks that are that big.

    By the way, what are you trying to do with this program? There are exceptions, so that's why I ask, but usually it is completely unnecessary and generally a bad idea to load something into an array that big.
     
    Last edited: Mar 25, 2007
  7. Mar 25, 2007 #6
    Thanks robphy.

    Playing around with this http://www.netflixprize.com

    The 24 mb array is for quick mapping. This is actually a very small array. The actual data requires an array of size 200-300 mb minimum.
     
  8. Mar 25, 2007 #7
    I see! :smile:

    If you want to be fast then don't use array indexing but, wherever possible, use pointer arithmetic.

    For instance, in your array initialization routine, you require the program each time to calculate the address instead of just incrementing some pointer.

    Keep that in mind when you run your statistical analysis, you could easily improve the speed of such a program by 50% if you minimize the use of array indexing and use pointer arithmetic.
     
  9. Mar 27, 2007 #8
    Thanks, I'll keep that in mind.
     
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook

Have something to add?