Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Memmapfile Memory Usage, what's wrong?

  1. Feb 18, 2009 #1
    Hello,

    In a function I'm mapping a ~30MB file like this:

    Code (Text):

    raw = memmapfile( ...
        sFileName, ...
        'Offset', iOffsetToData, ...
        'Format', { ...
            'int32' [1 3] 'a'; ...
         
            'uint16' [1 1] 'b'; ...
           
            'uint8' [1 1] 'c'; ...
                                                                         
            'uint8' [1 1] 'chClassification'; ...
                                     
           
            'int8' [1 1] 'chScanAngleRank';  ...
                                     
           
            'uint8' [1 1] 'chFileMarker';  ...
                                     
           
            'uint16' [1 1] 'rvUserBitField'; ...
           
            'double' [1 1] 'o'; ...
                                   
            } ...
        );
    All good.

    now I want to read the data so I write:

    Code (Text):
    myData=raw.Data;

    Now there are huge problems. Matlab goes from 300MB to 2.5GB of memory usage.

    whos mydata returns 1.2GB. (How is this even possible?)

    I do some computing and return a matrix that should be about 40MB.

    End of function.


    So, fine, I thought, the function is finished and matlab deallocates the memory it used. But no, it doesn't. I tried clearing it up manually in the function, but nothing happens. I also tried reading raw.Data(k).a in a loop, but Matlab appears to create the whole raw.Data array anyway.

    My biggest concern is that most computers won't handle the huge mem requirement increase. It is just too much. How can I fix it?

    I'm sure I can skip this memmapfile-method altogether, and use some read file function, but it's slower. Is it possible to read custom structures from files (as with the ease of memmapfile)?
     
  2. jcsd
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook

Can you offer guidance or do you also need help?
Draft saved Draft deleted



Similar Discussions: Memmapfile Memory Usage, what's wrong?
Loading...