Avoiding "Out of Memory" Error: Writing Values to Text File

Click For Summary

Discussion Overview

The discussion revolves around a programming issue in MATLAB related to writing values to a text file without encountering "out of memory" errors. Participants explore methods for appending data to a file rather than overwriting it, focusing on file handling techniques and coding practices.

Discussion Character

  • Technical explanation
  • Conceptual clarification
  • Homework-related

Main Points Raised

  • One participant describes their approach of storing a large number of 7-element vectors in an array and writing them to a file, which leads to memory issues.
  • Another participant suggests using the 'a+' mode in fopen to append data to the file instead of overwriting it.
  • A participant shares a code snippet attempting to write data to a file but notes that it overwrites the file each time it runs.
  • Further clarification is provided that the file should be opened for writing only once after all data points are generated, rather than opening and closing it for each data point.

Areas of Agreement / Disagreement

Participants generally agree on the need to append data to avoid overwriting, but there is no consensus on the best implementation method, as some participants continue to experience issues with their approaches.

Contextual Notes

Participants discuss the importance of file handling practices, including when to open and close files, but do not resolve specific coding errors or provide a definitive solution to the original problem.

smk037
Messages
67
Reaction score
1
I have been writing a program to compute and change a value a great number of times and writes each value into a file

I did this by continuously storing each value (which is a 7 element vector) into an array, and then writing that into a file using the cvswrite function

The problem I am having is that the array that the array eventually grows to the size that MATLAB gives me an "out of memory" error

so, to fix this, I have been trying to write each value into the .txt file, closing the file, computing the next vector into the same vector that it was computed from (to avoid the memory error), and writing it to the same file, but I can't seem to get it to work

I just end up overwriting the file each time

Is there anyway to get the next vector to print to the next line of the text file, as opposed to just overwriting the initial value in the text file?

any ideas?
Thank you.
 
Physics news on Phys.org
You can open the file with fopen(filename,'a+') and after that fprintf to write to the end of the file.
 
thanks for the advice

ok, so I've been trying to do something like this

x = [1 2 3 4 5 6]
y = [6 5 4 3 2 1]
fid = fopen('exp.txt', 'wt');
fprintf(fid, '%s\n', y);
fclose(fid)

but again, each time I run the code, the value just gets overwritten
 
smk037 said:
thanks for the advice

ok, so I've been trying to do something like this

x = [1 2 3 4 5 6]
y = [6 5 4 3 2 1]
fid = fopen('exp.txt', 'wt');
fprintf(fid, '%s\n', y);
fclose(fid)

but again, each time I run the code, the value just gets overwritten

If you're running this particular block of code repeatedly to generate your file then of course the data will be rewritten. At the end you're closing the file, but when you loop around again you're opening the file and rewriting its contents.

A simple example might help. Suppose that I want to create a file with a list of numbers in it; for the sake of argument, suppose I want to use Matlab to create a file "test.txt" that contains integers from one to ten. An easy way to do it is to use a for loop:

Code:
% First, open the file 'test.txt' for writing.

fid = fopen('test.txt', 'wt');

% Now comes the for loop. At each step of the loop we assign a value to x
% and write this value to the file. The loop starts at x = 1 and continues
% to x = 10

for i=1:10
    x = i;
    fprintf(fid, '%6.2f\n', x);
end

% Now that the loop is finished and we have written the data to the file, 
% close the file.

fclose(fid)

The important point to note is that you should close the file only once you've finished adding data to it, not after you add each data point. You should easily be able to modify the above example to work with your array.
 

Similar threads

  • · Replies 4 ·
Replies
4
Views
2K
  • · Replies 4 ·
Replies
4
Views
7K
  • · Replies 5 ·
Replies
5
Views
4K
  • · Replies 8 ·
Replies
8
Views
13K
Replies
35
Views
8K
Replies
5
Views
2K
  • · Replies 1 ·
Replies
1
Views
2K
Replies
1
Views
4K
  • · Replies 57 ·
2
Replies
57
Views
6K
  • · Replies 1 ·
Replies
1
Views
3K