Need help reading and writing from binary file

Click For Summary

Discussion Overview

The discussion revolves around the challenges of reading from and writing to a binary file in a programming context, particularly when the total amount of data to be written is unknown until runtime. The scope includes technical explanations and practical programming advice related to file handling in a stochastic simulation of chemical reactions.

Discussion Character

  • Technical explanation
  • Debate/contested

Main Points Raised

  • One participant seeks assistance on how to write the total number of records to the beginning of a binary file after writing the data, using functions like fseek and fwrite.
  • Another participant suggests that the end of the file is indicated by EOF and questions why the total number of bytes to be written is unknown, implying that data should be written from a buffer of known size.
  • A participant explains that the simulation is stochastic, meaning the number of reactions (and thus data points) cannot be predetermined, which justifies the need for writing the total count at the beginning of the file.
  • Another contributor proposes using a buffer to store data temporarily and suggests keeping track of the number of bytes written with a variable, while also mentioning that shifting bytes in a file is not feasible.
  • This participant also recommends reserving the first four bytes of the file for the total count and writing placeholder data initially if necessary.

Areas of Agreement / Disagreement

Participants express differing views on the necessity of writing the total number of records to the file and the methods to handle binary file writing. No consensus is reached on the best approach to the problem.

Contextual Notes

There are limitations regarding the assumptions about file size and the feasibility of modifying file contents after writing. The discussion does not resolve the technical challenges presented.

ptabor
Messages
14
Reaction score
0
I'm writing a large amount of data to a binary file. Trouble is, I don't know how much until the program finishes executing.

Given this, I'd like to rewind back to the beginning of the binary file and write an element indicating how many records I wrote to the binary file - this way I can create arrays in another program and read in the data to do some fun stuff with it.

I've tried:
fseek(filename, sizeof(int), SEEK_SET);
fwrite(&data, sizeof(int), 1, filename);

to no avail. Can someone please assist?
 
Technology news on Phys.org
When reading any binary file (or even ASCII), the end of the file will be indicated with an EOF (end of file). There's no need to append the file size. For example.

Code:
char buffer[100];
while( !feof( stream ) )
   {
      /* Attempt to read in 10 bytes: */
      count = fread( buffer, sizeof( char ), 100, stream );
   }
I'm also curious about this application. Why is it that you do not know the amount of bytes you are going to write. The write operations should be from a buffer, some sort of array whose size ( sizeof(arr) ) can be found.
 
It's a program to stochastically simulate chemical reactions.

Since it's stochastic, it is a probabilistic problem and there is no way to know beforehand how many reactions you will have in any given interval.

Basically, each time a reaction occurs I'm writing the time at which it occurs, and the reaction number to a binary file. At the end of the simulation I'd like to output the total number of reactions to the beginning of the file so I can write another section of code to parse this data - ie so I can know the state of the system at any point in time.
 
Unless the file you are writing is very very large, you can write all of this information into a buffer. Or of course, you can keep track of how many characters you have written internally in a variable. fwrite() returns the number of bytes it has written.

As far as I know however, it is not possible to shift the bytes in the file. If after all my suggestions you still for some reason want to write the written number of bytes into the file "header", then reserve the first four bytes for this purpose. I'm assuming that your file would not be larger than (2^32) bytes (~4200 Gigabytes). Write junk data to it first and then append it.
 

Similar threads

Replies
10
Views
2K
  • · Replies 7 ·
Replies
7
Views
2K
  • · Replies 4 ·
Replies
4
Views
2K
  • · Replies 12 ·
Replies
12
Views
2K
  • · Replies 8 ·
Replies
8
Views
6K
  • · Replies 5 ·
Replies
5
Views
5K
  • · Replies 12 ·
Replies
12
Views
2K
  • · Replies 75 ·
3
Replies
75
Views
7K
  • · Replies 2 ·
Replies
2
Views
3K
  • · Replies 5 ·
Replies
5
Views
2K