Fortran Fortran: Read data from a line in a file

Click For Summary
Reading data from a large text file formatted as a matrix (1,000,000 rows x 3 columns) poses challenges for accessing specific rows without sequentially reading prior lines. Directly accessing row 90,000 without reading the preceding 89,999 rows is not feasible due to the nature of file reading, which is typically sequential. A recommended solution is to read the entire file into memory, allowing for quick access to any row after the initial load. This approach, while requiring time and memory upfront, enables efficient retrieval of data from the desired row. For example, if each line contains three double precision numbers, the total memory usage would be approximately 24 MB, making it manageable for most systems.
sourish_SUNY
Messages
1
Reaction score
0
This is my problem:
I would like to read data (saved in a text file) from a file. My data is written in a matrix format (1,000,000 rows x 3 columns).
I want to read a data from a particular line, say row number 90,000.
Since the number of rows is large, it will be very expensive if I have to do an empty read() for 89,999 rows.
Is there a way I can directly go to row number 90,000 (without reading the lines before it) and read the corresponding data? I do have control over how the text file is created.
Will really appreciate any help or advice on this.

Thank You,
SC
 
Technology news on Phys.org
sourish_SUNY said:
This is my problem:
I would like to read data (saved in a text file) from a file. My data is written in a matrix format (1,000,000 rows x 3 columns).
I want to read a data from a particular line, say row number 90,000.
Since the number of rows is large, it will be very expensive if I have to do an empty read() for 89,999 rows.
Is there a way I can directly go to row number 90,000 (without reading the lines before it) and read the corresponding data? I do have control over how the text file is created.
Will really appreciate any help or advice on this.
I can't imagine that this would work very well for you. You're trying to perform random access on an object whose normal mode of reading is sequential access. Although it might seem that the data in your file is in matrix form, in reality, the data is in 1,000,000 million lines. It might be more realistic to read the data from the file, and store it in memory in an actual matrix (a two-d array). You didn't say what kind of data, so I can't say how much memory the matrix would use. If each line in your file consists of three double precision numbers, (at eight bytes each), the matrix would use about 24,000,000 bytes, or about 24 MB.

It would take a while to read the data into memory, but once there, you could access any element in the 90,000 row pretty quickly.
 
Learn If you want to write code for Python Machine learning, AI Statistics/data analysis Scientific research Web application servers Some microcontrollers JavaScript/Node JS/TypeScript Web sites Web application servers C# Games (Unity) Consumer applications (Windows) Business applications C++ Games (Unreal Engine) Operating systems, device drivers Microcontrollers/embedded systems Consumer applications (Linux) Some more tips: Do not learn C++ (or any other dialect of C) as a...

Similar threads

  • · Replies 8 ·
Replies
8
Views
2K
  • · Replies 5 ·
Replies
5
Views
5K
  • · Replies 12 ·
Replies
12
Views
3K
  • · Replies 8 ·
Replies
8
Views
3K
Replies
8
Views
2K
  • · Replies 2 ·
Replies
2
Views
3K
  • · Replies 2 ·
Replies
2
Views
1K
  • · Replies 1 ·
Replies
1
Views
4K
  • · Replies 16 ·
Replies
16
Views
3K
  • · Replies 2 ·
Replies
2
Views
2K