[Fortran] How to do I/O jobs and computing jobs at the same time

[xuphys]

It takes a lot of time for my Fortran program to do I/O jobs. Is there any way to do I/O jobs while at the same time it is doing computing jobs? To do I/O jobs and computing jobs at the same time would save a lot of time.

Thanks.

 

[D H]

Run the I/O-intensive and CPU-intensive aspects of your application in different threads of execution. There are Fortran wrappers for the POSIX threads library (but pthreads entails running on a Unix-like system). Another option is OpenMP.

Whenever you use threads you need to be concerned with one thread corrupting another. Your Fortran library for example may or may not be thread safe. (C as-is also has this problem, except that large chunks of the C library must be thread safe to be POSIX compliant. Similar concerns exist on and are addressed in the Windows C standard libraries with respect to the Windows threading mechanisms.)

Your program almost certainly is not thread safe as-is. Common blocks are notoriously unsafe. You will need to modify your program to make sure that (for example) the input thread is not writing input data to the same structure or common block that your compute thread is reading. You do not want your compute thread operating on an input buffer that contains some data from input record A and some from record A+1. Output has similar problems with corruption. Guarding against corruption can be a bit tricky.

One simplifying approach is to use a double buffer technique. Suppose you have an input thread, a compute thread, and an output thread. You'll have two pairs of buffers here, one pair for input and another pair for output. Your input thread will store input data in one of the input buffer pairs while your compute thread will operate on the input data that has previously been stored in the other input buffer. When the input thread finishes writing to an input buffer, it marks that newly-populated input buffer as ready for the compute thread and waits until the other input buffer is no longer in use by the compute thread. The compute thread needs to perform an analogous set actions when it finishes processing the data from an input buffer.

 

[rcgldr]

I'm not sure what I/O jobs the Fortran progam is doing, but with read ahead and write behind caching in hard drives and operating systems, you should be getting a lot of concurrency even with a single threaded application.

Perhaps the input file(s) are highly fragmented which would slow things down due to all the random accessing. You might consider defragging that partition.

You didn't mention what type of device was involved in the I/O. If you're taking real time readings from an array of instrumented sensors, that could be I/O intensive and a candiate for multi-threading if the drivers for those insturments aren't already buffering data.

You didn't mention what operating system you were using. I've written some simple multi-threaded apps for Windows in C, but I haven't used Fortan in a long time.

 

[uart]

I'm not familiar with the specifics of Fortran in the regard. But programs that read data one byte/word/whatever one at a time, process it and then write an output byte/word/whatever one at a time, can often be sped up dramatically by reading and writing the data in blocks to/from a buffer (array) of several kB or more.

 

[Louisa]

It takes a lot of time for my Fortran program to do I/O jobs. Is there any way to do I/O jobs while at the same time it is doing computing jobs? To do I/O jobs and computing jobs at the same time would save a lot of time.

You can do that in PL/I (in Windows and other systems).