Problem loading large files (IOSTAT=100)

[mrz1982]

Hi.
I am having problem loading large data files. The files consist of values defined as (with a extra space at the end of each line):

+1.326000e-005 -1.159306e+000 -1.171263e+000 
+1.327000e-005 -1.159202e+000 -1.173770e+000 
+1.328000e-005 -1.159563e+000 -1.175345e+000

The problem is when the number of colomns in my data file becomes more than 99. It is not a memory issue as 99 colomns with 4001 rows works, but 100 colomns with 1001 rows does not work. I am loading the files for 99 colomns like (var2 needs to be larger for later use):

REAL var1(1001,99), var2(1001,194)
WRITE(READFMT,'(1H(I2,5HE15.61H))')99
open(1,file='folder/file1',FORM='FORMATTED',IOSTAT=IOS,RECL=1486)
open(2,file='folder/file2',FORM='FORMATTED',IOSTAT=IOS,RECL=1471)
DO t=1,1001
 READ(1,FMT=READFMT,IOSTAT=IOS)(var1(t,x),x=1,99)
 READ(2,FMT=READFMT,IOSTAT=IOS)(var2(t,x),x=1,98)
ENDDO
close(1)
close(2)

and for 100 colomns like:

REAL var1(1001,100), var2(1001,195)
WRITE(READFMT,'(1H(I2,5HE15.61H))')100
open(1,file='folder/file1',FORM='FORMATTED',IOSTAT=IOS,RECL=1501)
open(2,file='folder/file2',FORM='FORMATTED',IOSTAT=IOS,RECL=1486)
DO t=1,1001
 READ(1,FMT=READFMT,IOSTAT=IOS)(var1(t,x),x=1,100)
 READ(2,FMT=READFMT,IOSTAT=IOS)(var2(t,x),x=1,99)
ENDDO
close(1)
close(2)

I have looked at the IOSTAT value, it returns 100. I have found many values for IOSTAT but not for 100.
I am noticing that as the XXX value in WRITE(READFMT,'(1H(I2,5HE15.61H))')XXX becomes a three digit number the loading is failing i.e. the files can be any size, if the XXX value is larger than 99 the variables, var1 and var2, will be zero.
How can this be solved?

Kind regards!
/Z

I changed the

READ(1,FMT=READFMT,IOSTAT=IOS)(var1(t,x),x=1,100)
READ(2,FMT=READFMT,IOSTAT=IOS)(var2(t,x),x=1,99)

to

READ(1,FMT=*,IOSTAT=IOS)(var1(t,x),x=1,100)
READ(2,FMT=*,IOSTAT=IOS)(var2(t,x),x=1,99)

and the problem was solved! :)

 

[lofgran]

Hi Z,

Thank you for sharing your issue with loading large data files. It seems like the issue may be with the formatting of your READFMT statement. I would recommend trying to change the format to something like '(1H(I2,5HE15.61H))' which specifies the number of characters to be read. Additionally, you may want to try using the "*" format option in your READ statement, which allows for variable length input records.

I hope this helps solve your problem. If not, please provide more information about the specific error message you are receiving and I will do my best to assist further.

 

[quicknote]

Dear Z,

Thank you for reaching out with your issue. From what you have described, it seems that the problem lies in the formatting of your data files. Specifically, you are encountering an IOSTAT=100 error, which indicates that there is an issue with the FORMAT statement used in your code.

After reviewing your code, I noticed that you are using a specific format statement, WRITE(READFMT,'(1H(I2,5HE15.61H))'), to read in your data. This format statement is designed to read in a specific number of columns, as denoted by the 'XXX' value in the statement. However, as you have discovered, when this value exceeds 99, the loading fails.

To solve this issue, I suggest using a more flexible format statement, such as WRITE(READFMT,'(1H(I2,5HE15.61H))'), which will allow you to read in any number of columns. This should resolve the issue of your variables being set to zero when the number of columns exceeds 99.

I hope this helps to solve your issue. If you continue to experience problems, I recommend seeking assistance from a colleague or consulting with your institution's IT department for further support.

Best regards,