Read file from multiples .dat files and choose only one point

[s_hy]

Hi all.

I have 1200 .dat files containing x,y,z . What I need to do is to open all the files and choose only one single point from all the files, in example point (x120,y120,z120) from all 1200 .dat file and save it into new .dat file. Can anyone teach me how to do that either in fortran or matlab



thank you

 

[jedishrfu]

Hi all.

I have 1200 .dat files containing x,y,z . What I need to do is to open all the files and choose only one single point from all the files, in example point (x120,y120,z120) from all 1200 .dat file and save it into new .dat file. Can anyone teach me how to do that either in fortran or matlab



thank you

Why not use AWK or some other scripting language?

In awk, this would be particularly easy pretty much a one-liner.

 

[s_hy]

actually, I did wrote code in matlab

numFiles = 822;
myData = cell(1,numFiles);
startRow = 6886;
endRow = 6886;

for fileNum = 1:numFiles
    fileName = sprintf('*.dat',fileNum);
    myData{fileNum} = importfile(fileName,startRow,endRow);
end

%% Format string for each line of text:
formatSpec = '%*24s%12f%[^\n\r]';

%% Open the text file.
fileID = fopen(filename,'r');

% This call is based on the structure of the file used to generate this
% code. If an error occurs for a different file, try regenerating the code
% from the Import Tool.
dataArray = textscan(fileID, formatSpec, endRow-startRow+1, 'Delimiter', '', 'WhiteSpace', '', 'HeaderLines', startRow-1, 'ReturnOnError', false);

%% Close the text file.
fclose(fileID);

%% Allocate imported array to column variable names
VarName3 = dataArray{:, 1};

but I got this error

Error using textscan
Invalid file identifier. Use fopen to generate a valid file identifier.

Error in importfile (line 34)
dataArray = textscan(fileID, formatSpec, endRow(1)-startRow(1)+1, 'Delimiter', '', 'WhiteSpace',
'', 'HeaderLines', startRow(1)-1, 'ReturnOnError', false);

Error in readdata (line 18)
myData{fileNum} = importfile(fileName,startRow,endRow);

and I am actually don't understand the reason of this error. Only one in my mind is the filename of datafile is data_Ez 0001.dat and so on. Is it the white space in filename maybe the reason? If yes, how I can remove all the whitespace in all 822 datafile?

thank you

 

[jedishrfu]

The whitespace is most likely the culprit. Try an experiment and change the names removing the space and see what happens.

This feature of windows has caused many problems over the years. The most common case is that you create a bat file where you pass in the filename with the space and script in turn interprets it as two arguments not one.

An alternative if this is windows is to try the short name of the file ie file 'xxx yyy zzz.dat' would be xxx~1.dat or something like that. You can use the dir command to find our what it actually is.

You could replace it with an underscore using the 'ren' or 'mv' commands depending on what OS you're working on.

Also MATLAB may have a solution here:

http://www.mathworks.com/matlabcent...use-fopen-to-generate-a-valid-file-identifier

or here if its the spaces issue:

http://www.mathworks.com/matlabcent...s-with-load-when-path-or-filename-has-space-s

 

[FactChecker]

The format in sprintf is wrong. As the error message says, you are trying to open a file with an invalid file name. Remove the semicolon from the end of that line and work it till the printed fileNames are what you want. You might as well skip or comment out other lines till you get your file names right.

Try using fileList = dir( '*.dat') to get the list of files in the directory.