Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Read file from multiples .dat files and choose only one point

  1. Apr 20, 2014 #1
    Hi all.

    I have 1200 .dat files containing x,y,z . What I need to do is to open all the files and choose only one single point from all the files, in example point (x120,y120,z120) from all 1200 .dat file and save it into new .dat file. Can anyone teach me how to do that either in fortran or matlab

    thank you

    Attached Files:

  2. jcsd
  3. Apr 20, 2014 #2


    Staff: Mentor

    Why not use AWK or some other scripting language?

    In awk, this would be particularly easy pretty much a one-liner.
  4. Apr 21, 2014 #3
    actually, I did wrote code in matlab

    Code (Text):

    numFiles = 822;
    myData = cell(1,numFiles);
    startRow = 6886;
    endRow = 6886;

    for fileNum = 1:numFiles
        fileName = sprintf('*.dat',fileNum);
        myData{fileNum} = importfile(fileName,startRow,endRow);

    %% Format string for each line of text:
    formatSpec = '%*24s%12f%[^\n\r]';

    %% Open the text file.
    fileID = fopen(filename,'r');

    % This call is based on the structure of the file used to generate this
    % code. If an error occurs for a different file, try regenerating the code
    % from the Import Tool.
    dataArray = textscan(fileID, formatSpec, endRow-startRow+1, 'Delimiter', '', 'WhiteSpace', '', 'HeaderLines', startRow-1, 'ReturnOnError', false);

    %% Close the text file.

    %% Allocate imported array to column variable names
    VarName3 = dataArray{:, 1};

    but I got this error

    Error using textscan
    Invalid file identifier. Use fopen to generate a valid file identifier.

    Error in importfile (line 34)
    dataArray = textscan(fileID, formatSpec, endRow(1)-startRow(1)+1, 'Delimiter', '', 'WhiteSpace',
    '', 'HeaderLines', startRow(1)-1, 'ReturnOnError', false);

    Error in readdata (line 18)
    myData{fileNum} = importfile(fileName,startRow,endRow);

    and I am actually don't understand the reason of this error. Only one in my mind is the filename of datafile is data_Ez 0001.dat and so on. Is it the white space in filename maybe the reason? If yes, how I can remove all the whitespace in all 822 datafile?

    thank you
  5. Apr 22, 2014 #4


    Staff: Mentor

    The whitespace is most likely the culprit. Try an experiment and change the names removing the space and see what happens.

    This feature of windows has caused many problems over the years. The most common case is that you create a bat file where you pass in the filename with the space and script in turn interprets it as two arguments not one.

    An alternative if this is windows is to try the short name of the file ie file 'xxx yyy zzz.dat' would be xxx~1.dat or something like that. You can use the dir command to find our what it actually is.

    You could replace it with an underscore using the 'ren' or 'mv' commands depending on what OS you're working on.

    Also MATLAB may have a solution here:


    or here if its the spaces issue:

  6. May 11, 2014 #5


    User Avatar
    Science Advisor
    Gold Member
    2017 Award

    The format in sprintf is wrong. As the error message says, you are trying to open a file with an invalid file name. Remove the semicolon from the end of that line and work it till the printed fileNames are what you want. You might as well skip or comment out other lines till you get your file names right.

    Try using fileList = dir( '*.dat') to get the list of files in the directory.
    Last edited: May 11, 2014
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook