Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Easy: importing a batch of data into MATLAB

  1. Jul 24, 2009 #1
    I've been going crazy trying to figure this out- I'm not seeing it in the documentation. I have a few hundred files named "data*" where * goes from 000 to 300. The file type is really just a text file without any real type- they aren't .txt. Is there a way to 'batch' import all of them without selecting each one individually and importing? They are all in one folder, and they are all formatted identically as having two columns.

    I've been trying all sorts of tricks using data* and the 'importdata' syntax, but I'm stuck. Maybe there is a loop I can try, if you know of any.

    Any suggestions?
     
  2. jcsd
  3. Jul 24, 2009 #2
    Why not just write a script file that loops through all the files in the directory and imports them if they meat your criteria.
     
  4. Jul 25, 2009 #3

    MATLABdude

    User Avatar
    Science Advisor

    This may be easy for you, and easy for me, but not so easy for someone new(ish) to MATLAB.

    If you're just starting out, MATLAB's (internal) documentation is pretty good, and the online documentation is even better (and gives quite helpful suggestions for related functions--think Amazon without trying to sell you something). For instance, typing the following brings up the internal help for the dlmread function, delimited read from ASCII text file:
    >>help dlmread

    The online documentation for the dlmread function would be at:
    http://www.mathworks.com/access/helpdesk/help/techdoc/ref/dlmread.html

    Okay, basics aside, you can harness the interpretive nature of MATLAB using the eval function, which evaluates a text string (which you can generate on the fly!) Okay, let's assume that you want to import myData000.txt to myData999.txt (numerical data stored without a header row, and in 2 columns--separated by commas--with 100 rows--separated by carriage return). The following code would probably work, and produce a cell array (myData) with 1000 entries, each containing a 2x100 matrix of values.

    Code (Text):

    for i=0:999
        % The setup
        comStr=['myData{', num2str(i), '}=dlmread(''myData'];

        % Handling leading zeros
        if i<10
            comStr=[comStr, '00', num2str(i), '.txt)'''];
        else if i<100
            comStr=[comStr, '0', num2str(i), '.txt)'''];
        else
            comStr=[comStr, num2str(i), '.txt)'''];
        end %if
       
        % Make it so!
        eval(comStr);
    end %for
     
    I don't have appropriate datasets, nor a copy of MATLAB on this computer, so you'll probably need to do some tweaking to make this work properly. And some more tweaking to make it work the way you want it to work. Plus some reading.

    Good luck, and remember the MATLAB in-program and online documentation!

    EDIT: Forgot that when you need to put an apostrophe inside a text string, you need two of them together: '' (and no, not shift-apostrophe). Three of them together is interpreted as: put an apostrophe into the string, then terminate the string.
     
    Last edited: Jul 25, 2009
  5. Jul 25, 2009 #4

    Yes, I certainly ran into this problem when learning MATLAB. Unfortunatly I've always found MATLAB really bad for this. It has been pointed out how MATLAB allows you to do this (by writing a script). However, I've always found it more convenient to write a shell script or a c++ program to parse the data into a new MATLAB friendly file rather then deal with how MATLAB scripts these things.
     
  6. Jul 25, 2009 #5
    I'm not sure how I would go about writing a script in C++...

    Thanks, MATLABdude, I'll try to get the dlmread function to work for my particular scenario and report back.
     
  7. Jul 26, 2009 #6
    I figured out a way to do this. It's very inelegant, but it definitely suffices:

    for i=1:50 (just 50, say)
    if i<10
    x=['data00',num2str(i)];
    else
    x=['data0',num2str(i)];
    end %if
    data(:,2*i-1:2*i) = importdata(x);
    !this stores all the data in a matrix, 2 columns of data at a time
    end %for

    Again, nothing fancy, but it works.

    If I want all the first columns on the left side of the 'data' matrix and all the second columns on the right (instead of alternating first, second, first, second..) I suppose I can write to the data matrix like this:

    data(:,i:50:50+i)

    But I haven't tried that yet.

    Thanks for the ideas, everyone!
     
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook




Similar Discussions: Easy: importing a batch of data into MATLAB
  1. Matlab and Batch mode (Replies: 2)

Loading...