Easy: importing a batch of data into MATLAB


by imost
Tags: batch, data, importing, matlab
imost
imost is offline
#1
Jul24-09, 08:04 PM
P: 3
I've been going crazy trying to figure this out- I'm not seeing it in the documentation. I have a few hundred files named "data*" where * goes from 000 to 300. The file type is really just a text file without any real type- they aren't .txt. Is there a way to 'batch' import all of them without selecting each one individually and importing? They are all in one folder, and they are all formatted identically as having two columns.

I've been trying all sorts of tricks using data* and the 'importdata' syntax, but I'm stuck. Maybe there is a loop I can try, if you know of any.

Any suggestions?
Phys.Org News Partner Science news on Phys.org
Lemurs match scent of a friend to sound of her voice
Repeated self-healing now possible in composite materials
'Heartbleed' fix may slow Web performance
John Creighto
John Creighto is offline
#2
Jul24-09, 09:12 PM
P: 813
Quote Quote by imost View Post
I've been going crazy trying to figure this out- I'm not seeing it in the documentation. I have a few hundred files named "data*" where * goes from 000 to 300. The file type is really just a text file without any real type- they aren't .txt. Is there a way to 'batch' import all of them without selecting each one individually and importing? They are all in one folder, and they are all formatted identically as having two columns.

I've been trying all sorts of tricks using data* and the 'importdata' syntax, but I'm stuck. Maybe there is a loop I can try, if you know of any.

Any suggestions?
Why not just write a script file that loops through all the files in the directory and imports them if they meat your criteria.
MATLABdude
MATLABdude is offline
#3
Jul25-09, 02:23 AM
Sci Advisor
P: 1,724
Quote Quote by John Creighto View Post
Why not just write a script file that loops through all the files in the directory and imports them if they meat your criteria.
This may be easy for you, and easy for me, but not so easy for someone new(ish) to MATLAB.

If you're just starting out, MATLAB's (internal) documentation is pretty good, and the online documentation is even better (and gives quite helpful suggestions for related functions--think Amazon without trying to sell you something). For instance, typing the following brings up the internal help for the dlmread function, delimited read from ASCII text file:
>>help dlmread

The online documentation for the dlmread function would be at:
http://www.mathworks.com/access/help...f/dlmread.html

Okay, basics aside, you can harness the interpretive nature of MATLAB using the eval function, which evaluates a text string (which you can generate on the fly!) Okay, let's assume that you want to import myData000.txt to myData999.txt (numerical data stored without a header row, and in 2 columns--separated by commas--with 100 rows--separated by carriage return). The following code would probably work, and produce a cell array (myData) with 1000 entries, each containing a 2x100 matrix of values.

for i=0:999
	% The setup
	comStr=['myData{', num2str(i), '}=dlmread(''myData'];

	% Handling leading zeros
	if i<10
		comStr=[comStr, '00', num2str(i), '.txt)'''];
	else if i<100
		comStr=[comStr, '0', num2str(i), '.txt)'''];
	else
		comStr=[comStr, num2str(i), '.txt)'''];
	end %if
	
	% Make it so!
	eval(comStr);
end %for
I don't have appropriate datasets, nor a copy of MATLAB on this computer, so you'll probably need to do some tweaking to make this work properly. And some more tweaking to make it work the way you want it to work. Plus some reading.

Good luck, and remember the MATLAB in-program and online documentation!

EDIT: Forgot that when you need to put an apostrophe inside a text string, you need two of them together: '' (and no, not shift-apostrophe). Three of them together is interpreted as: put an apostrophe into the string, then terminate the string.

maverick_starstrider
maverick_starstrider is offline
#4
Jul25-09, 04:54 AM
maverick_starstrider's Avatar
P: 1,153

Easy: importing a batch of data into MATLAB


Quote Quote by imost View Post
I've been going crazy trying to figure this out- I'm not seeing it in the documentation. I have a few hundred files named "data*" where * goes from 000 to 300. The file type is really just a text file without any real type- they aren't .txt. Is there a way to 'batch' import all of them without selecting each one individually and importing? They are all in one folder, and they are all formatted identically as having two columns.

I've been trying all sorts of tricks using data* and the 'importdata' syntax, but I'm stuck. Maybe there is a loop I can try, if you know of any.

Any suggestions?

Yes, I certainly ran into this problem when learning MATLAB. Unfortunatly I've always found MATLAB really bad for this. It has been pointed out how MATLAB allows you to do this (by writing a script). However, I've always found it more convenient to write a shell script or a c++ program to parse the data into a new MATLAB friendly file rather then deal with how MATLAB scripts these things.
imost
imost is offline
#5
Jul25-09, 11:24 AM
P: 3
I'm not sure how I would go about writing a script in C++...

Thanks, MATLABdude, I'll try to get the dlmread function to work for my particular scenario and report back.
imost
imost is offline
#6
Jul26-09, 01:17 PM
P: 3
I figured out a way to do this. It's very inelegant, but it definitely suffices:

for i=1:50 (just 50, say)
if i<10
x=['data00',num2str(i)];
else
x=['data0',num2str(i)];
end %if
data(:,2*i-1:2*i) = importdata(x);
!this stores all the data in a matrix, 2 columns of data at a time
end %for

Again, nothing fancy, but it works.

If I want all the first columns on the left side of the 'data' matrix and all the second columns on the right (instead of alternating first, second, first, second..) I suppose I can write to the data matrix like this:

data(:,i:50:50+i)

But I haven't tried that yet.

Thanks for the ideas, everyone!


Register to reply

Related Discussions
Matlab - importing data from excel & interpolating Programming & Computer Science 1
Importing MATLAB Plots into MATHCAD Math & Science Software 0
Importing data into MATLAB Math & Science Software 0
Matlab Importing Files Math & Science Software 1