Python equivalent of MATLAB textscan?

[JesseC]

Is there one?

Or do I really have to write something like this:

from numpy import *

with open('file.txt','r') as f:
    #read only data, ignore headers
    lines = f.readlines()[31:]
    
    # convert strings to floats and put into arrays
    for i in xrange(len(lines)):
        s = lines[i].split()
        y1[i] = float(s[0])
        y2[i] = float(s[1])
        y3[i] = float(s[2])

 

[Ibix]

A quick Google search suggests that - slightly to my surprise - there isn't a built in parser like textscan (similar to fscanf, for C/C++ aficionados). The canonical solution is to learn regular expressions and use the re module - a useful skill, by the way. However, if you're just reading in lists of floats, your way is good enough.

A slightly neater version of what you've done is:

with open('file.txt','r') as f:
    #read only data, ignore headers
    lines = f.readlines()[31:]
    # create the arrays (you forgot this step)
    y1=[0]*len(lines)
    y2=[0]*len(lines)
    y3=[0]*len(lines)
    # convert strings to floats and put into arrays
    for i in xrange(len(lines)):
        y1[i],y2[i],y3[i] = float(s) for s in lines[i].split()

which has the added advantage of erroring out if your assumption that there are exactly three floats per line is wrong.

A more pythonic way of doing it, if you aren't too wedded to your variable names, is:

with open('file.txt','r') as f:
    #read only data, ignore headers
    y=[[float(s) for s in line.split()] for line in f.readlines()[31:]]

which gives you a list of lists. What you called y1 is now y[0], y2 is now y[1], and y3 is y[2]. Presumably you're just going to make a numpy array anyway, so you can just transpose() if the array indices are now in the wrong order. Note that this version doesn't care if the data file isn't in the right format and will happily load a ragged list if that's what's in the file - so the first option may be better.

Note also that I haven't checked whether readlines() strips trailing newlines - you may need to do that to prevent the float() failing on the last element.

Edit: Note that I haven't actually run any of the code above - it looks right, but caveat programptor.

 

[JesseC]

Thanks Ibix, that's really helpful. I need to keep variable names because in reality they are not actually y1, 2 etc but are physical variables like temperature and pressure. Guess I'll stick to what I've done for the moment.

 

[Ibix]

Fair enough. I'd still suggest my first bit of code as a slight improvement - it gives you a bit better chance of spotting messed-up data (it'll fail if there's too much data on a line as well as too little) when you try to load it instead of when the results make no sense. If you trust your data, it makes no difference.

 

[gsal]

don't know MATLAB nor textscan

but in python with numpy and its genfromtxt (generate from text) function...you can read an entire array at a time and have a way to skip heading line, trailing line, skip desired columns, assign default value to missing ones, etc...read up and see if it is something you can use.

 

[JesseC]

Thanks gsal that is exactly what I was looking for! It has shortened my code massively:

from numpy import *
       
# open test data
testdata = genfromtxt('file.txt',dtype='float',skip_header=31)
y1 = testdata[:,0]; y2 = testdata[:,1]; y3 = testdata[:,2]

Not sure how I missed that in my google searching, I guess the function name is kinda unusual.