Import and plot multiple data sets in Mathematica

[abonatto]

I have a dat file with multiple data sets, with the following structure:

# t = 0.0 , ...
-10.000 0.00001 1.000001 ...
-9.000 0.00002 0.900001 ...
...
10.000 0.00005 1.000001 ...


# t = 0.2 , ...
-10.000 0.00301 1.000203 ...
-9.000 0.02222 0.900043 ...
...
10.000 0.00025 1.000551 ...

What I need to do is to plot a 3d surface, where the axis are:

- the block data number; in gnuplot I do this using the index as a pseudocolumn "-2", of the data block (splot 'phi_02.dat' u -2:1:4 w l ); this axis is like the time evolution of my system (every 0.2 secs I write a data block in the dat file, for example).

- Column 1 of the dat file (every block has the same values in this column; it is the spatial coordinate, from -10 to 10, for example).

- Any of the other 6 columns (Column 4, for example; I will need to make a plot for each column).

I tried to import the file (data=Import["phi_02.dat"]), but I just could not figure out how to avoid the first "text line" (#t = 0.0, ...), before each data block, and how to plot what I need.

Any suggestions?

 

[natski]

I know it is not optimal but I would suggest writing a shell script to split the data up into unique files and import each file separately.

 

[abonatto]

Well, it is a possibility. But I really would like to learn how to do this in Mathematica...

 

[Bill Simpson]

This will read in a file s.txt in your Mathematica folder as a list with each line as one string.

lines=Import["s.txt","Lines"]

This will break it into blocks based on lines starting with # and discard that # line.

blocks=Map[Rest, Split[lines, StringTake[#2, 1] != "#" &]]

This will break lines on whitespace and turn into numbers.

blocks=ToExpression[Map[StringSplit,blocks]]

This will put the block number in front of each line.

numberblocklines[{b_, l_}] := Map[Prepend[#, b] &, l];
nb = Map[numberblocklines, Transpose[{Range[Length[blocks]], blocks}]]

This will extract a particular block and then extract columns from that block.
The nb[[2]] extracts block 2. The {1,2,4} extracts columns 1, 2 and 4 where column 1 is the block number, 2 was your 1 and is your coordinate and 4 is some column you are interested in plotting.

extractcolumns[m_, l_] := Transpose[Map[m[[All, #]] &, l]];
extractcolumns[nb[[2]], {1, 2, 4}]

Study each of the pieces carefully, working from the inside out, and try them on small examples until you can see how and why this works.

 

[abonatto]

Dear Bill,

Thank you so much for taking your time to help me with this problem. I have just tried with my dat file and I had some issues but, as you suggested (and as I have just started to use Mathematica)), I will take a small file to understand every command you have used.

Cheers.

Alex

 

[abonatto]

Bill:

The issue I had was because the 2 empty lines that I have between each block (I use it because I usually plot them in gnuplot, using the index function): they are transformed in empty elements {} in the list:

{{"10.0 .9406", "20.0 .8749", "30.0 .8030", "40.0 .7195",
"50.0 .6190"}, {}, {}, {"10.0 .9546", "20.0 .9066", "30.0 .8552",
"40.0 .8013", "50.0 .7436"}, {}, {}, {"10.0 .9647", "20.0 .9283",
"30.0 .8904", "40.0 .8518", "50.0 .8124"}, {}, {}, {"10.0 .9720",
"20.0 .9438", "30.0 .9153", "40.0 .8866", "50.0 .8579"}}

I have tried to change (without success) your expression:

blocks = Map[Rest, Split[lines, StringTake[#2, 1] != "#" &]]

to include some condition to discard the empty elements, but I was not able to do it.

Then, I used the Take command:

blocks = Take[blocks, {1, Length[blocks], 3}]

With this, I was able to choose the non-empty elements (1st of every 3) for this specific situation.

Do you have any suggestion about how to delete the empty elements not by their position (as I did choosing the non-empty), but by their "content"?

Cheers.

 

[Bill Simpson]

In[1]:= lines={"this","that","the other","","","#stuff"};
nonemptylines=DeleteCases[lines,""]

Out[2]= {this,that,the other,#stuff}

 

[abonatto]

In your example it worked, but my data is a little bit different (and it is not working):

lines={{1. 10.}, {2. 20.}, {}, {}, {3. 30.}, {4. 40.}, ...}

Reading Mathematica help about DeleteCases, I understood that it can be used to delete elements from a line, but not delete the line itself...

 

[Bill Simpson]

lines=Import["s.txt","Lines"];
nonemptylines=DeleteCases[lines,""];
blocks=Map[Rest, Split[nonemptylines, StringTake[#2, 1] != "#" &]];
blocks=ToExpression[Map[StringSplit,blocks]];
...

 

[abonatto]

It worked perfectly. Thank you so much for all your help!

Mathematica is really an amazing software: I really want to learn more and more about how to use it!

Do you have any suggestion of a good book to learn more about working with imported data? (not any "for dummies" series, with info that we can get straight from Help files, but with good examples!)

Cheers.

Alex.

 

[Bill Simpson]

"Applied Mathematica: Getting Started, Getting it Done"
Very old, but covers the basics and discusses using imported data.
Sometimes available on abebooks.com or amazon.com for a dollar or two.

"Mathematica Navigator: Graphics and Methods of Applied Mathematics"
Several editions, 1st is outdated, 2nd covers up to MMA 5.x, 3rd covers MMA 6.x and a little later.
Considerably more expensive. Encyclopedia of examples.

"Mathematica Cookbook"
Much newer and more expensive. Lots of examples.

A caution: Mathematica in many ways looks like languages you might have seen. But deep down in side it is very very different from what you probably think it is. After hundreds of hours of study and practice you will think you are finally getting to the point where you understand it. Perhaps by 500-1000 hours you will be to the point where you begin to realize it is nothing like you think it is and you will be ready to move to the next level.

 

[abonatto]

Thank you for all these options! I have just bought Applied Mathematica: Getting Started, Getting it Done" from Amazon for 2.24 dollars :)

I will analyze both other options and choose one of them also (to have a newer book than the one I have just bought).

Thanks for your advice also! As a newbie (first time I saw Mathematica, it was version 7, and I have just bought a license for Mathematica 8), I still have everything to learn.

Cheers

 

[abonatto]

One more question:

When I plot my data in a 3D graphic, it seems that Mathematica "cuts" some data...

I was wondering if its something related to the sample that it uses to make the surface. Do you have any suggestion about this?

Thanks again.

 

[a-tom-ic]

Try PlotRange-> Full Option?

 

[abonatto]

Thanks a-tom-ic !

Now it is exactly like I was expecting!

Cheers.