Mathematica - sorting NSolve solutions

[MartinV]

Hello, everyone.

I have a code written in Mathematica that loads a single file in turn. Each file contains a 200×2 array of numbers which are x,y coordinates of points. I feed each line of each file in a loop that contains a NSolve equation system. The equation system returns 5 outputs for every input. The problem is that those solutions are not sorted.

The input points represent positions of a circle, each file a different position of that circle. The five outputs sometimes give me a whole circle (or whatever shape the input circle turns into), sometimes two or three files contain fractions of those shapes. Sometimes the outputs are even complex. If I plot everything and make an animation with all the different circle positions, I only get real solutions and all the shapes are complete. But now I want to use those output arrays into another program, one that would calculate the surface of those shapes. If I want to do this, I need to put those fractions of shapes together so that each file contains points that mark a single shape.

Here is the code I use for doing the NSolve bit:

SetDirectory["C:\Documents and Settings..."];

p = 1.0; q = 0; eps = 0.5; M = 187;

For[j = 1, j <= M, j++,
P = Import["P" <> ToString[j] <> ".txt", "Table"]; b = {}; c = {};
d = {}; e = {}; f = {};
For[i = 1, i <= 200, i++,
a = NSolve[{x - (1 - eps)*x/(x^2 + y^2) -
eps*(x - p)/((x - p)^2 + (y - q)^2) == P[[i, 1]],
y - (1 - eps)*y/(x^2 + y^2) -
eps*(y - q)/((x - p)^2 + (y - q)^2) == P[[i, 2]]}, {x, y}];
aa = Sort[a];
b = Append[b, {aa[[1, 1, 2]], aa[[1, 2, 2]]}];
c = Append[c, {aa[[2, 1, 2]], aa[[2, 2, 2]]}];
d = Append[d, {aa[[3, 1, 2]], aa[[3, 2, 2]]}];
e = Append[e, {aa[[4, 1, 2]], aa[[4, 2, 2]]}];
f = Append[f, {aa[[5, 1, 2]], aa[[5, 2, 2]]}];
]
Export["Q" <> ToString[j] <> ".txt", b];
Export["R" <> ToString[j] <> ".txt", c];
Export["S" <> ToString[j] <> ".txt", d];
Export["T" <> ToString[j] <> ".txt", e];
Export["U" <> ToString[j] <> ".txt", f];
]

This particular code exports the outputs into .txt files just like the input files are. I have a similar code, only this one plots all the solutions, making an animation. I tried using Sort[] but it didn't do anything. Up until now I tried using Matlab to import the newly made files but that is also giving me problems. Now I realize I need to have both the animations and the surface graphs on one screen so I need to make this work in Mathematica now.

Any help would be greatly appreciated.

 

[Dale]

The problem is that those solutions are not sorted. ... I tried using Sort[] but it didn't do anything.

Have you tried using something other than the default ordering function:
Sort[{a,b,c},f]
where f is some function that takes two elements of your list and returns True if they are in the correct order and False if they are not.

 

[MartinV]

I thought of doing a Sort rule but I just can't figure out what to use. I'm quite new with Mathematica and don't know how to use it properly. Do you have any ideas?

 

[Dale]

How can you tell if two elements of the list are in order or out of order?

 

[MartinV]

I plot them. I want each of the output arrays to represent a single object. If the input is 200 points that represent a circle, outputs are usually something ellipse-like. Sometimes that ellipse is drawn by two or more colours, meaning the points belong to several arrays. I could sort them by x and y-axis or by absolute value which would mean distance from zero.

 

[Dale]

So then you would make a function like:
f[a_,b_]:=OrderedQ[{Abs[a],Abs}]

and then call
Sort[{a,b,c},f]

 

[MartinV]

I would never think of that myself. Thanks. I'll try it and report tomorrow how it worked.

 

[MartinV]

Been sick for a few days.

Anyway, that sorting thing didn't work unless I'm missing something.

 

[Dale]

Are the elements of the list in ascending order of absolute value? Some more detail would help. I can't tell if the problem is that the Sort itself didn't work or if it worked but the result wasn't what you really needed.

 

[MartinV]

No.

The five files I need to sort through are outputs of NSolve. The input file contains points of a circle, starting at the right (if the circle would have the center at 0,0, then I start at R,0). So the x values first go down and y goes up, then y goes down while x still goes down, then x starts to climb while y descends and then climbs back up to the starting point.

 

[Bill Simpson]

It is somewhat difficult to tell what it is you want to do because we can't see your input data files of hundreds of data points or your hundreds of solutions or what it is you want sorted into what order.

Would it be possible for you to temporarily scale this back to say 12 points and show exactly the input you want to sort and exactly what you want the output from sort to be on those 12 points? With the actual Mathematica form of the input and your manually created result then it would probably be feasible for someone to write a line or two of code that will sort these into the order you desire.

 

[Dale]

Yes, Bill Simpson has a good idea. Note that we are just talking about the inputs and desired outputs of the Sort, not your overall procedure and the files.

 

[MartinV]

That's a good idea.

This is the input file.
5.4045085e-001 2.2938926e-001
5.1545085e-001 2.4755283e-001
4.8454915e-001 2.4755283e-001
4.5954915e-001 2.2938926e-001
4.5000000e-001 2.0000000e-001
4.5954915e-001 1.7061074e-001
4.8454915e-001 1.5244717e-001
5.1545085e-001 1.5244717e-001
5.4045085e-001 1.7061074e-001
5.5000000e-001 2.0000000e-001

Left column x coordinates, right line y coordinates. Points roughly represent the circle. I put these points in the code above. Result for one point looks like this:

{{x -> 1.4768, y -> -0.402992}, {x -> 0.595122,
y -> -0.733527}, {x -> 0.596497, y -> 0.999175}, {x -> 0.51045,
y -> -0.0682166}, {x -> -0.50395, y -> -0.298782}}

Strangely this particular calculation didn't spawn complex values but I'm not interested in the complex ones anyway.

This is how unsorted numbers look like when plotted:

I need each elipse to be in a separate array.

I hope it's now clear what I'm trying to achieve.

 

[Dale]

5.4045085e-001 2.2938926e-001
5.1545085e-001 2.4755283e-001
4.8454915e-001 2.4755283e-001
4.5954915e-001 2.2938926e-001
4.5000000e-001 2.0000000e-001
4.5954915e-001 1.7061074e-001
4.8454915e-001 1.5244717e-001
5.1545085e-001 1.5244717e-001
5.4045085e-001 1.7061074e-001
5.5000000e-001 2.0000000e-001

...

{{x -> 1.4768, y -> -0.402992}, {x -> 0.595122,
y -> -0.733527}, {x -> 0.596497, y -> 0.999175}, {x -> 0.51045,
y -> -0.0682166}, {x -> -0.50395, y -> -0.298782}}

This doesn't make sense as input and output for the Sort function. The input and the output for the Sort will be the same list elements, just in a different order. That is what sorting does.

Btw, are you sure that you really want the list of points sorted. Wouldn't you rather have it grouped into separate ellipses?

 

[MartinV]

The first array is the input of the entire code. The bracketted numbers are what I need to have sorted.

Grouped into separate ellipses is good but they need to be in proper order because I will be calculating the surface area of the ellipses.

BTW, the sorting is suppose to group the points into the right ellipses. Hence for each input point I get five outputs, each a part of its own ellipse.

 

[Bill Simpson]

I asked for unsorted points so I could show how to sort them. You responded with a list of 5 points. Excellent. Exactly what I asked for. So the first thing I do is plot them to see what I am facing, how they are out of order and what I am going to need to do to get them to plot as a nice ellipse..

ListPlot[Map[{x, y} /.#&, {{x->1.4768, y-> -0.402992}, {x->0.595122, y-> -0.733527}, {x->0.596497, y->0.999175}, {x->0.51045, y-> -0.0682166}, {x-> -0.50395, y-> -0.298782}}], PlotJoined->True]

Try that. See what I'm seeing here now. Then suppose you have nothing else to go on and you need to decide how to sort that so those points will plot as a nice ellipse. I'm perplexed.

Now two steps backward. When I look at your two ellipse plots you showed above I see two different colored arcs in each one. I also see the "end points" of the arcs are "off the chart." But I don't think I see any hint of 200 points on an ellipse that the order of the points has been scrambled so that the plot is zig zagging back and forth across the ellipse and if I just sorted them into order all the zigging would disappear and I would have a nice ellipse.

Item 1: I'll guess that the two different colored arcs in your plots mean that if you look at your data really carefully you will see your points are in two different lists. Early versions of ListPlot could only plot a single list of points. Later versions of ListPlot when given a list of two or more lists of points will plot each list separately and will do each in a separate color. I realize in a forest of points you could easily overlook that you have multiple lists being given to ListPlot. But go carefully look and see if this is the case. If so try to figure out why and see if you can fix that.

Item 2: I don't believe your problem has anything to do with sorting. I believe somehow your "endpoints" of your arcs are just wrong. I can't even begin to imagine why. But I would go look very carefully at the data and the equations and see whether a couple of your equations result in wild values for solutions.

 

[MartinV]

Item 1: I'll guess that the two different colored arcs in your plots mean that if you look at your data really carefully you will see your points are in two different lists. Early versions of ListPlot could only plot a single list of points. Later versions of ListPlot when given a list of two or more lists of points will plot each list separately and will do each in a separate color. I realize in a forest of points you could easily overlook that you have multiple lists being given to ListPlot. But go carefully look and see if this is the case. If so try to figure out why and see if you can fix that.

This is what I was talking about the whole time. I find one set of solutions in two or possibly more arrays. I want to sort them so I have one solution in one array, second solution in another array.

 

[Dale]

That is not sorting, that is grouping or clustering. There is a built in function for clustering.

 

[MartinV]

I did not know clustering exists. I am rather new in using Mathematica.

I'm not sure how to use the FindCluster method. Do I use it everytime a loop does the NSolve for a new point? Because that's only five pairs of numbers and clustering them seems a bit bizzare. I can use the cluster thing when I have all 1000 pairs of points (5 outputs × 200 points) but the points are already separated in the five arrays. Should I throw all five groups together and then use FindCluster?

I know I look like a complete "#$ by now but I just can't seem to get a clear picture of Mathematica.

 

[Bill Simpson]

If you had 0.01% of the trouble with a toaster that you WILL have with Mathematica you would have thrown the toaster down the stairs and got one that was easier to get along with.

Now, back to your real problem. Can you plot all those ellipses at once, forget trying to think about Mathematica syntax, look at the plot and say something like "well the first ellipse is always going to have y values between .5 and .7, the second ellipse is always going to have y values between .9 and 1.2, etc"?

Write down five lines of english, one for each ellipse, that you could tell a mindless assistant to use to divide all the points up into five different bins. Then get ALL your {x->1.,y->.8} into a single large list, not nested lists of nested lists. Call that say gobsofpoints. Note: If they are not in a single list you can get rid of the nesting using something like gobsofpoints = Partition[Flatten[mynestedgobofpoints], 2]

Now to extract the points for one of the ellipses. Suppose the first ellipse has .5<=y<=.7. Use this to extract only those points.

firstellipse = Select[gobsofpoints, .5 <= Last[{x, y} /. #] <= .7 &]

Change the .5 and .7 to capture your secondellipse, thirdellipse, etc.

If your ellipses are perhaps overlapping and you cannot divide them up this way then it is going to get more interesting.

 

[Dale]

the points are already separated in the five arrays.

What does this mean?

 

[MartinV]

It means that if you look at the code in the first post, you will see that the equation system gives five outputs for each input and every solution of NSolve goes into its own array.

Here I have plotted each solution in different color. You can see that sometimes the color jumps from up high to way down. My goal is to make every ellipse whole, not jumping up and down.

Here is the whole plot. The red circles are the input files, the rest (yellow, black, blue, magenta, green) are the five outputs.

 

[Dale]

So each solution from NSolve gives you exactly one point from each of 5 ellipses. The first point is from one of the yellow ellipses, the second point is from one of the black ellipses, the third from blue, fourth from magenta, and fifth from green? And the problem is that sometimes the first point, the yellow one, is actually in a region that should be blue or magenta, and vice versa. So you want to look at the solution, determine if the yellow one is out of place, and if so, swap the yellow, blue, and magenta solutions so that they are each in the right region. Is that correct?

Is there actually any significance to the color of the ellipses? I.e. do you really need to see the transition between the yellow and black ellipses, or is this just a way to avoid the big streaks and otherwise you don't care about the colors?

 

[MartinV]

Yes, that is what I want.

I used colors so all five series are distinguishable.
The thing is when two ellipses come close, they start to form a combined shape. The number of points is always the same (5 × 200) but some have complex value. I don't include complex points in the plot. Here's an example:

 

[MartinV]

I tried doing it like this;

(looped as before)
a = NSolve[...]; % my function as before
b = Append[b, a]; % new solutions are added to the previous for each of the 200 input points
c = Partition[Flatten, 2]; % to create 1000 lines of {x -> 111, y -> 222}

Now I try to do FindClusters[c,5] and I get:

FindClusters::amtd:
FindClusters is unable to automatically select an appropriate dissimilarity function for the input data

What am I doing wrong (again)?

 

[Dale]

Try this:
sortCoords[{yel_, bla_, blu_, mag_, gre_}] := Join[Sort[{yel, blu, mag}, OrderedQ[{#1[[2, 2]], #2[[2, 2]]}] &], {bla, gre}]

and replace Sort[a] by sortCoords[a]

 

[MartinV]

I did the sortCoords thing. The coloring of the plot is different, so some numbers have shifted from one group to another. It's still mixed, unfortunately.

 

[Dale]

Can you show me?

 

[MartinV]

 

[Dale]

It looks fine and unmixed to me.

 

[MartinV]

The blue color is in two ellipses as is the red and the green. All shapes are composed of multiple colors.

 

[Dale]

Oh, I can't see that. Can you re-do the second plot from 22 with the new sort function?

But try this first:
sortCoords[{yel_, bla_, blu_, mag_, gre_}] := Join[Sort[{bla, blu, mag}, OrderedQ[{#1[[2, 2]], #2[[2, 2]]}] &], {yel, gre}]

 

[Bill Simpson]

Watching this exchange reminds me of something I read ten or twenty years ago. The AI researchers had spent vast amounts of time and effort trying to get a vision system and robot machine to pick odd shaped parts of of a pile of parts in a bin. Then one afternoon someone asked the question:

Why are you tossing the parts in the bin and then asking the robot to do the hard work of recognizing them and taking them back out, why don't you just not throw the parts in the bin in the first place?

Could your code perhaps be organized in a slightly different way so that the process of identifying the individual ellipses would already be done for you?

 

[MartinV]

This is done with the last sortCoords:

sortCoords[{yel_, bla_, blu_, mag_, gre_}] := Join[Sort[{bla, blu, mag}, OrderedQ[{#1[[2, 2]], #2[[2, 2]]}] &], {yel, gre}]

It's an improvement from before but it's not exactly what it's suppose to be.

 

[MartinV]

Could your code perhaps be organized in a slightly different way so that the process of identifying the individual ellipses would already be done for you?

I'm willing to try every aspect. Do you have anything specific in mind?