Inverse best fit slope expected vs calculated

[paqqj]

inverse best fit slope...expected vs calculated

Given a set of data (X,Y) pairs, we can get the best fit line using many techniques. Assume the correlation is not perfect. We can use the least squares method to get the best fit line slope and intercept. So we have Y = mX + b

In other words, given some X, we can determine the most likely value of Y.

However, I am confused with this next step.
Wouldn't the inverse also hold true? In other words, given some Y, shouldn't I be able to calculate the most likely X based on rearranging the formula Y = mX + b into
X = Y/m - b/m ?

But why is it that if I plot my original data on opposite axes, so that the X is on the Yaxis and vice versa, and I calculate the best fit slope and intercept on this data, I don't get the expected slope of 1/m nor the expected intercept of -b/m ?

I will cheat a little here and just post the answers I got using Excel to show you the difference.

The three data points (1,3) (2,5) and (3,12) result in m=4.5 and b= -2.3333333
Y = 4.5 X - 2.33333333

Rearrange that to get X = 0.22222222 Y + 0.5185185

However, if the original points are interchanged into (3,1) (5,2) and (12,3)
using Excel or other least squares techniques yields m=0.2014925 and b=0.656716

Shouldn't the expected slope be the same as the calculated one?P.S. The other data set I'm working with has 2,000 points, and the expected slope is 1.71 while the calculated slope is 0.96. Definitely not the 1/m I was expecting!

 

[Stephen Tashi]

If regression treated x and y variables on the same footing then the regression line would minimize the squares of the distances between each data point and the regression line, but it does not. The shortest distance between a data point and the regression line is the distance from the point perpendicular to the line. If you treat where that distance hits the line as the predicted (xp,yp) and the data as the actual (xd,yd) then the error between the predicted and actual involves both an error in x and an error in y. (There has been at least one book written on the problem finding a line that minimizes the squares of the distances from the data points to a line. The problem has a special name that I don't recall at the moment. I can look it up if you are interested.)

In regression analysis, the regression line minimizes the errors of prediction assuming that we know one variable x perfectly and are trying to predict y. It minimizes only the errors in y, which are measured "straight up and down" , not perpendicular to the regression line. So if you reverse the role of the variables x and y, you are changing the assumptions of the problem and shouldn't expect to get the same line or the same line with the axes reversed.

 

[AlephZero]

The difference between the two fitted lines can be used to measure how "good" the straight line fits are.

Google for "covariance matrix" and "correlation coefficient" for the math.

 

[paqqj]

Thanks I get what you are both saying. I did notice that when the correlation was 1.0 then X and Y could be swapped and the new best fit slope became 1/m.

Is there a way to get a slope such that the inverse of Y=mX +b is X = Y/m - b/m when X and Y are swapped?

Here are the specifics of the problem I am trying to solve. I have hourly wind data from 2 nearby stations. There is some degree of relationship between the two (correlation = 0.75). However, the data sets for the time period I'm looking at are incomplete for each station, so I would like to "best guess" what the wind is at the missing station for some hours, and also best guess for the other station at some other hours.

When I use X = wind at station 1
Y = wind at station 2
I get a slope around 0.59 and intercept +5.
Say X=50KT, then Y could be guessed to be 34.5


But when X = wind at station 2
Y = wind at station 1
I get a slope 0.96 and intercept +7

But if Y=34.5, then X becomes 40.0, not 50.

So in my case "best fit" is not the BEST fit. Has someone already created what I'm looking for, or is the mathematics simply too complicated? I would like to think this must be a common problem.

 

[Stephen Tashi]

You aren't clearly defining "the problem".

On the one hand, this might be a real world problem in the sense that we might assume a physical model for the process. If so, we can apply mathematics to describe it. Are the results going to be used for some practical purpose? If so, we might be able to precisely define what a "best" solution is and find it.

On the other hand this might simply be collection of clerical requirements and your personal preferences. For example, why not use two different regression lines? Is that inconvenient? Do you think it is "wrong"? The line that minimizes the squares of the distances from each of the (known) data points to the line is found by solving "the linear total least squares" problem, http://en.wikipedia.org/wiki/Total_least_squares. That line has the symmetry you want, if this is merely a question of aesthetics.

 

[paqqj]

I have archived data of several years of hourly wind speed from 2 nearby stations.
The first station no longer reports anything, but the second one does.
I want to know what the best guess of the wind speed at the first station is, based on the latest reading from the second one.

X = known station #2
Y = missing station #1

I have determined that would be Y=0.59X + 5 using regression.


So, then I wanted to know what wind would station #2 be reporting in order for the wind at the non-reporting station exceed 50KT.
So solving the above equation yields X = Y/0.59 - 5/0.59
= 76KT

In other words, when the wind is blowing at 76KT at station #2, it is most likely blowing at 50KT at station #1.

BUT.........

But if I start from the beginning again, and plot the archived data again, this time letting X=station#1
Y=station#2
If I determine the best fit line using data this way, I get Y= 0.96X +7
So now the missing station #1 (aka X) would most likely be reading 50KT when the known station is reading only 55KT.


So the problem is should I be expecting the wind at the non-reporting station to be 50KT when the known station is reading 55 or 76?

 

[AlephZero]

Find the correlation coefficient between the X and Y values. Since you are getting very different least squares equations depending which variable is independent, it will probably be a low value (close to 0).

Also plot the two "best fit" lines and show all the (X,Y) points at dots.

Quite likely, the situation is that the wind speed at one site is not strongly related to the speed at the other. You would not expect them to be closely related if they were a long way apart, or at very different altitudes (e.g one on top of a mountain and the other at the bottom).

 

[Stephen Tashi]

I want to know what the best guess of the wind speed at the first station is, based on the latest reading from the second one.

I have determined that would be Y=0.59X + 5 using regression.

If you really believe those statements, then let's assume the scenario for a typical regression model. We assume the data is generated by the process
Y = 0.59 X + 5 + u where u is an independent random draw from a distribution that models the "error". u[] has mean zero. If we assume it's a normal distribution, you could estimate its variance from the observed errors in the data.

So, then I wanted to know what wind would station #2 be reporting in order for the wind at the non-reporting station exceed 50KT.

I don't know if 50KT is a significant as a threshold value (for example, the minimum speed that certain type of wind turbine could use) or whether you simply chose it as an arbitrary example of estimating the reading at station #2 that is producing a given reading at station #1. Can you clarify your goal?

 

[paqqj]

Can you clarify your goal?

Structural integrity of a building requires certain action when winds exceed 50KT, but due to its remote location and lack of on site anemometer I'd like to estimate the speed based on a known nearby station.

Correlation between the two is 0.75, so not great, but somewhat.

 

[Stephen Tashi]

Is the bottom line of this work to determine something like "Estimate many hours per year the winds at station #1 will exceed 50KT given the data from station #2"?

Or is it more of a monitoring problem like: "Given the reading at station #2 is some value s, what is the probability that the reading at station #1 is exceeding 50KT"?

 

[paqqj]

Is the bottom line of this work to determine something like "Estimate many hours per year the winds at station #1 will exceed 50KT given the data from station #2"?

Or is it more of a monitoring problem like: "Given the reading at station #2 is some value s, what is the probability that the reading at station #1 is exceeding 50KT"?

Your second point... It is a monitoring problem.

 

[Stephen Tashi]

I must do some home repairs now, but I'll think more about this problem this evening.

One immediate practical suggestion (if you have not already done it) is to see how much the data points for low wind velocities are affecting your regression line. For example data points like (10,3), (4,8) are intuitively not very relevant unless you also get data like (58,3), (62,8). Are you doing the regression based on every single reading that you have?

 

[paqqj]

I did filter out some obviously bad data...there were some archived data that had zero velocity instead of "missing".

I also considered throwing out all the data from either station that was less than some fraction of the other, or greater than some multiplier of the other but I wasn't sure of any bias I might introduce doing that. So didn't proceed with that yet.

Here are two Excel displays showing all the data. Note the different slopes of each graph.

 

[Stephen Tashi]

To me, the simplest approach is assume the equation: Y = 0.59 X + 5 + u
You can compute the data from the u by subtracting the actual value from the predicted value. Then you can plot the distribution of the u (or run some statistical tests on it) to see if it looks like a normal distribution with mean zero and some variance (which you can also estimate).

If u is a normal distribution, then given any X (= a station 2 measurment) you can compute the probability that Y is in any given numerical range of interest using the theory of the normal distribution.

The regression like you would get by reversing X and Y is not relevant to predicting Y given X. I look at it this way: The probability of Y given X need not be the same as the probability of X given Y. The mean value Y_m of all the Y's that are paired with a value of X = X_0 need not be the same as the mean value of all the X's that are paired with a Y = Y_m.

 

[Stephen Tashi]

...and it's probably not wise to get too fixated on that particular math if there are possibilities for getting other data. For example, you might make a better prediction of Y given more variables, such as temperature or other wind data.

 

[paqqj]

Thank you for your help. I admit am having a difficult time understanding the concept. To me, if the speed I need to keep my eye on (when the wind most likely is 50KT at the unknown station) is when the real wind is 76KT at the known one, then in a real world situation one would think that if one was standing at the unknown station and the wind was measured with a temporary anemometer at 50KT, then the wind would most likely be 76KT at the other "known" one... but, these formulas don't show that. They show that if 50 at unknown, then the known one is only 55KT not 76KT.

I want a result that will give me an answer like 50 at the unknown when 66 at the known, and vice versa, when 66 at the known it is 50 at the unknown. That would make my brain happier.
How do I find this relationship?

I am trying to see this from the perspective indicated in the second paragraph. Mathematics be damned! I need a real world visualization!

Sorry if I am just not getting it.

I also appreciate your advice of looking at some other parameters. I plan to look at wind direction and place data into bins.

 

[Stephen Tashi]

Suppose the (X,Y) data is this:
(45,60)
(45,60)
(45,60)
(45,58)
(55,60)
(55,60)
(55,60)
(55,60)
(55,60)
(55,64)

When X = 45, what is the most likely value of Y? Answer 60.

When Y = 60, what is the most likely value of X? Answer 55.

 

[paqqj]

Suppose the (X,Y) data is this:
(45,60)
(45,60)
(45,60)
(45,58)
(55,60)
(55,60)
(55,60)
(55,60)
(55,60)
(55,64)

When X = 45, what is the most likely value of Y? Answer 60.

When Y = 60, what is the most likely value of X? Answer 55.

Those are not the answers I would get.

Using those 10 data points in your example, the best fit relationship between X and Y is:
Y=0.116667 X + 54.25

So when X=45, then Y=59.5

Part two is where I am unsure.
If I simply rearrange Y=0.116667 X + 54.25 to solve for X:

X=8.571404 Y - 465

gives a value for X when Y=60 of 49.28

If I get the best fit relationship between Y and X, that is
X=1.428571 Y - 35

which gives a value for X when Y=60 of 50.71.
What I want is: when X=X1 then Y=Y1, and if Y=Y1 then X=X1.

 

[Stephen Tashi]

You're missing the point of the example. It shows that asking for X when Y is given is a different problem than asking for Y when X is given. There is no reason the two problems should have the same answer. (Your calculations illustrate this, but they do not necessarily answer the question I asked, which was "what is the most likely", not what is the value predicted by linear regression. )