Finding Slope of a Ladder: Is it a Good Idea?

[GlenLaRocca]

I am new here and my response may have lost context info. This is a response to Dale.

As to whether it is a good idea. I considered the example of a ladder tilted out from the base of a building and picked a rather insane way to measure slope, that is with various noisy measurements of height at various distances. Would you use the known information of the ladder starting at the base and find the slope that minimized the weighted sum of squares? In this case, if you had one good measurement near the top and either very poor ones near the bottom or no measurements near the bottom, it would make quite a difference. But then I realized that the measurement near the top mattered more not just if it was accurate but because it is further from the base and gives more observability to the slope, given the constraint--a given lateral error gives a smaller angular error at a longer range. Obviously the tilted ladder is not a weighted LSF problem and yours probably isn't either.

So, like Dale said, don't do it.

 

[GlenLaRocca]

I added the word weighted.

The problem described in the original post asks how to do a weighted least squares fit. A weighted least squares fit takes into account the different standard deviations associated with the different y-values. Hence it is more complicated that minimizing the root sum square of the deviations from a line. Minimizing the root sum square of the deviations without using weights is appropriate when the y-values all have the same standard deviations.

 

[Dale]

Would you use the known information of the ladder starting at the base and find the slope that minimized the weighted sum of squares?

No, I would not, for the reasons both you and I mentioned.

In this case, if you had one good measurement near the top and either very poor ones near the bottom or no measurements near the bottom, it would make quite a difference.

A weighted least squares technique would automatically account for the better quality of that data and adjust the estimate of the intercept accordingly.

 

[GlenLaRocca]

I sense that you did not get my point.

No, I would not, for the reasons both you and I mentioned.
A weighted least squares technique would automatically account for the better quality of that data and adjust the estimate of the intercept accordingly.

 

[GlenLaRocca]

I have thought about it some more and changed my mind. I would definitely use the known information that the ladder is against the base of the building. The obvious case is when you have one measurement of height of the ladder. If you don't use the information, you get no solution. If your one measurement happens to be accurate and far from the base of the ladder, you will get an accurate solution. The expected error in the slope calculation can be calculated based on height accuracy and offset from the building of the measurement. Now what if I had a equally accurate measurement at half the range. I could use that one also and weight it half as much as my far away one. This would improve my slope estimate accuracy. For the leaning ladder problem, you de-weight the closer measurements by its closeness ratio.

No, I would not, for the reasons both you and I mentioned.
A weighted least squares technique would automatically account for the better quality of that data and adjust the estimate of the intercept accordingly.

 

[Dale]

I have thought about it some more and changed my mind. I would definitely use the known information that the ladder is against the base of the building. The obvious case is when you have one measurement of height of the ladder. If you don't use the information, you get no solution. If your one measurement happens to be accurate and far from the base of the ladder, you will get an accurate solution.

If you have only one point then you aren’t doing a fit at all, so it isn’t related to the above discussion. However, in that case you can simply do a propagation of errors to find the error in your calculated value.

 

[GlenLaRocca]

Given one or more measurements of the leaning ladder, there is an unbiased optimal in the least squares sense estimate of the slope that can be calculated based on knowledge of the origin. What is being minimized in the least squares sense is not the original measurements but (in Kalman filter language) the innovations which capture how the raw measurements tell you about what you are trying to measure. The Kalman uses an H matrix to project state space, in this case the state is the slope and the H scalar is x_i. The H value captures how much each measurement tells you about what you are trying to measure (observability)--a measurement right near the origin tells you nothing and the H value is nearly zero and the weighted innovation least square error is nearly zero ( because H is in the weighting).

If you have only one point then you aren’t doing a fit at all, so it isn’t related to the above discussion. However, in that case you can simply do a propagation of errors to find the error in your calculated value.

 

[Dale]

None of that makes coercing the intercept a good idea statistically.

 

[GlenLaRocca]

This is not doing a straight least squares fit. It is another estimation method altogether that is optimal in a least squares sense. The leaning ladder can be cast as a trivial Kalman filtering problem. The Kalman filter is unbiased optimal estimator (in the least squares sense) for a changing system with states with known dynamics and measurements at various times. The measurements may or may not be directly of the states but are related by a measurement matrix. The leaning ladder's state is constant, so the dynamics are zero (constant), and we get all the measurements at once, but the method still applies.

None of that makes coercing the intercept a good idea statistically.

 

[Stephen Tashi]

[Edit: Now that posts have been moved to a new thread, the "OP" is no longer the OP referred to in this post]

The leaning ladder can be cast as a trivial Kalman filtering problem.

The relevance of a leaning ladder problem to the problem in the OP is unclear. Presumably the problem in the OP is focused on predicting the actual $Y_i$ values from measured data $(x_i, y_i)$. The proposed method of prediction implicit in the OP is to take one measured value $y_4$ as a reference, and then predict the values of $Y_i$ by fitting a line to $(x_i, y_4 - y_i)$ data then using that line to predict $Y_i$ by $\hat{Y}_i = y_4 - (\hat{S} x_i + \hat{D})$, where the line $\hat{S}x_i + \hat{D}$ predicts $y_4 - y_i$. The bottom line for errors in prediction is not how well the measured values $y_i$ or $y_4 - y_i$ are predicted and it is not how well $\hat{S}$ predicts the actual slope. The bottom line is how well the actual values $Y_i$ are predicted. Any error in measuring $y_4$ will affect errors in predicting $Y_i$.

The problem is not "What's the best method to predict the $\Delta y_i$ ?".

 

[hutchphd]

This is not that difficult in my (sometimes) humble opinion. We are talking about a fit that minimizes some net RMS variation right? The OP wanted to fit a straight line to five data maybe with errors.

Using a weighting (normalized inverse variances are a good idea) does not really change the RMS fitting formalism. If you want to heavily weight the "origin" give it a small variance (but all the other weights are then renormalized.)...use this as a parameter to adjust if you want to. I agree with @Dale that demanding the "origin" to be invariant is not appropriate.

i frankly don't understand most of the rest of the discussion.

 

[Vanadium 50]

The relevance of a leaning ladder problem to the problem in the OP is unclear

I agree. This thread seems now to be about two different problems. Two different problems might or might not have the same solution.

 

[GlenLaRocca]

I considered the leaning ladder problem because it was a simple way for me to gain insight into a LSF problem where you know one point. I have now gained that insight. I was able to get a much more accurate estimate of the slope than a straight LSF could. That was because I knew (with absolute certainty) that the ladder was straight. When you do a straight LSF you assume nothing about the relationship between the data points, you are only calculating a straight line because it is easy to do.

If the ladder was bent, I couldn't make the strong assumptions that I did. The technique that I applied to the ladder is applicable to the extent you know how linear your data is a priori. If you did know the origin and had some good model for how the linear your data was, you can do better than a straight LSF. The optimal solution always uses all of the information available--that is the measurements and everything you know about what you are trying to measure. Sometimes you might not have a mathematical technique that allows you to do it. Read Gelb, Applied Optimal Estimation for more on the subject.

The relevance of a leaning ladder problem to the problem in the OP is unclear. Presumably the problem in the OP is focused on predicting the actual $Y_i$ values from measured data $(x_i, y_i)$. The proposed method of prediction implicit in the OP is to take one measured value $y_4$ as a reference, and then predict the values of $Y_i$ by fitting a line to $(x_i, y_4 - y_i)$ data then using that line to predict $Y_i$ by $\hat{Y}_i = y_4 - (\hat{S} x_i + \hat{D})$, where the line $\hat{S}x_i + \hat{D}$ predicts $y_4 - y_i$. The bottom line for errors in prediction is not how well the measured values $y_i$ or $y_4 - y_i$ are predicted and it is not how well $\hat{S}$ predicts the actual slope. The bottom line is how well the actual values $Y_i$ are predicted. Any error in measuring $y_4$ will affect errors in predicting $Y_i$.

The problem is not "What's the best method to predict the $\Delta y_i$ ?".

 

[Stephen Tashi]

In the ladder problem, we have to be clear about what the objective function is. There is a distinction between a goal of estimating the slope of the ladder versus estimating the height $y_i$ of the ladder above various points that are distances $x_i$ from the base of the wall. For example if the position of the ladder is estimated by a line that is above the actual position of the ladder but parallel to it then we correctly estimate the slope of the ladder, but incorrectly estimate the position of the ladder vis-a-vis various points on the ground.

We also have to specify the model for how the measurment errors occur. Are there errors in measuring the $x_i$ as well as the $y_i$. Do we make the typical assumption that the errors are independent, mean zero, and identically distributed?

 

[GlenLaRocca]

Understood. In the ladder problem the stated objective was to estimate the slope. The calculations that I described give nonbiased optimal estimate of the slope using the knowledge one end was constrained and the ladder was absolutely linear. I would assert that if you have an optimal unbiased estimate of the slope and an absolutely known intercept, the entire solution is going to be unbiased and optimal.

My solution does not minimize the raw weighted sum of squares of measurements. It know some of them matter less to estimating the slope and re-weight each based on (x offset which is) how much it helps estimate the slope. It is optimal for this admittedly very special problem.

Statisticians are so used to getting all of the information from the measurements. Engineers often know a whole lot of things about what we are measuring. We have learned to exploit that.

Of course!

In the ladder problem, we have to be clear about what the objective function is. There is a distinction between a goal of estimating the slope of the ladder versus estimating the height $y_i$ of the ladder above various points that are distances $x_i$ from the base of the wall. For example if the position of the ladder is estimated by a line that is above the actual position of the ladder but parallel to it then we correctly estimate the slope of the ladder, but incorrectly estimate the position of the ladder vis-a-vis various points on the ground.

We also have to specify the model for how the measurment errors occur. Are there errors in measuring the $x_i$ as well as the $y_i$. Do we make the typical assumption that the errors are independent, mean zero, and identically distributed?

 

[Stephen Tashi]

To make any mathematical conclusions we need a specific model of how the data is generated. As I understand the problem, it can modeled as a line segment of unknown slope, known to intersect the origin. We have measurements of the heights $y_i$ of the line segment above various points $x_i$ on the x-axis.

The model ignores certain aspects of reality , such as the ground not being precisely level and the problem of exactly determining an "origin" where a real ladder intersects the ground. Granting we ignore those aspects, we still need to model something stochastic about the situation or else we don't have a problem in statistics.

One model is that the measured distances $x_i$ have no error and the measured distances $y_i$ are $y_i = Y_i + e_i$ where $Y_i$ is the actual height of line segment and $e_i$ is a random error. The $e_i$ are mutually independent random variables with mean zero and the same distribution. Is that the model you have in mind?