Minimize parameter for Least Absolute Deviation LAD

Click For Summary

Discussion Overview

The discussion centers on the computation of the parameter \(\beta\) for the Least Absolute Deviation (LAD) method in regression analysis. Participants explore the absence of a closed-form solution for this optimization problem and the implications for practical computation.

Discussion Character

  • Technical explanation, Debate/contested

Main Points Raised

  • One participant inquires about the computation of \(\beta\) as the argument that minimizes the sum of absolute deviations.
  • Another participant states that there is no closed-form solution for the LAD method, contrasting it with least squares, and suggests consulting software documentation for various computational methods.
  • A participant expresses interest in understanding the calculation rather than applying it to real data, noting that the estimate is the median of the data.
  • A later reply challenges the assertion that estimates are the medians of the \(x\) values, clarifying that while minimizing the sum of absolute deviations from a constant yields the sample median, this does not extend to the regression context.

Areas of Agreement / Disagreement

Participants do not reach consensus on the relationship between the LAD estimates and the sample median in the context of regression, indicating a disagreement on this point.

Contextual Notes

The discussion highlights the limitations of applying median calculations directly to regression estimates and the lack of a closed-form solution for the LAD method.

dabd
Messages
25
Reaction score
0
How to compute \beta = arg min_\beta \sum_{i=1}^N {|y_i - x_i^T \beta|
 
Physics news on Phys.org
There is no closed-form solution for this (contrary to the situation with least squares). The software you use (R, SAS, etc) use a variety of methods. check the relevant documentation for those programs.
 
statdad said:
There is no closed-form solution for this (contrary to the situation with least squares). The software you use (R, SAS, etc) use a variety of methods. check the relevant documentation for those programs.

I was just interested in the calculation not in applying it to real data.
The estimate is the median of the data x1,...,xn and I wanted to see how they derived that result.
 
No, the estimates are not the medians of the x values - if that were the case we would have a "closed form" method of calculation.

Where you may be confused is this: if you want to minimize

<br /> \sum_{i=1}^n |x_i - a|<br />

as a function of a, the solution is the sample median. This does not generalize to regression.
 

Similar threads

Undergrad Extract error from simulated data
  • · Replies 13 ·
Replies
13
Views
2K
Undergrad Maximum likelihood to fit a parameter of this model
  • · Replies 1 ·
Replies
1
Views
2K
Graduate Uncertainties on Bayes' theorem
  • · Replies 19 ·
Replies
19
Views
3K
Undergrad Determining the covariance of two parameters given their explicit relationship
  • · Replies 13 ·
Replies
13
Views
2K
MHB Likelihood Ratio Test for Common Variance from Two Normal Distribution Samples
  • · Replies 1 ·
Replies
1
Views
4K
MHB Binomial Distribution: Likelihood Ratio Test for Equality of Several Proportions
  • · Replies 1 ·
Replies
1
Views
3K
Graduate Chernoff Bounds using importance sampling identity
  • · Replies 0 ·
Replies
0
Views
2K
Math terminology in my Taylor Series expansion?
  • · Replies 12 ·
Replies
12
Views
3K
Undergrad How can I convert chi-square into a matrix and graph likelihood contours?
  • · Replies 14 ·
Replies
14
Views
3K
MHB Calculate the density - Unbiased estimator for θ
  • · Replies 1 ·
Replies
1
Views
2K