Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Ax + b Least Squares Minimization Standard Form

  1. Jun 25, 2009 #1
    All -

    Given a set of data {(xi, yi)| i = 1,2,...,m} and the regression equation f(x) = ax + b, I want to use the simplex method to minimize the equation Sigma [(yi - f(xi))/f(xi)]^2. However, I am stuck on how to initially organize the problem. I am not sure whether the equation, Sigma [(yi - f(xi))/f(xi)]^2, needs to be put into some sort of standard form or not. Also, I am having trouble comprehending how to turn the individual [(y - f(x))/f(x)]^2 equations into constraints.

    The end goal for this is to turn it into a program. The simplex method should run sufficiently fast for the type of data I will be feeding the program. However, I first need to understand the logic before any code gets written.

    Let me know if I'm not stating the problem clear enough. Thank you for helping.
  2. jcsd
  3. Jul 2, 2009 #2


    User Avatar
    Homework Helper

    You are trying to minimize

    S(a,b) = \sum_{i=1}^m {\left(\frac{y_i - (a + bx_i)}{a+bx_i}\right)^2}

    is this correct? I'm not sure what this will give you - certainly not a regression result.

    I'm not aware of any way to use the simplex method - designed for linear optimization problems, for a problem that is so far from being linear. (L1 regression problems can be solved as linear programming problems, but that is the closest item I know of.)
  4. Jul 3, 2009 #3
    statdad -

    Correct. That's the right equation. It's the Minimum Squares Percent Error (MSPE) equation used in econometrics. I figured out there are a handful of algorithms/methods to minimize nonlinear equations. Variations of Newton's Method are fairly popular for this task. Thank you for your input!
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook