Optimization to find parameters?

[johnpjust]

Hi guys, I'm a bit rusty on optimization, but i remember enough to be able to setup some problems that I am facing in my research, and would like to know how to go about solving such problems in general. The equations below may have more variables and be in slightly different forms at times, but ultimately we have measured variables (such as pressure and volume, and ground truth of weight) and need conversion parameters (such as a and b). Please be detailed on a few different methods if possible and the reasons for them. Also, if there is a way that I can see the "significance" of the parameters (statistically) and/or the affect they have on reducing error, that would be stellar! Thanks!

minimize:
$^{m}_{j=1} \sum(^{n}_{i=1} \sum((a + b*[pressure]_{ij})*[volume]_{ij}) - [weight]_{j})^{2})$

and

minimize
$^{m}_{j=1} \sum(^{n}_{i=1} \sum(a*[pressure]_{ij} + b*[volume]_{ij}) - [weight]_{j})^{2})$

 

[Office_Shredder]

There are only two parameters a and b which are multiplying all the pressures and volumes, right?

To minimize a function you can just take the partial derivatives with respect to each variable, and set them equal to zero. To use your first example, the partial derivative with respect to a is
$$\sum_{j} \sum_{i} 2((a+b*[pressure]_{ij})*[volume]_{ij})-[weight]_{j})*[volume]_{ij}$$
and the derivative with respect to b is
$$\sum_{j} \sum_{i} 2((a+b*[pressure]_{ij})*[volume]_{ij})-[weight]_{j})*[volume]_{ij}*[pressure]_{ij}$$
This looks bad but some fairly easy algebra can re-write these both as a*(something) + b*(something) + something. Setting them equal to zero gives two equations in two unknowns that you need to solve (and should be able to solve).

I don't understand the last part, is your error the expression you are trying to minimize?

 

[johnpjust]

Hi, yes, I am minimizing the error. But the problem is that I will have many, many equations with only two variables. I am almost certain that this will require some kind of iterative solution. I don't think it has a closed form solution, does it?

edit - ok i understand your last question. This is to be used for estimating weight using measured variables of pressure and volume...but don't get hung up on those. Just think of them as some kind of random variables (statistically) that correlate with the weight. What I'm doing has a strong statistical component so I am trying to minimize the error in my prediction of many weights (think something like regression for an analogy)

 

[Office_Shredder]

Hi, yes, I am minimizing the error. But the problem is that I will have many, many equations with only two variables. I am almost certain that this will require some kind of iterative solution. I don't think it has a closed form solution, does it?

Do you mean you have many error functions that you want to minimize? Given an error function such as you posted, it has a closed form solution which is not too difficult to obtain. No harder than finding the closed form solution for a and b in terms of x,y,z,m,n and p in the following:
ax+by+z = 0
am+bn+p = 0

edit - ok i understand your last question. This is to be used for estimating weight using measured variables of pressure and volume...but don't get hung up on those. Just think of them as some kind of random variables (statistically) that correlate with the weight. What I'm doing has a strong statistical component so I am trying to minimize the error in my prediction of many weights (think something like regression for an analogy)

OK, so your overall question is how statistically significant the values of a and b are once you solve for them?

 

[johnpjust]

Edit: I think you are right after thinking more about this...since I do have a linear system here...

 

[FactChecker]

Both minimization problems are standard multiple linear regression problems. You are trying to minimize the total some squared errors of two models:
weight = a*volume + b*pressure*volume
weight = a*volume +b*volume

For each problem, a statistics package can estimate the values of a and b that will minimize the sum-squared errors. It will also give information of how statistically strong the relationship is and how much is gained from including the a-term and the b-term.