Determining parameters of a function from its outputs

[nkinar]

Suppose that I have a function $$f(a,b,c,d) = g$$, where $${a,b,c,d}$$ are four independent variables and $$g$$ is the dependent variable. Now let's say that I evaluate the function four times, each time using different inputs, and the function produces four different outputs:

$$f(a_1,b_1,c_1,d_1) = g_1$$
$$f(a_2,b_2,c_2,d_2) = g_2$$
$$f(a_3,b_3,c_3,d_3) = g_3$$
$$f(a_4,b_4,c_4,d_4) = g_4$$

Using a system of equations (linear or non-linear), is there a way to determine the values of the inputs $$a_1,b_1,c_1,d_1,a_2,b_2,c_2,d_2,a_3,b_3,c_3,d_3,a_4,b_4,c_4,d_4$$ if I know the function $$f(a,b,c,d)$$ and the values of the outputs $$g_1,g_2,g_3,g_4$$ for these inputs?

What type of algorithm could I use to determine the values of the inputs? Is there a good reference available?

 

[CompuChip]

There is probably some nice theorem in mathematics (multivariable inverse function theorem, or something like that) which tells you if this is or is not possible.

But I suspect you are looking for something a little more practical, am I right?
In that case, I would say it heavily depends on the function (I doubt that it is possible - in a practical calculational sense - for a general function, but perhaps for simple linear functions you can do it.

So can you give us a bit more info... telling us what f is exactly would help a lot, for instance.

 

[nkinar]

Sure, CompuChip; thank you very much for your response.

Here's an example function:

$$T(a,b,c,d)= \frac{a}{4 \pi c} E_i \left( \frac{b^2}{ 4 \frac{c}{de}t } \right)$$

In the above expression, $$a,b,c,d$$ are the independent variables, and $$e,t$$ are known constants. The $$E_i$$ function is the exponential integral. I would consider this to be a non-linear equation.

Perhaps Levenberg-Marquardt optimization could be used, or some sort of non-linear curve fitting?

 

[CompuChip]

Hmm, I was hoping for ... well, f(a, b, c, d) = a b + c - 2d, or something like that :P
I didn't expect an exponential integral, that's definitely non-linear :)
Is the c in the prefactor also one of the variables, or is "4 pi c" some physical constant?

Actually I think there is little chance of you being able to do anything useful here, except perhaps feed it to a program like Mathematica, asking it to numerically solve for a, b, c, d, and keep your fingers crossed (or do it "by hand" with your favorite optimisation technique, as you proposed).

 

[nkinar]

Hmm, I was hoping for ... well, f(a, b, c, d) = a b + c - 2d, or something like that :P

Yeah, I was hoping for something like that as well! ;-) The variable c in the prefactor is also one of the variables.

I agree that using an optimization technique seems to be the way to deal with this expression. I'll be able to explore this idea further in the following weeks as I start to work more with the mathematics.

Thank you very much for your response!

 

[LCKurtz]

What about two equations in two unknowns:

f(x,y) = sin(x)sin(y)

I give you outputs f(a,b) = 0, f(c,d) = 0. No hope I would say.

 

[nkinar]

Ah, yes - that is a good example of a function where it doesn't work; thanks for posting this, LCKurtz. But I think that if the four outputs $$g_1,g_2,g_3,g_4$$ are different and unique, then perhaps optimization could be used. Perhaps uniqueness of the outputs implies that the inputs can be found.

 

[deluks917]

This question is equivalent to finding the roots of function of four variables as you are essentiually settinbg f(a,b,c,d) - g = 0 and solving for (a, b,c,d). If the g1,g2,g3,g4 are close to each other finding one might help find the others. Do you need only one input for each g or the set of all inputs s.t f (a,b,c,d) = g. Is there any interval you want the solutions to be in?

You talk about uniqueness of the outputs. Is it given that no matter what four different inputs you use that f will always give four different outputs? Then f has an inverse/your solutions will be unique. In addition if this is true many versions of Newton's method should work.

 

[nkinar]

Thanks, deluks917. I still need to explore the mathematics of this problem further, but I do agree that a root-finding algorithm such as Newton's method may be useful.

 

[hotvette]

I believe the problem as originally stated can be viewed as an underdetermined system of equations (i.e. there are more unknowns than equations). If there is a solution there are likely infinitely many of them. One option is to find the minimum norm solution (i.e. optimization approach as has already been mentioned). The math is straightforward:

http://www.stanford.edu/class/ee263/lectures/min-norm.pdf

The fact that the function is non-linear in an of itself doesn't necessarily pose a problem, but the exponential integral might. What is the independent variable of the exponential integral?

 

[nkinar]

hotvette, thank you very much for your response.

Looking back at the original equation, I would like to determine $${a,b,c,d}$$ from a dataset.

$$T(a,b,c,d)= \frac{a}{4 \pi c} E_i \left( \frac{b^2}{ 4 \frac{c}{de}t } \right)$$

The idea that I've been toiling with for quite a long time is how to determine the parameters $${a,b,c,d}$$ given a number of observations. Suppose that I am to re-write the equation as a physical problem.

$$\Delta T(q,k,r,\alpha)= \frac{q}{4 \pi k} E_i \left( \frac{-r^2}{ 4 \alpha t } \right)$$

In the equation above, $$\Delta T(q,k,r,\alpha)$$ is the change in temperature at a given radius $$r$$ from a heater needle that has been inserted into a medium. The time $$t$$ is the time since $$t=0$$ that the heater needle has been warming up. The heat liberated is $$q$$, the thermal conductivity of the medium is $$k$$, and $$\alpha$$ is the thermal diffusivity.

Now suppose that I collect a series of measurements, $${\Delta T_1, \Delta T_2, \Delta T_3, \Delta T_4}$$ taken at different radii $${r_1,r_2,r_3,r_4}$$ from the heater needle. I can also measure the change in temperature at different times $${t_1,...,t_n}$$, where $$n$$ is the total number of time points.

With this information, can I determine the $$q,k,r,\alpha$$ parameters? If not, what information would I need?

 

[nkinar]

Similar problems for heat conduction are described in James V. Beck and Kenneth J. Arnold's 1977 book, "Parameter Estimation in Engineering and Science."

On pg. 434, the authors write, "Without prior information the minimum number of measurements n needed to estimate p parameters is n = p. In this case X is a square matrix."

On pg. 432, the authors also state, "X represents the sensitivity matrix."

 

[RobertT]

Sure, CompuChip; thank you very much for your response.

Here's an example function:

$$T(a,b,c,d)= \frac{a}{4 \pi c} E_i \left( \frac{b^2}{ 4 \frac{c}{de}t } \right)$$

In the above expression, $$a,b,c,d$$ are the independent variables, and $$e,t$$ are known constants. The $$E_i$$ function is the exponential integral. I would consider this to be a non-linear equation.

Perhaps Levenberg-Marquardt optimization could be used, or some sort of non-linear curve fitting?

For this example, each output has its own unique solution. You can't use a linear system equation in the form of Ax = b to solve this example because you're dealing with the same equation (same coefficient) only with different outputs. Which implies that the inputs (a*(b^2)*d*e)/(c^2) must have different unique values.

 

[nkinar]

Okay, thanks for pointing this out, RobertT. I suppose that I have to try something else, as per post #11 or #12 above.

 

[RobertT]

Okay, thanks for pointing this out, RobertT. I suppose that I have to try something else, as per post #11 or #12 above.

To solve for a, b, c, d and e you need 4 more different equations than that function T that describe their relationships. Only then can you use Ax = b to find each unknown!

 

[nkinar]

Ah, okay; what about the following. Suppose that I change the radius $$r_1,r_2,r_3,r_4$$, and take samples at four separate distances. Would this qualify as "different enough" to have a system of four non-linear equations?

$$\Delta T(q,k,r,\alpha)= \frac{q}{4 \pi k} E_i \left( \frac{-r_1^2}{ 4 \alpha t } \right)$$

$$\Delta T(q,k,r,\alpha)= \frac{q}{4 \pi k} E_i \left( \frac{-r_2^2}{ 4 \alpha t } \right)$$

$$\Delta T(q,k,r,\alpha)= \frac{q}{4 \pi k} E_i \left( \frac{-r_3^2}{ 4 \alpha t } \right)$$

$$\Delta T(q,k,r,\alpha)= \frac{q}{4 \pi k} E_i \left( \frac{-r_4^2}{ 4 \alpha t } \right)$$

 

[RobertT]

Ah, okay; what about the following. Suppose that I change the radius $$r_1,r_2,r_3,r_4$$, and take samples at four separate distances. Would this qualify as "different enough" to have a system of four non-linear equations?

$$\Delta T(q,k,r,\alpha)= \frac{q}{4 \pi k} E_i \left( \frac{-r_1^2}{ 4 \alpha t } \right)$$

$$\Delta T(q,k,r,\alpha)= \frac{q}{4 \pi k} E_i \left( \frac{-r_2^2}{ 4 \alpha t } \right)$$

$$\Delta T(q,k,r,\alpha)= \frac{q}{4 \pi k} E_i \left( \frac{-r_3^2}{ 4 \alpha t } \right)$$

$$\Delta T(q,k,r,\alpha)= \frac{q}{4 \pi k} E_i \left( \frac{-r_4^2}{ 4 \alpha t } \right)$$

No, they are all the same function! In this case, to find the inputs q,k,r,\alpha you need three more functions that are different than $$\Delta T(q,k,r,\alpha)= \frac{q}{4 \pi k} E_i \left( \frac{-r^2}{ 4 \alpha t } \right)$$

 

[nkinar]

Okay, thanks Robert; I think that I understand now. Essentially what is required are four unique functions that describe some sort of physical relationship between the required quantities. The key here is that the functions must be different.

Once again, thank you.

 

[RobertT]

Okay, thanks Robert; I think that I understand now. Essentially what is required are four unique functions that describe some sort of physical relationship between the required quantities. The key here is that the functions must be different.

Once again, thank you.

Yes! :)