How do you solve a system of equations with 4 variables and 4 equations?

[noowutah]

How do you solve the system of equations

$$e^{x_{1}+y_{1}}+e^{x_{1}+y_{2}}=a_{1}$$
$$e^{x_{2}+y_{1}}+e^{x_{2}+y_{2}}=a_{2}$$
$$e^{x_{1}+y_{1}}+e^{x_{2}+y_{1}}=b_{1}$$
$$e^{x_{1}+y_{2}}+e^{x_{2}+y_{2}}=b_{2}$$

x1, x2, y1, y2 are the variables for which I want to solve the equations, a1, a2, b1, b2 are known.

Context: I need to solve this in order to get the unknown maximum entropy joint probabilities

$$p_{ij}=e^{-1-x_{i}-y_{j}}$$

$$\mbox{for the known marginal probabilities (}a_{i}\mbox{ and }b_{j}\mbox{).}$$

i know there is way to do this in information theory, but I need to solve it algebraically.

 

[chiro]

Hey stlukits.

This is basically a multi-equation root finding problem which involves multi-variable calculus and optimization.

Take a look at this for more information:

http://homes.soic.indiana.edu/classes/spring2012/csci/b553-hauserk/Newtons_method.pdf

 

[noowutah]

I don't need a numeric solution or an approximation. I want to solve the equation for x1, x2, y1, y2, if possible. If not, I'll have to live with it.

 

[willem2]

There's a problem here. First subsitute $u_1 = e^{x1}$ $v_1 = e^{y1}$ etc.

Than the four equations become:

$$u_1 v_1 + u_1 v_2 = a_1$$
$$u_2 v_1 + u_2 v_2 = a_2$$
$$u_1 v_1 + u_2 v_1 = b_1$$
$$u_1 v_2 + u_2 v_2 = b_2$$

No if you add the first two you get:

$$u_1 v_1 + u_1 v_2 + u_2 v_1 + u_2 v_2 = a_1 + a_2$$

and if you add the last two you get

$$u_1 v_1 + u_1 v_2 + u_2 v_1 + u_2 v_2 = b_1 + b_2$$

These can't both be true unless $a_1 + a_2 = b_1 + b_2$

And if that is the case, you have only 3 equations left for 4 unknowns, so there won't be an unique solution.

 

[noowutah]

Thank you! Quick reply here: yes, a1+a2=b1+b2 because they are marginal probabilities and sum to 1. Also,

$$u_{1}v_{1}+u_{1}v_{2}+u_{2}v_{1}+u_{2}v_{2}=1$$

because these are the joint probabilities. Sorry! I should have mentioned that. I will be back in half an hour to report if this gives me enough information to solve the system.

 

[willem2]

If you write the equations as:

$$u_1 (v_1 + v_2) = a_1$$
$$v_1 (u_1 + u_2) = b_1$$
$$(v_1 + v_2)(u_1 + u_2) = 1$$

it's easy to see if $(u_1,u_2,v_1,v_2)$ is a solution, so is $(c u_1, c u_2, \frac {v_1}{c}, \frac {v_2}{c} )$

to get a solution you can set u1 + u2 = 1 so v1

this gets you

$$u_1 = c a_1$$
$$u_2 = c a_2$$
$$v_1 = \frac {b_1}{c}$$
$$v_2 = \frac {b_2}{c}$$

as the complete solution.

 

[noowutah]

Got it. Thank you, willem2. There was information hiding here that I didn't take into account (basically, that it is sufficient to know p_ij for i=1,...,m-1 and j=1,...,n-1 in order to know the m x n dimensional joint probability matrix). Be that as it may, problem solved!