How do you solve a system of equations with 4 variables and 4 equations?

[noowutah]

How do you solve the system of equations

e^{x_{1}+y_{1}}+e^{x_{1}+y_{2}}=a_{1}
e^{x_{2}+y_{1}}+e^{x_{2}+y_{2}}=a_{2}
e^{x_{1}+y_{1}}+e^{x_{2}+y_{1}}=b_{1}
e^{x_{1}+y_{2}}+e^{x_{2}+y_{2}}=b_{2}

x1, x2, y1, y2 are the variables for which I want to solve the equations, a1, a2, b1, b2 are known.

Context: I need to solve this in order to get the unknown maximum entropy joint probabilities

p_{ij}=e^{-1-x_{i}-y_{j}}

\mbox{for the known marginal probabilities (}a_{i}\mbox{ and }b_{j}\mbox{).}

i know there is way to do this in information theory, but I need to solve it algebraically.

 

[chiro]

Hey stlukits.

This is basically a multi-equation root finding problem which involves multi-variable calculus and optimization.

Take a look at this for more information:

http://homes.soic.indiana.edu/classes/spring2012/csci/b553-hauserk/Newtons_method.pdf

 

[noowutah]

I don't need a numeric solution or an approximation. I want to solve the equation for x1, x2, y1, y2, if possible. If not, I'll have to live with it.

 

[willem2]

There's a problem here. First subsitute u_1 = e^{x1} v_1 = e^{y1} etc.

Than the four equations become:

u_1 v_1 + u_1 v_2 = a_1
u_2 v_1 + u_2 v_2 = a_2
u_1 v_1 + u_2 v_1 = b_1
u_1 v_2 + u_2 v_2 = b_2

No if you add the first two you get:

u_1 v_1 + u_1 v_2 + u_2 v_1 + u_2 v_2 = a_1 + a_2

and if you add the last two you get

u_1 v_1 + u_1 v_2 + u_2 v_1 + u_2 v_2 = b_1 + b_2

These can't both be true unless a_1 + a_2 = b_1 + b_2

And if that is the case, you have only 3 equations left for 4 unknowns, so there won't be an unique solution.

 

[noowutah]

Thank you! Quick reply here: yes, a1+a2=b1+b2 because they are marginal probabilities and sum to 1. Also,

u_{1}v_{1}+u_{1}v_{2}+u_{2}v_{1}+u_{2}v_{2}=1

because these are the joint probabilities. Sorry! I should have mentioned that. I will be back in half an hour to report if this gives me enough information to solve the system.

 

[willem2]

If you write the equations as:

u_1 (v_1 + v_2) = a_1
v_1 (u_1 + u_2) = b_1
(v_1 + v_2)(u_1 + u_2) = 1

it's easy to see if (u_1,u_2,v_1,v_2) is a solution, so is (c u_1, c u_2, \frac {v_1}{c}, \frac {v_2}{c} )

to get a solution you can set u1 + u2 = 1 so v1

this gets you

u_1 = c a_1
u_2 = c a_2
v_1 = \frac {b_1}{c}
v_2 = \frac {b_2}{c}

as the complete solution.

 

[noowutah]

Got it. Thank you, willem2. There was information hiding here that I didn't take into account (basically, that it is sufficient to know p_ij for i=1,...,m-1 and j=1,...,n-1 in order to know the m x n dimensional joint probability matrix). Be that as it may, problem solved!