How do you solve the system of equations [tex]e^{x_{1}+y_{1}}+e^{x_{1}+y_{2}}=a_{1}[/tex] [tex]e^{x_{2}+y_{1}}+e^{x_{2}+y_{2}}=a_{2}[/tex] [tex]e^{x_{1}+y_{1}}+e^{x_{2}+y_{1}}=b_{1}[/tex] [tex]e^{x_{1}+y_{2}}+e^{x_{2}+y_{2}}=b_{2}[/tex] x1, x2, y1, y2 are the variables for which I want to solve the equations, a1, a2, b1, b2 are known. Context: I need to solve this in order to get the unknown maximum entropy joint probabilities [tex]p_{ij}=e^{-1-x_{i}-y_{j}}[/tex] [tex]\mbox{for the known marginal probabilities (}a_{i}\mbox{ and }b_{j}\mbox{).}[/tex] i know there is way to do this in information theory, but I need to solve it algebraically.
Hey stlukits. This is basically a multi-equation root finding problem which involves multi-variable calculus and optimization. Take a look at this for more information: http://homes.soic.indiana.edu/classes/spring2012/csci/b553-hauserk/newtons_method.pdf
I don't need a numeric solution or an approximation. I want to solve the equation for x1, x2, y1, y2, if possible. If not, I'll have to live with it.
There's a problem here. First subsitute [itex] u_1 = e^{x1} [/itex] [itex] v_1 = e^{y1} [/itex] etc. Than the four equations become: [tex] u_1 v_1 + u_1 v_2 = a_1 [/tex] [tex] u_2 v_1 + u_2 v_2 = a_2 [/tex] [tex] u_1 v_1 + u_2 v_1 = b_1 [/tex] [tex] u_1 v_2 + u_2 v_2 = b_2 [/tex] No if you add the first two you get: [tex] u_1 v_1 + u_1 v_2 + u_2 v_1 + u_2 v_2 = a_1 + a_2 [/tex] and if you add the last two you get [tex] u_1 v_1 + u_1 v_2 + u_2 v_1 + u_2 v_2 = b_1 + b_2 [/tex] These can't both be true unless [itex] a_1 + a_2 = b_1 + b_2 [/itex] And if that is the case, you have only 3 equations left for 4 unknowns, so there won't be an unique solution.
Thank you! Quick reply here: yes, a1+a2=b1+b2 because they are marginal probabilities and sum to 1. Also, [tex]u_{1}v_{1}+u_{1}v_{2}+u_{2}v_{1}+u_{2}v_{2}=1[/tex] because these are the joint probabilities. Sorry! I should have mentioned that. I will be back in half an hour to report if this gives me enough information to solve the system.
If you write the equations as: [tex] u_1 (v_1 + v_2) = a_1 [/tex] [tex] v_1 (u_1 + u_2) = b_1 [/tex] [tex] (v_1 + v_2)(u_1 + u_2) = 1 [/tex] it's easy to see if [itex] (u_1,u_2,v_1,v_2) [/itex] is a solution, so is [itex] (c u_1, c u_2, \frac {v_1}{c}, \frac {v_2}{c} ) [/itex] to get a solution you can set u1 + u2 = 1 so v1 this gets you [tex] u_1 = c a_1 [/tex] [tex] u_2 = c a_2 [/tex] [tex] v_1 = \frac {b_1}{c} [/tex] [tex] v_2 = \frac {b_2}{c} [/tex] as the complete solution.
Got it. Thank you, willem2. There was information hiding here that I didn't take into account (basically, that it is sufficient to know p_ij for i=1,...,m-1 and j=1,...,n-1 in order to know the m x n dimensional joint probability matrix). Be that as it may, problem solved!