1. Limited time only! Sign up for a free 30min personal tutor trial with Chegg Tutors
    Dismiss Notice
Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

4 equations, 4 variables

  1. Feb 9, 2014 #1
    How do you solve the system of equations


    x1, x2, y1, y2 are the variables for which I want to solve the equations, a1, a2, b1, b2 are known.

    Context: I need to solve this in order to get the unknown maximum entropy joint probabilities


    [tex]\mbox{for the known marginal probabilities (}a_{i}\mbox{ and }b_{j}\mbox{).}[/tex]

    i know there is way to do this in information theory, but I need to solve it algebraically.
  2. jcsd
  3. Feb 9, 2014 #2


    User Avatar
    Science Advisor

  4. Feb 9, 2014 #3
    I don't need a numeric solution or an approximation. I want to solve the equation for x1, x2, y1, y2, if possible. If not, I'll have to live with it.
  5. Feb 9, 2014 #4
    There's a problem here. First subsitute [itex] u_1 = e^{x1} [/itex] [itex] v_1 = e^{y1} [/itex] etc.

    Than the four equations become:

    [tex] u_1 v_1 + u_1 v_2 = a_1 [/tex]
    [tex] u_2 v_1 + u_2 v_2 = a_2 [/tex]
    [tex] u_1 v_1 + u_2 v_1 = b_1 [/tex]
    [tex] u_1 v_2 + u_2 v_2 = b_2 [/tex]

    No if you add the first two you get:

    [tex] u_1 v_1 + u_1 v_2 + u_2 v_1 + u_2 v_2 = a_1 + a_2 [/tex]

    and if you add the last two you get

    [tex] u_1 v_1 + u_1 v_2 + u_2 v_1 + u_2 v_2 = b_1 + b_2 [/tex]

    These can't both be true unless [itex] a_1 + a_2 = b_1 + b_2 [/itex]

    And if that is the case, you have only 3 equations left for 4 unknowns, so there won't be an unique solution.
  6. Feb 9, 2014 #5
    Thank you! Quick reply here: yes, a1+a2=b1+b2 because they are marginal probabilities and sum to 1. Also,


    because these are the joint probabilities. Sorry! I should have mentioned that. I will be back in half an hour to report if this gives me enough information to solve the system.
  7. Feb 9, 2014 #6
    If you write the equations as:

    [tex] u_1 (v_1 + v_2) = a_1 [/tex]
    [tex] v_1 (u_1 + u_2) = b_1 [/tex]
    [tex] (v_1 + v_2)(u_1 + u_2) = 1 [/tex]

    it's easy to see if [itex] (u_1,u_2,v_1,v_2) [/itex] is a solution, so is [itex] (c u_1, c u_2, \frac {v_1}{c}, \frac {v_2}{c} ) [/itex]

    to get a solution you can set u1 + u2 = 1 so v1

    this gets you

    [tex] u_1 = c a_1 [/tex]
    [tex] u_2 = c a_2 [/tex]
    [tex] v_1 = \frac {b_1}{c} [/tex]
    [tex] v_2 = \frac {b_2}{c} [/tex]

    as the complete solution.
  8. Feb 9, 2014 #7
    Got it. Thank you, willem2. There was information hiding here that I didn't take into account (basically, that it is sufficient to know p_ij for i=1,...,m-1 and j=1,...,n-1 in order to know the m x n dimensional joint probability matrix). Be that as it may, problem solved!
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook