1. Limited time only! Sign up for a free 30min personal tutor trial with Chegg Tutors
    Dismiss Notice
Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Homework Help: Need Help! calculating Entropy & probabilities of a simple Markov Process

  1. Aug 28, 2010 #1
    1. The problem statement, all variables and given/known data

    I am in a hurry with the following problem:

    We have a source that produces binary symbols, 0 and 1.
    0 follows a 0 at probability 7/8
    1 follows a 1 at probability 1/2

    A) Calculate probability of the symbols 0 and 1 to appear.
    B) Calculate entropy of source.

    3. The attempt at a solution

    A) We can say that:

    0 follows a 0 at probability p(0,0) = 7/8 => 1 follows a 0 at probability p(0,1) = 1/8
    1 follows a 1 at probability p(1,1) = 1/2 => 0 follows a 1 at probability p(1,0) = 1/2

    p(0) = p(0) p(0,0) + p(1) p(1,0) = p(0) 7/8 + p(1) 1/2
    p(1) = p(0) p(0,1) + p(1) p(1,1) = p(1) 1/2 + p(0) 1/8

    Now, we have to get p(0) and p(1) but how ? I only come up with p(0) = 4 p(1)
    (according to "a solution", p(0) should be 4/5 and p(1) 1/5)

    B) I think it can be calculated in 2 ways:

    Way 1:
    H(S) = p(0) H0 + p(1) H1
    (sum of the probabilities of getting a 0 and 1 multiplied by its respective variabilities, where
    H0 = -7/8log(7/8) - 1/8log(1/8)
    H1 = -1/2log(1/2) - 1/2log(1/2)

    Way 2:
    H(S) = -p(0)log(p(0)) -p(1)log(p(1))

    Which way is the correct one?

    Thanks in advance!
     
  2. jcsd
  3. Aug 29, 2010 #2
    Anyone? I am a bit in a hurry, need to have this clear for tomorrow...
     
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook