(adsbygoogle = window.adsbygoogle || []).push({}); 1. The problem statement, all variables and given/known data

I am in a hurry with the following problem:

We have a source that produces binary symbols, 0 and 1.

0 follows a 0 at probability 7/8

1 follows a 1 at probability 1/2

A) Calculate probability of the symbols 0 and 1 to appear.

B) Calculate entropy of source.

3. The attempt at a solution

A) We can say that:

0 follows a 0 at probability p(0,0) = 7/8 => 1 follows a 0 at probability p(0,1) = 1/8

1 follows a 1 at probability p(1,1) = 1/2 => 0 follows a 1 at probability p(1,0) = 1/2

p(0) = p(0) p(0,0) + p(1) p(1,0) = p(0) 7/8 + p(1) 1/2

p(1) = p(0) p(0,1) + p(1) p(1,1) = p(1) 1/2 + p(0) 1/8

Now, we have to get p(0) and p(1) but how ? I only come up with p(0) = 4 p(1)

(according to "a solution", p(0) should be 4/5 and p(1) 1/5)

B) I think it can be calculated in 2 ways:

Way 1:

H(S) = p(0) H0 + p(1) H1

(sum of the probabilities of getting a 0 and 1 multiplied by its respective variabilities, where

H0 = -7/8log(7/8) - 1/8log(1/8)

H1 = -1/2log(1/2) - 1/2log(1/2)

Way 2:

H(S) = -p(0)log(p(0)) -p(1)log(p(1))

Which way is the correct one?

Thanks in advance!

**Physics Forums | Science Articles, Homework Help, Discussion**

Join Physics Forums Today!

The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

# Homework Help: Need Help! calculating Entropy & probabilities of a simple Markov Process

**Physics Forums | Science Articles, Homework Help, Discussion**