Need Help calculating Entropy & probabilities of a simple Markov Process

AI Thread Summary
The discussion revolves around calculating the probabilities and entropy of a binary Markov process where the transition probabilities are given. The user is trying to determine the probabilities of symbols 0 and 1, noting that they have derived a relationship where p(0) equals 4 times p(1). They seek clarification on how to find the exact values of p(0) and p(1), which are suggested to be 4/5 and 1/5, respectively. Additionally, the user is exploring two methods for calculating entropy, asking which is the correct approach. The urgency of the request highlights the need for a clear and prompt resolution to these calculations.
degs2k4
Messages
72
Reaction score
0

Homework Statement



I am in a hurry with the following problem:

We have a source that produces binary symbols, 0 and 1.
0 follows a 0 at probability 7/8
1 follows a 1 at probability 1/2

A) Calculate probability of the symbols 0 and 1 to appear.
B) Calculate entropy of source.

The Attempt at a Solution



A) We can say that:

0 follows a 0 at probability p(0,0) = 7/8 => 1 follows a 0 at probability p(0,1) = 1/8
1 follows a 1 at probability p(1,1) = 1/2 => 0 follows a 1 at probability p(1,0) = 1/2

p(0) = p(0) p(0,0) + p(1) p(1,0) = p(0) 7/8 + p(1) 1/2
p(1) = p(0) p(0,1) + p(1) p(1,1) = p(1) 1/2 + p(0) 1/8

Now, we have to get p(0) and p(1) but how ? I only come up with p(0) = 4 p(1)
(according to "a solution", p(0) should be 4/5 and p(1) 1/5)

B) I think it can be calculated in 2 ways:

Way 1:
H(S) = p(0) H0 + p(1) H1
(sum of the probabilities of getting a 0 and 1 multiplied by its respective variabilities, where
H0 = -7/8log(7/8) - 1/8log(1/8)
H1 = -1/2log(1/2) - 1/2log(1/2)

Way 2:
H(S) = -p(0)log(p(0)) -p(1)log(p(1))

Which way is the correct one?

Thanks in advance!
 
Physics news on Phys.org
Anyone? I am a bit in a hurry, need to have this clear for tomorrow...
 
Thread 'Collision of a bullet on a rod-string system: query'
In this question, I have a question. I am NOT trying to solve it, but it is just a conceptual question. Consider the point on the rod, which connects the string and the rod. My question: just before and after the collision, is ANGULAR momentum CONSERVED about this point? Lets call the point which connects the string and rod as P. Why am I asking this? : it is clear from the scenario that the point of concern, which connects the string and the rod, moves in a circular path due to the string...
Thread 'A cylinder connected to a hanged mass'
Let's declare that for the cylinder, mass = M = 10 kg Radius = R = 4 m For the wall and the floor, Friction coeff = ##\mu## = 0.5 For the hanging mass, mass = m = 11 kg First, we divide the force according to their respective plane (x and y thing, correct me if I'm wrong) and according to which, cylinder or the hanging mass, they're working on. Force on the hanging mass $$mg - T = ma$$ Force(Cylinder) on y $$N_f + f_w - Mg = 0$$ Force(Cylinder) on x $$T + f_f - N_w = Ma$$ There's also...
Back
Top