3 Quick questions about information theory

1 reply · 3K views · last reply
degs2k4
Messages
72
Reaction score
0

Homework Statement



A) Select uniquely decodable codes and instantaneous codes from Code1 to 5 of the image below:
Table of five prefix-style binary code sets, Code1 to Code5, for symbols A to H, with a note that each code word is transmitted and received from its leftmost bit.


B) Personal question about second-order extension probabilites. If we have:
probability of a symbol a P(a) = p1
probability of a symbol b P(b) = q1
Which is the probability of symbols aa, ab, bb of the second-order extension of the source?

C) What is "rate"?

The Attempt at a Solution



A)

Uniquely decodable codes: 1,3,4,5 (in code 2 the extension(AE)=00100=extension(BA) )
Instantaneous codes: 1,4,5 (code 2 was not uniquely decodable, and in code 3 A is prefix of other codes)

B)

P(aa) = p1*p1
P(ab) = p1*q1
P(bb) = q1*q1

c)

It depends on what it refers to:
" Information rate R " : avg bits/symbol, R< Channel Capacity implies theoretical error-free transmission (Shannon)

" transmission rate " : bits/time(sec)

Is everything ok?

Thanks in advance.