3 Quick questions about information theory

Click For Summary
SUMMARY

This discussion addresses three key questions related to information theory. For part A, the uniquely decodable codes identified are 1, 3, 4, and 5, while the instantaneous codes are 1, 4, and 5, with code 2 being non-decodable due to prefix issues. In part B, the probabilities for the second-order extension of the source are calculated as P(aa) = p1*p1, P(ab) = p1*q1, and P(bb) = q1*q1. Part C clarifies the definitions of "rate," distinguishing between "Information rate R," which is the average bits per symbol, and "transmission rate," defined as bits per time (seconds).

PREREQUISITES
  • Understanding of uniquely decodable and instantaneous codes in coding theory
  • Knowledge of probability theory, specifically second-order probabilities
  • Familiarity with Shannon's information theory concepts
  • Basic understanding of channel capacity and error-free transmission
NEXT STEPS
  • Study the properties of uniquely decodable and instantaneous codes in coding theory
  • Learn about second-order Markov processes and their applications in probability
  • Explore Shannon's theorem on channel capacity and its implications for data transmission
  • Investigate the differences between information rate and transmission rate in communication systems
USEFUL FOR

This discussion is beneficial for students and professionals in information theory, coding theory, and telecommunications, particularly those studying data encoding and transmission efficiency.

degs2k4
Messages
72
Reaction score
0

Homework Statement



A) Select uniquely decodable codes and instantaneous codes from Code1 to 5 of the image below:
2ut2fxe.png


B) Personal question about second-order extension probabilites. If we have:
probability of a symbol a P(a) = p1
probability of a symbol b P(b) = q1
Which is the probability of symbols aa, ab, bb of the second-order extension of the source?

C) What is "rate"?

The Attempt at a Solution



A)

Uniquely decodable codes: 1,3,4,5 (in code 2 the extension(AE)=00100=extension(BA) )
Instantaneous codes: 1,4,5 (code 2 was not uniquely decodable, and in code 3 A is prefix of other codes)

B)

P(aa) = p1*p1
P(ab) = p1*q1
P(bb) = q1*q1

c)

It depends on what it refers to:
" Information rate R " : avg bits/symbol, R< Channel Capacity implies theoretical error-free transmission (Shannon)

" transmission rate " : bits/time(sec)

Is everything ok?

Thanks in advance.
 
Physics news on Phys.org
Noone ?
 

Similar threads

Graduate On the origin and the evolution of information in the Universe
  • · Replies 1 ·
Replies
1
Views
3K
Graduate Asymmetry between probability distributions
  • · Replies 7 ·
Replies
7
Views
2K
Information Theory - Shannon's Self-Information units
  • · Replies 1 ·
Replies
1
Views
5K
Graduate Algorithmic information theory's QM view of reality
  • · Replies 2 ·
Replies
2
Views
2K
Undergrad Questions about QFT and the reality of subatomic particles
  • · Replies 6 ·
Replies
6
Views
2K
Graduate Need sum help (information theory)
  • · Replies 9 ·
Replies
9
Views
4K
April Fool's Physics Papers 2026
  • · Replies 2 ·
Replies
2
Views
518
Graduate Reconstruction of Quantum Theory by Lucien Hardy
  • · Replies 2 ·
Replies
2
Views
6K
Graduate How Do Various Theories Define Information Through Entropy?
  • · Replies 13 ·
Replies
13
Views
10K
Graduate Information Preservation in Quantum Gravity
  • · Replies 42 ·
2
Replies
42
Views
23K