Shannon entropy - use to calculate the bit needed to encode a symbol

Click For Summary
SUMMARY

The discussion focuses on calculating the bits required to encode symbols using Shannon entropy principles. It establishes that encoding six symbols requires 18 bits if each symbol is represented by 3 bits. However, it emphasizes that less than 3 bits can be used on average due to the non-utilization of all possible symbols. The conversation also highlights the flexibility in choosing specific binary representations for symbols, such as encoding "W" with a custom 3-bit string.

PREREQUISITES
  • Understanding of Shannon entropy and its application in information theory
  • Basic knowledge of binary encoding and bit representation
  • Familiarity with symbol encoding techniques
  • Ability to create and interpret encoding tables
NEXT STEPS
  • Study Shannon entropy calculations in detail
  • Learn about variable-length encoding techniques such as Huffman coding
  • Explore binary encoding strategies for different symbol sets
  • Research information theory textbooks that cover encoding methods
USEFUL FOR

This discussion is beneficial for students in computer science, data compression engineers, and anyone interested in the principles of encoding and information theory.

Outrageous
Messages
373
Reaction score
0
To encode a symbol in binary form, I need 3 bits ,and I have 6 symbols.
So I need 6*3=18 bits to encode "We are" into binary form. As shown in http://www.shannonentropy.netmark.pl/calculate
My question: 3 bits to encode one then I have to use 16 bits, _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _.
How to encode"W" into _ _ _ ? The _ _ _ is filled by 1 or 0 .
3 bits is calculated from entropy And what is that relate to the entropy? Please help. Really appreciate.
 
Physics news on Phys.org
You need less than 3 bits on average, as you do not use all 8 symbols you could encode with 3 bits. For example, you can choose one symbol and encode it with just two bits (this "blocks" two 3-bit-strings).

How to encode"W" into _ _ _ ?
That is completely your choice. Pick anything you like, just provide a table where you describe the encoded symbols.

I moved your thread to our homework section.
 
mfb said:
You need less than 3 bits on average, as you do not use all 8 symbols you could encode with 3 bits. For example, you can choose one symbol and encode it with just two bits (this "blocks" two 3-bit-strings).

That is completely your choice. Pick anything you like, just provide a table where you describe the encoded symbols.

I moved your thread to our homework section.
Thanks. Can you please give me some simple links to read? Please
 
I don't have links, but every textbook and a lot of websites should cover that.
 

Similar threads

  • · Replies 5 ·
Replies
5
Views
2K
  • · Replies 8 ·
Replies
8
Views
3K
  • · Replies 9 ·
Replies
9
Views
4K
  • · Replies 3 ·
Replies
3
Views
3K
  • · Replies 2 ·
Replies
2
Views
2K
  • · Replies 8 ·
Replies
8
Views
3K
  • · Replies 3 ·
Replies
3
Views
2K
Replies
1
Views
3K
  • · Replies 9 ·
Replies
9
Views
3K
  • · Replies 6 ·
Replies
6
Views
2K