Shannon entropy - use to calculate the bit needed to encode a symbol

[Outrageous]

To encode a symbol in binary form, I need 3 bits ,and I have 6 symbols.
So I need 6*3=18 bits to encode "We are" into binary form. As shown in http://www.shannonentropy.netmark.pl/calculate
My question: 3 bits to encode one then I have to use 16 bits, _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _.
How to encode"W" into _ _ _ ? The _ _ _ is filled by 1 or 0 .
3 bits is calculated from entropy And what is that relate to the entropy? Please help. Really appreciate.

 

[mfb]

You need less than 3 bits on average, as you do not use all 8 symbols you could encode with 3 bits. For example, you can choose one symbol and encode it with just two bits (this "blocks" two 3-bit-strings).

How to encode"W" into _ _ _ ?

That is completely your choice. Pick anything you like, just provide a table where you describe the encoded symbols.

I moved your thread to our homework section.

 

[Outrageous]

You need less than 3 bits on average, as you do not use all 8 symbols you could encode with 3 bits. For example, you can choose one symbol and encode it with just two bits (this "blocks" two 3-bit-strings).

That is completely your choice. Pick anything you like, just provide a table where you describe the encoded symbols.

I moved your thread to our homework section.

Thanks. Can you please give me some simple links to read? Please

 

[mfb]

I don't have links, but every textbook and a lot of websites should cover that.

 

[rezeew]

Shannon entropy is a measure of the uncertainty or randomness in a system. In this case, it is being used to calculate the minimum number of bits needed to represent a symbol in binary form. The formula for Shannon entropy is H = -∑P(x)log2P(x), where P(x) is the probability of a particular symbol occurring.

In the given example, there are 6 symbols and each symbol requires 3 bits to be represented in binary form. This means that each symbol has a probability of 1/6 (since there are 6 symbols in total). Plugging this into the Shannon entropy formula, we get H = -(1/6)*log2(1/6)*6 = 3 bits.

So, for each symbol, we need 3 bits to encode it in binary form. For the word "We are", which has 6 symbols, we would need 6*3 = 18 bits to encode it completely.

Now, for your question about encoding "W" in 3 bits, it is not possible to do so using Shannon entropy. This is because the formula assumes that all symbols have equal probability, which is not the case for the word "W". In this case, the probability of "W" occurring is 1/2, while the probabilities of the other symbols are 1/6. So, we would need more than 3 bits to encode "W" in binary form.

Overall, Shannon entropy is a useful tool for determining the minimum number of bits needed to represent symbols in a system. However, it is important to note that it is based on certain assumptions and may not always accurately reflect the actual number of bits needed for specific symbols.