# Shannon entropy of logic gate output.

Tags:
1. Jan 1, 2016

### Dazed&Confused

1. The problem statement, all variables and given/known data
A particular logic gate takes two binary inputs $A$ and $B$ and has two binary outputs $A'$ and $B'$. I won't reproduce the truth table. Suffice to say every combination of $A$ and $B$ is given. The output is produced by $A' = \text{NOT} \ A$ and $B' = \text{NOT} \ B$. The input has Shannon entropy of 2 bits. Show that the output has a Shannon entropy of 2 bits.

A second logic has output produced by $A' = A \ \text{OR} \ B$ and $B' = A \ \text{AND} \ B$. Show that the output now has an entropy of $\frac32$ bits.

2. Relevant equations
$$S = - \sum_{i} k P_i \log P_i$$

3. The attempt at a solution
From what I (don't) understand, $P = \frac12$ in the first example for $A, B, A',B'$ so the total number of bits is the same for both input and output. For the second example, I would say $P_{A'} = \frac34$ and $P_{B'} = \frac14$, but that does not produce the correct number of bits.

2. Jan 1, 2016

### Fightfish

It helps greatly if you draw up a truth table to represent the inputs and outcomes. The $P_{i}$'s that appear in the formula for Shannon entropy represent the probabilities of the various outcomes, so expressions like $P_{A'}$ make no sense at all.
As a quick example to illustrate the use of the Shannon entropy formula, consider the flipping of a fair coin. There are two possible outcomes - heads or tails, each which occur with 50% probability. The Shannon entropy is then $S = - \frac{1}{2} \log_{2} \frac{1}{2} - \frac{1}{2} \log_{2} \frac{1}{2} = 1$.

3. Jan 1, 2016

### Dazed&Confused

Thanks. There is a truth table, so do you mean I should add one here? By [/itex] P_{A'} [/itex] I meant the probability of $A'$ being 1 but I agree this is confusing. I'm still not sure about the answer though. It works out fine for the first example as it is $S = -4 \times \frac12 \log_2 \frac12 = 2$ but for the second example I think it is $$S = - \frac32 \log_2 \frac34 - \frac12 \log_2 \frac14.$$

4. Jan 1, 2016

### Fightfish

Take note that there are three outcomes for the second example. Unlike the first example, the 'events' A' and B' are not independent.
Your result doesn't agree with mine, so perhaps you might want to list down the truth table for this example?

5. Jan 1, 2016

### Dazed&Confused

Oh I see where I went wrong. I'm confusing what the 'events' are. I have the correct answer now.

Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook

Have something to add?
Draft saved Draft deleted