Excellent!
Shannon's entropy model is used in a lot of disparate fields.
Ecology: Species diversity can be defined precisely as you described H, Shannon's entropy. If we limit an example to tree species: Boreal and montane systems have low diversity of tree species, cloud forest systems have very high diversity of tree species - such that you often have less than one individual of a given species per hectare. Abusing your nice model, it appears cloud forest species diversity would equate to a sample of glyphs from a language with thousands of "letters". If you mix in all of the species: trees, shrubs, epiphytes, invertebrates, vertebrates, Eukaryotes on down to single-celled Prokaryotes, etc., you easily have a language with an absurd number of "letters". No wonder we don't fully understand tropical systems.
Your explanation is far better than the one I used for years to help students to understand Shannon's entropy. Darn it all...