A good book to introduce information theory?

Click For Summary

Discussion Overview

The discussion centers around recommendations for introductory books on information theory, aimed at individuals with a background in physics and a desire to learn the subject. Participants share various resources and personal preferences, highlighting different approaches to the topic.

Discussion Character

  • Exploratory, Technical explanation

Main Points Raised

  • One participant seeks a good book to learn the basics of information theory, indicating a willingness to engage with challenging material.
  • Another participant recommends two physics books that utilize an information-theoretical approach, suggesting they may provide relevant insights.
  • A third participant mentions a book by Pierce as an easy and enjoyable introduction to the topic, providing a link for reference.
  • This same participant also points to David MacKay's online notes and textbook as useful resources for learning information theory.
  • Shannon's original work is highlighted by multiple participants as a seminal text in the field, with one noting its continued availability in print and affordability.

Areas of Agreement / Disagreement

Participants generally agree on the value of Shannon's work and the recommendations provided, but there is no consensus on a single best introductory book, as different participants suggest various texts based on their own experiences.

Contextual Notes

Some recommendations depend on the reader's background in mathematics and physics, and the effectiveness of each suggested resource may vary based on individual learning preferences.

marksman95
Messages
16
Reaction score
2
Hi all!

I would like to learn the basics of information theory and want a good book to do so.

My math level is that of a second year undergraduate physics student, but I don't mind if I have to struggle a bit through it.

Thanks!
 
Physics news on Phys.org
I only know physics books on statistical physics using the information-theoretical approach. My favorites are

Amnon Katz. Principles of Statistical Mechanics. W. H. Freeman and Company, San Francisco and London, 1967.
Arthur Hobson. Concepts in Statistical Mechanics. Gordon and Breach Science Publishers, 2 edition, 1987.
 
Shannon's original work, referenced by jason as the original Bell Labs paper, is also available in a paperback that is still in print 50+ years after having been written. It's "the Mathematical Theory of Communication" and is the seminal work in the field. You can get it used on Amazon for less than $20.
 

Similar threads

  • · Replies 6 ·
Replies
6
Views
5K
  • · Replies 5 ·
Replies
5
Views
4K
  • · Replies 10 ·
Replies
10
Views
3K
  • · Replies 8 ·
Replies
8
Views
1K
  • · Replies 15 ·
Replies
15
Views
4K
  • · Replies 9 ·
Replies
9
Views
2K
  • · Replies 4 ·
Replies
4
Views
2K
Replies
7
Views
1K
  • · Replies 9 ·
Replies
9
Views
4K
  • · Replies 2 ·
Replies
2
Views
1K