A good book to introduce information theory?

[marksman95]

Hi all!

I would like to learn the basics of information theory and want a good book to do so.

My math level is that of a second year undergraduate physics student, but I don't mind if I have to struggle a bit through it.

Thanks!

 

[vanhees71]

I only know physics books on statistical physics using the information-theoretical approach. My favorites are

Amnon Katz. Principles of Statistical Mechanics. W. H. Freeman and Company, San Francisco and London, 1967.
Arthur Hobson. Concepts in Statistical Mechanics. Gordon and Breach Science Publishers, 2 edition, 1987.

 

[jasonRF]

I am not an expert on this subject. But an easy and fun read on this topic is by Pierce:
https://www.amazon.com/dp/0486240614/?tag=pfamazon01-20
EDIT: I highly recommend Pierce.

David MacKay has some notes and a related textbook online:
http://www.inference.phy.cam.ac.uk/mackay/info-theory/course.html
http://www.inference.phy.cam.ac.uk/mackay/itila/

Shannon's original work may also be useful:
http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6773024

jason

 

[phinds]

Shannon's original work, referenced by jason as the original Bell Labs paper, is also available in a paperback that is still in print 50+ years after having been written. It's "the Mathematical Theory of Communication" and is the seminal work in the field. You can get it used on Amazon for less than $20.