An Introduction to Probability Theory and Its Applications by Feller

[micromass]

• Author: William Feller
• Title: An Introduction to Probability Theory and Its Applications
• Amazon Link:
  https://www.amazon.com/dp/0471257087/?tag=pfamazon01-20
  https://www.amazon.com/dp/0471257095/?tag=pfamazon01-20
• Prerequisities:

Table of Contents for Volume I:

[LIST]
[*] Introduction: The Nature of Probability Theory
[LIST]
[*] The Background
[*] Procedure
[*] "Statistical" Probability
[*] Summary
[*] Historical Note
[/LIST]
[*] The Sample Space
[LIST]
[*] The Empirical Background
[*] Examples
[*] The Sample Space. Events
[*] Relations among Events
[*] Discrete Sample Spaces
[*] Probabilities in Discrete Sample Spaces: Preparations
[*] The Basic Definitions and Rules
[*] Problems for Solution
[/LIST]
[*] Elements of Combinatorial Analysis
[LIST]
[*] Preliminaries
[*] Ordered Samples
[*] Examples
[*] Subpopulations and Partitions
[*] Application to Occupancy Problems
[LIST]
[*] Bose-Einstein and Fermi-Dirac Statistics
[*] Application to Runs
[/LIST]
[*] The Hypergeometric Distribution
[*] Examples for Waiting Times
[*] Binomial Coefficients
[*] Stirling's Formula
[*] Problems for Solution:
[LIST]
[*] Exercises and Examples
[*] Problems and Complements of a Theoretical Character
[*] Problems and Identities Involving Binomial Coefficients
[/LIST]
[/LIST]
[*] Fluctuations in Coin Tossing and Random Walks
[LIST]
[*] General Orientation. The Reflection Principle
[*] Random Walks: Basic Notions and Notations
[*] The Main Lemma
[*] Last Visits and Long Leads
[*] Changes of Sign
[*] An Experimental Illustration
[*] Maxima and First Passages
[*] Duality. Position of Maxima
[*] An Equidistribution Theorem
[*] Problems for Solution
[/LIST]
[*] Combination of Events
[LIST]
[*] Union of Events
[*] Application to the Classical Occupancy Problem
[*] The Realization of m among N events
[*] Application to Matching and Guessing
[*] Miscellany
[*] Problems for Solution
[/LIST]
[*] Conditional Probability. Stochastic Independence
[LIST]
[*] Conditional Probability
[*] Probabilities Defined by Conditional Probabilities. Urn Models
[*] Stochastic Independence
[*] Product Spaces. Independent Trials
[*] Applications to Genetics
[*] Sex-Linked Characters
[*] Selection
[*] Problems for Solution
[/LIST]
[*] The Binomial and the Poisson Distributions
[LIST]
[*] Bernoulli Trials
[*] The Binomial Distribution
[*] The Central Term and the Tails
[*] The Law of Large Numbers
[*] The Poisson Approximation
[*] The Poisson Distribution
[*] Observations Fitting the Poisson Distribution
[*] Waiting Times. The Negative Binomial Distribution
[*] The Multinomial Distribution
[*] Problems for Solution
[/LIST]
[*] The Normal Approximation to the Binomial Distribution
[LIST]
[*] The Normal Distribution
[*] Orientation: Symmetric Distributions
[*] The DeMoivre-Laplace Limit Theorem
[*] Examples
[*] Relation to the Poisson Approximation
[*] Large Deviations
[*] Problems for Solution
[/LIST]
[*] Unlimited Sequences of Bernoulli Trials
[LIST]
[*] Infinite Sequences of Trials
[*] Systems of Gambling
[*] The Borel-Cantelli Lemmas
[*] The Strong Law of Large Numbers
[*] The Law of the Iterated Logarithm
[*] Interpretation in Number Theory Language
[*] Problems for Solution
[/LIST]
[*] Random Variables; Expectation
[LIST]
[*] Random Variables
[*] Expectations
[*] Examples and Applications
[*] The Variance
[*] Covariance; Variance of a Sum
[*] Chebyshev's Inequality
[*] Kolmogorov's Inequality
[*] The Correlation Coefficient
[*] Problems for Solution
[/LIST]
[*] Laws of Large Numbers
[LIST]
[*] Identically Distributed Variables
[*] Proof of the Law of Large Numbers
[*] The Theory of "Fair" Games
[*] The Petersburg Game
[*] Variable Distributions
[*] Applications to Combinatorial Analysis
[*] The Strong Law of Large Numbers
[*] Problems for Solution
[/LIST]
[*] Integral Valued Variables. Generating Functions
[LIST]
[*] Generalities
[*] Convolutions
[*] Equalizations and Waiting Times in Bernoulli Trials
[*] Partial Fraction Expansions
[*] Bivariate Generating Functions
[*] The Continuity Theorem
[*] Problems for Solution
[/LIST]
[*] Compound Distributions. Branching Processes
[LIST]
[*] Sums of a Random Number of Variables
[*] The Compound Poisson Distribution
[LIST]
[*] Processes with Independent Increments
[/LIST]
[*] Examples for Branching Processes
[*] Extinction Probabilities in Branching Processes 
[*] The Total Progeny in Branching Processes
[*] Problems for Solution
[/LIST]
[*] Recurrent Events. Renewal Theory
[LIST]
[*] Informal Preparations and Examples
[*] Definitions
[*] The Basic Relations
[*] Examples
[*] Delayed Recurrent Events. A General Limit Theorem
[*] The Number of Occurrences of [itex]\mathcal{E}[/itex]
[*] Application to the Theory of Success Runs
[*] More General Patterns
[*] Lack of Memory of Geometric Waiting Times
[*] Renewal Theory
[*] Proof of the Basic Limit Theorem
[*] Problems for Solution
[/LIST]
[*] Random Walk and Ruin Problems
[LIST]
[*] General Orientation
[*] The Classical Ruin Problem
[*] Expected Duration of the Game
[*] Generating Functions for the Duration of the Game and for the First-Passage Times
[*] Explicit Expressions
[*] Connection with Diffusion Processes
[*] Random Walks in the Plane and Space
[*] The Generalized One-Dimensional Random Walk (Sequential Sampling)
[*] Problems for Solution
[/LIST]
[*] Markov Chains
[LIST]
[*] Definition 
[*] Illustrative Examples
[*] Higher Transition Probabilities
[*] Closures and Closed Sets
[*] Classification of States
[*] Irreducible Chains. Decompositions
[*] Invariant Distributions
[*] Transient Chains
[*] Periodic Chains
[*] Application to Card Shuffling
[*] Invariant Measures. Ratio Limit Theorems
[*] Reversed Chains. Boundaries
[*] The General Markov Process
[*] Problems for Solution
[/LIST]
[*] Algebraic Treatment of Finite Markov Chains
[LIST]
[*] General Theory
[*] Examples
[*] Random Walk with Reflecting Barriers
[*] Transient States; Absorption Probabilities
[*] Application to Recurrence Times
[/LIST]
[*] The Simplest Time-Dependent Stochastic Processes
[LIST]
[*] General Orientation. Markov Processes
[*] The Poisson Process
[*] The Pure Birth Process
[*] Divergent Birth Processes
[*] The Birth and Death Process
[*] Exponential Holding Times
[*] Waiting Line and Servicing Problems
[*] The Backward (Retrospective) Equations
[*] General Processes
[*] Problems for Solution
[/LIST]
[*] Answers to Problems
[*] Index 
[/LIST]

Table of Contents for Volume II:

[LIST]
[*] The Exponential and the Uniform Densities
[LIST]
[*] Introduction
[*] Densities. Convolutions
[*] The Exponential Density
[*] Waiting Time Paradoxes. The Poisson Process
[*] The Persistence of Bad Luck
[*] Waiting Times and Order Statistics
[*] The Uniform Distribution
[*] Random Splittings
[*] Convolutions and Covering Theorems
[*] Random Directions
[*] The Use of Lebesgue Measure
[*] Empirical Distributions
[*] Problems for Solution
[/LIST]
[*] Special Densities. Randomization
[LIST]
[*] Notations and Conventions
[*] Gamma Distributions
[*] Related Distributions of Statistics
[*] Some Common Densities
[*] Randomization and Mixtures
[*] Discrete Distributions
[*] Bessel Functions and Random Walks
[*] Distributions on a Circle
[*] Problems for Solution
[/LIST]
[*] Densities in Higher Dimensions. Normal Densities and Processes
[LIST]
[*] Densities
[*] Conditional Distributions
[*] Return to the Exponential and the Uniform Distributions
[*] A Characterization of the Normal Distribution
[*] Matrix Notation. The Covariance Matrix
[*] Normal Densities and Distributions
[*] Stationary Normal Processes
[*] Markovian Normal Densities
[*] Problems for Solution
[/LIST]
[*] Probability Measures and Spaces
[LIST]
[*] Baire Functions
[*] Interval Functions and Integrals in [itex]\mathcal{R}^r[/itex]
[*] [itex]\sigma[/itex]-Algebras. Measurability
[*] Probability Spaces. Random Variables
[*] The Extension Theorem
[*] Product Spaces. Sequences of Independent Variables
[*] Null Sets. Completion
[/LIST]
[*] Probability Distributions in [itex]\mathcal{R}^r[/itex]
[LIST]
[*] Distributions and Expectations
[*] Preliminaries
[*] Densities
[*] Convolutions
[*] Symmetrization
[*] Integration by Parts. Existence of Moments
[*] Chebyshev's Inequality
[*] Further Inequalities. Convex Functions
[*] Simple Conditional Distributions. Mixtures
[*] Conditional Distributions
[*] Conditional Expectations
[*] Problems for Solution
[/LIST]
[*] A Survey of some Important Distributions and Processes
[LIST]
[*] Stable Distributions in [itex]\mathcal{R}^1[/itex]
[*] Examples
[*] Infinitely Divisible Distributions in [itex]\mathcal{R}^1[/itex]
[*] Processes with Independent Increments
[*] Ruin Problems in Compound Poisson Processes
[*] Renewal Processes
[*] Examples and Problems
[*] Random Walks
[*] The Queuing Process
[*] Persistent and Transient Random Walks
[*] General Markov Chains
[*] Martingales
[*] Problems for Solution
[/LIST]
[*] Laws of Large Numbers. Applications in Analysis
[LIST]
[*] Main Lemma and Notations
[*] Bernstein Polynomials. Absolutely Monotone Functions 
[*] Moment Problems
[*] Application to Exchangeable Variables
[*] Generalized Taylor Formula and Semi-Groups
[*] Inversion Formulas for Laplace Transforms
[*] Laws of Large Numbers for Identically Distributed Variables
[*] Strong Laws
[*] Generalization to Martingales
[*] Problems for Solution
[/LIST]
[*] The Basic Limit Theorems
[LIST]
[*] Convergence of Measures
[*] Special Properties
[*] Distributions as Operators
[*] The Central Limit Theorem
[*] Infinite Convolutions
[*] Selection Theorems
[*] Ergodic Theorems for Markov Chains
[*] Regular Variation
[*] Asymptotic Properties of Regularly Varying Functions
[*] Problems for Solution
[/LIST]
[*] Infinitely Divisible Distributions and Semi-Groups
[LIST]
[*] Orientation
[*] Convolution Semi-Groups
[*] Preparatory Lemmas
[*] Finite Variances
[*] The Main Theorems
[*] Example: Stable Semi-Groups
[*] Triangular Arrays with Identical Distributions
[*] Domains of Attraction
[*] Variable Distributions. The Three-Series Theorem
[*] Problems for Solution
[/LIST]
[*] Markov Processes and Semi-Groups
[LIST]
[*] The Pseudo-Poisson Type
[*] A Variant: Linear Increments
[*] Jump Processes
[*] Diffusion Processes in [itex]\mathbb{R}^1[/itex]
[*] The Forward Equation. Boundary Conditions
[*] Diffusion in Higher Dimensions
[*] Subordinated Processes
[*] Markov Processes and Semi-Groups
[*] The "Exponential Formula" of Semi-Group Theory
[*] Generators. The Backward Equation
[/LIST]
[*] Renewal Theory
[LIST]
[*] The Renewal Theorem
[*] Proof of the Renewal Theorem
[*] Refinements
[*] Persistent Renewal Processes
[*] The Number [itex]N_t[/itex] of Renewal Epochs
[*] Terminating (Transient) Processes
[*] Diverse Applications
[*] Existence of Limits in Stochastic Processes
[*] Renewal Theory on the Whole Line 
[*] Problems for Solution
[/LIST]
[*] Random Walks in [itex]\mathcal{R}^1[/itex]
[LIST]
[*] Basic Concepts and Notations
[*] Duality. Types of Random Walks
[*] Distribution of Ladder Heights. Wiener-Hopf Factorization
[LIST]
[*] The Wiener-Hopf Integral Equation
[/LIST]
[*] Examples
[*] Applications
[*] A Combinatorial Lemma
[*] Distribution of Ladder Epochs
[*] The Arc Sine Laws
[*] Miscellaneous Complements
[*] Problems for Solution
[/LIST]
[*] Laplace Transforms. Tauberian Theorems. Resolvents
[LIST]
[*] Definitions. The Continuity Theorem
[*] Elementary Properties 
[*] Examples 
[*] Completely Monotone Functions. Inversion Formulas
[*] Tauberian Theorems
[*] Stable Distributions
[*] Infinitely Divisible Distributions
[*] Higher Dimensions
[*] Laplace Transforms for Semi-Groups
[*] The Hille-Yosida Theorem
[*] Problems for Solution
[/LIST]
[*] Applications of Laplace Transforms
[LIST]
[*] The Renewal Equation: Theory
[*] Renewal-Type Equations: Examples
[*] Limit Theorems Involving Arc Sine Distributions
[*] Busy Periods and Related Branching Processes
[*] Diffusion Processes
[*] Birth-and-Death Processes and Random Walks
[*] The Kolmogorov Differential Equations
[*] Example: The Pure Birth Process
[*] Calculation of Ergodic Limits and of First-Passage Times
[*] Problems for Solution
[/LIST]
[*] Characteristic Functions
[LIST]
[*] Definition. Basic Properties
[*] Special Distributions. Mixtures
[LIST]
[*] Some Unexpected Phenomena
[/LIST]
[*] Uniqueness. Inversion Formulas
[*] Regularity Properties
[*] The Central Limit Theorem for Equal Components
[*] The Lindeberg Conditions
[*] Characteristic Functions in Higher Dimensions
[*] Two Characterizations of the Normal Distribution
[*] Problems for Solution
[/LIST]
[*] Expansions Related to the Central Limit Theorem
[LIST]
[*] Notations
[*] Expansions for Densities
[*] Smoothing
[*] Expansions for Distributions
[*] The Berry-Esseen Theorems
[*] Expansions in the Case of Varying Components
[*] Large Deviations
[/LIST]
[*] Infinitely Divisible Distributions
[LIST]
[*] Infinitely Divisible Distributions 554 
[*] Canonical Forms. The Main Limit Theorem
[LIST]
[*] Derivatives of Characteristic Functions
[/LIST] 
[*] Examples and Special Properties
[*] Special Properties
[*] Stable Distributions and Their Domains of Attraction
[*] Stable Densities
[*] Triangular Arrays
[*] The Class L
[*] Partial Attraction. "Universal Laws"
[*] Infinite Convolutions
[*] Higher Dimensions
[*] Problems for Solution
[/LIST]
[*] Applications of Fourier Methods to Random Walks
[LIST]
[*] The Basic Identity
[*] Finite Intervals. Wald's Approximation
[*] The Wiener-Hopf Factorization
[*] Implications and Applications
[*] Two Deeper Theorems
[*] Criteria for Persistency
[*] Problems for Solution
[/LIST]
[*] Harmonic Analysis
[LIST]
[*] The Parseval Relation
[*] Positive Definite Functions
[*] Stationary Processes
[*] Fourier Series
[*] The Poisson Summation Formula
[*] Positive Definite Sequences
[*] [itex]L^2[/itex] Theory
[*] Stochastic Processes and Integrals
[*] Problems for Solution
[/LIST]
[*] Answers to Problems
[*] Some Books on Cognate Subjects
[*] Index 
[/LIST]

 

[MarneMath]

William Feller's volumes are considered classics in the probability community. Feller was an extremely influential mathematician and motivated a great deal of study into the analytic study of the field, along with the application. These volumes were originally written back in the 60's with the, I believe, the encouragement of the naval war college. Feller goal was to write a good overview of probability without losing the mathematical sophistication but not forgetting that it's important to see and understand how to apply the concepts.

Feller can be a bit verbose and it's easy to lose important information glazing over the text. However, for a student interested in seeing plenty of examples of concepts, you can't go wrong with Feller.

Feller wrote two volumes, the first is probability theory, but it only deals with discrete. He does this to keep the mathematics simple and focus more on the the theory. I found this to be reasonable, since often times in probability a less than stellar student will get confused in technical terms and even struggle with setting up integrals for the continuous case. Also, it's relatively easy to extend the concepts learned here to the continuous case, he also points out some of the nuances between the two.

If you're lucky enough to find volume 2, then hold onto it and never let it go. The second volume uses measure theory to describe probability. In probability, you can either view probability as branch of measure theory or see measure theory as a tool for probability. I'm a fan of the second view, and so is this book. This volume focuses more on the intuitive reasoning behind why we need to use certain ideas in measure theory, instead of focusing on the less motivated frame work of extending measure theory results and then explaining it's probability importance.

With all that said, this book isn't easy at all. It requires deep and careful consideration of the problems and words written. It's dense and knowing formulas mean very little. Some say this book isn't suitable for a beginner, I disagree. It isn't suitable for students who need formulas and answer, but have confidence in their ability to prove and check their answers with reality. Is that easy? No, but it's only when we struggle with our lack of understanding can real learning occur. I would definitely recommend these volumes to any student who wishes to see the creativity that probability can have and the wide reaching use.

 

[mathwonk]

I don't have a real good memory of this book, but we used vol.1 in an introductory probability course in college at Harvard in about 1964 or 1965. I remember it as hard but solid. Unfortunately the main thing I recall was getting only a B+ in spite of working very hard for an A, because of one 50 point question the professor, a fresh young PhD apparently in a different area, put on the final exam.

It was a homework problem so hard no one got it, even grad students, so he worked it out in detail in class. I happened to be out sick that one day, and could not work it on the final in the time allotted. That brought down my A average. He was generous enough to give me a B+ for the course, but it still seemed unfair somehow.

So I do recall that the book has a few very hard problems. It has always been considered a standard source to my knowledge. It is also one of the very few books I ultimately gave away or sold, to my later regret as always in such cases. So although I only recommended it lightly, it probably deserves a stronger recommendation.