Characteristic function of the sum of random variables

In summary: X_n## variables. The convolution of these two pdfs is##f(x) = \int_{-\infty}^{\infty}g(x-x_1)~dx_1 \int_{-\infty}^{\infty}g(x-x_2)~dx_2##
  • #1
binbagsss
1,254
11

Homework Statement



charofx.png
I am trying to understand the very last equality for (let me replace the tilda with a hat ) ##\hat{P_{X}(K)}=\hat{P(k_1=k_2=...=k_{N}=k)}##(1)

Homework Equations



jointchar.png


I also thought that the following imaginary exponential delta identity may be useful, due to the equality of the ##k_i##, but see comments below:

##\int dk \exp^{ikx} = \delta(x=0) ##

The Attempt at a Solution



So it sees to me the goal is something like expressing ##\hat{P_{X}(K)}## in terms of ##P(\hat{k_i})## ?

So these are given by ##\hat{P(k_1)...P(k_n)}= \int d^{N} \vec{x} p(\vec{x}) \exp^{-i \sum\limits_j x_j k_j } ##

I thought I`d first try to look at the simplified case of indepedent random variables to understand (1) but still can`t seem to get it.

So in this case ##p(\vec{x}) = \Pi_{i} p(x_i)##

And then we have

(If I am correct in that the notation is that ##\Pi_{i} dx_i = d^N (\vec{x}) ##)
##\hat{P(k_1)...P(k_n)}= \int \Pi_{i} dx_i p(x_i) \exp^{-i \sum\limits_j x_j k_j } ##
and then you can separate the integrals and so we have:

##\hat{P(k_1)...P(k_n)}= \hat{P_{x_1}(k_1)} ...\hat{P_{x_n}(k_n)} ## (2)

Now if I consider the independent case in ##\hat{P_{X}(K)}## I have:

##\hat{P_{X}(K)} =\int \Pi_{i} dx_i p(x_i) \exp^{-i k \sum\limits_j x_j }
= \int dx_1 p(x_1) e^{-ix_1 k} \int dx_2 p(x_2) e^{-ix_2 k}...\int dx_N p(x_N) e^{-ikx_N}
= \hat{P_{x_1}(k)} ...\hat{P_{x_n}(k)} ##

So if I compare this to (2), and can reason( I'm not sure you can) that it does matter whether you have ##k_i## or ##k##, this is just the label of the Fourier transform, but look at the lower notation that gives the distribution, that ##\hat{P_{x_1}(k)}= \hat{P_{x_1}(k_1)} ## and then I have ##k_1=k## and can do the same for each ##k_i## etc.

Without independence instead i have:

##\int d^{N} \vec{x} p(\vec{x}) \exp^{-i k \sum\limits_j x_j } ##

and I can't see how you can make any conclusions without knowing what ## p(\vec{x}) ## is?

Many thanks
 

Attachments

  • jointchar.png
    jointchar.png
    56.8 KB · Views: 963
  • charofx.png
    charofx.png
    63.2 KB · Views: 1,126
Physics news on Phys.org
  • #2
binbagsss said:

Homework Statement



View attachment 226899I am trying to understand the very last equality for (let me replace the tilda with a hat ) ##\hat{P_{X}(K)}=\hat{P(k_1=k_2=...=k_{N}=k)}##(1)

Homework Equations



View attachment 226898

I also thought that the following imaginary exponential delta identity may be useful, due to the equality of the ##k_i##, but see comments below:

##\int dk \exp^{ikx} = \delta(x=0) ##

The Attempt at a Solution



So it sees to me the goal is something like expressing ##\hat{P_{X}(K)}## in terms of ##P(\hat{k_i})## ?

So these are given by ##\hat{P(k_1)...P(k_n)}= \int d^{N} \vec{x} p(\vec{x}) \exp^{-i \sum\limits_j x_j k_j } ##

I thought I`d first try to look at the simplified case of indepedent random variables to understand (1) but still can`t seem to get it.

So in this case ##p(\vec{x}) = \Pi_{i} p(x_i)##

And then we have

(If I am correct in that the notation is that ##\Pi_{i} dx_i = d^N (\vec{x}) ##)
##\hat{P(k_1)...P(k_n)}= \int \Pi_{i} dx_i p(x_i) \exp^{-i \sum\limits_j x_j k_j } ##
and then you can separate the integrals and so we have:

##\hat{P(k_1)...P(k_n)}= \hat{P_{x_1}(k_1)} ...\hat{P_{x_n}(k_n)} ## (2)

Now if I consider the independent case in ##\hat{P_{X}(K)}## I have:

##\hat{P_{X}(K)} =\int \Pi_{i} dx_i p(x_i) \exp^{-i k \sum\limits_j x_j }
= \int dx_1 p(x_1) e^{-ix_1 k} \int dx_2 p(x_2) e^{-ix_2 k}...\int dx_N p(x_N) e^{-ikx_N}
= \hat{P_{x_1}(k)} ...\hat{P_{x_n}(k)} ##

So if I compare this to (2), and can reason( I'm not sure you can) that it does matter whether you have ##k_i## or ##k##, this is just the label of the Fourier transform, but look at the lower notation that gives the distribution, that ##\hat{P_{x_1}(k)}= \hat{P_{x_1}(k_1)} ## and then I have ##k_1=k## and can do the same for each ##k_i## etc.

Without independence instead i have:

##\int d^{N} \vec{x} p(\vec{x}) \exp^{-i k \sum\limits_j x_j } ##

and I can't see how you can make any conclusions without knowing what ## p(\vec{x}) ## is?

Many thanks
I don't see that you have considered convolution of the distribution functions here. The sum of two iid random variables ##X = X_1+X_2## has a pdf
##f(x) = \int_{-\infty}^{\infty}g(x-x_1)g(x_1)~dx_1##
where ##g(x_i)## is the pdf of the ##X_1## and ##X_2##.
 
  • #3
tnich said:
I don't see that you have considered convolution of the distribution functions here. The sum of two iid random variables ##X = X_1+X_2## has a pdf
##f(x) = \int_{-\infty}^{\infty}g(x-x_1)g(x_1)~dx_1##
where ##g(x_i)## is the pdf of the ##X_1## and ##X_2##.
Looking again at your problem statement, I can see that you are trying to do something with random variables that are not independent, but you never state the question that you want to answer. You say you can't draw any conclusions about ##p(\vec{x})##. What kind of conclusions do you want to draw?
 
  • #4
tnich said:
Looking again at your problem statement, I can see that you are trying to do something with random variables that are not independent, but you never state the question that you want to answer. You say you can't draw any conclusions about ##p(\vec{x})##. What kind of conclusions do you want to draw?
see bottom of my OP under section 1. I am trying to understand the last equailty...?
 
  • #5
binbagsss said:
see bottom of my OP under section 1. I am trying to understand the last equailty...?
So am I. I am having trouble parsing the right-hand side of
##\tilde P_{X}(k)=\tilde P(k_1=k_2=...=k_{N}=k)##
It is a characteristic function, so it is the expectation of a function ##exp\{-ikf(x_1,x_2,. . .,x_n)\}##, and we know what that function looks like. But ##\tilde P(k_1=k_2=...=k_{N}=k)## seems to be a meaningless expression.
 
  • #6
tnich said:
So am I. I am having trouble parsing the right-hand side of
##\tilde P_{X}(k)=\tilde P(k_1=k_2=...=k_{N}=k)##
It is a characteristic function, so it is the expectation of a function ##exp\{-ikf(x_1,x_2,. . .,x_n)\}##, and we know what that function looks like. But ##\tilde P(k_1=k_2=...=k_{N}=k)## seems to be a meaningless expression.

If ##\tilde{P}_X(k_1, k_2, \ldots, k_n)= \tilde{P}_X(\mathbf{k})## is the (multivariate) characteristic function of the multivariate density ##p(x_1, x_2, \ldots, x_n) = p(\mathbf{x})##, then, by definition we have
$$\tilde{P}_X(\mathbf{k}) = \int \exp(-i \: \mathbf{k \cdot x})\, p(\mathbf{x}) \, d^n \mathbf{x} $$
Also, by definition, if ##f_Y(y)## is the density function of the random variable ##Y = X_1 + X_2 + \cdots + X_n##, its characteristic function is
$$\tilde{P}_Y(k) = \int \exp(-i \, k y) f_Y(y) \, dy.$$
However, because of the Law of the Unconscious Statistician (LOTUS), we can write
$$\tilde{P}_Y(k) = \int \exp(-i k \sum_j x_j)\, p(\mathbf{x}) \, d^n \mathbf{x} \hspace{3em} (1) $$
(Note that in principle, we cannot simply write this down; we need to justify it. That is where the LOTUS comes in.)

Once we have equation (1) we can simply say that
$$\tilde{P}_Y(k) = \tilde{P}_X(k,k,k,\ldots,k).$$
That is probably what the author meant by his/her ill-advised notation.

For more on the LOTUS, see, eg.,
https://en.wikipedia.org/wiki/Law_of_the_unconscious_statistician (just the bare basics, with no proofs)
or
https://math.stackexchange.com/questions/415196/proving-the-law-of-the-unconscious-statistician (statement and proof)

Apparently, some authors have dropped that name in newer additions of their books, because of complaints from statisticians. Nevertheless, some other books have shown the continuing need for such a result, because some authors really do use the result 'unconsciously", or at least, without alerting the reader to the issue.
 
Last edited:
  • Like
Likes tnich
  • #7
Ray Vickson said:
If ##\tilde{P}_X(k_1, k_2, \ldots, k_n)= \tilde{P}_X(\mathbf{k})## is the (multivariate) characteristic function of the multivariate density ##p(x_1, x_2, \ldots, x_n) = p(\mathbf{x})##, then, by definition we have
$$\tilde{P}_X(\mathbf{k}) = \int \exp(-i \: \mathbf{k \cdot x})\, p(\mathbf{x}) \, d^n \mathbf{x} $$
Also, by definition, if ##f_Y(y)## is the density function of the random variable ##Y = X_1 + X_2 + \cdots + X_n##, its characteristic function is
$$\tilde{P}_Y(k) = \int \exp(-i \, k y) f_Y(y) \, dy.$$
However, because of the Law of the Unconscious Statistician (LOTUS), we can write
$$\tilde{P}_Y(k) = \int \exp(-i k \sum_j x_j)\, p(\mathbf{x}) \, d^n \mathbf{x} \hspace{3em} (1) $$
(Note that in principle, we cannot simply write this down; we need to justify it. That is where the LOTUS comes in.)

Once we have equation (1) we can simply say that
$$\tilde{P}_Y(k) = \tilde{P}_X(k,k,k,\ldots,k).$$
That is probably what the author meant by his/her ill-advised notation.

For more on the LOTUS, see, eg.,
https://en.wikipedia.org/wiki/Law_of_the_unconscious_statistician (just the bare basics, with no proofs)
or
https://math.stackexchange.com/questions/415196/proving-the-law-of-the-unconscious-statistician (statement and proof)

Apparently, some authors have dropped that name in newer additions of their books, because of complaints from statisticians. Nevertheless, some other books have shown the continuing need for such a result, because some authors really do use the result 'unconsciously", or at least, without alerting the reader to the issue.
Thanks! I think you meant ##\tilde{P}_Y(k) = \tilde{P}_X(k,k,k,\ldots,k)##, and you have given a good answer to the question in the OP about what that equality means.
 

1. What is the characteristic function of the sum of random variables?

The characteristic function of the sum of random variables is a mathematical function that describes the probability distribution of the sum of two or more independent random variables. It is defined as the expectation of the complex exponential function of the sum of the random variables.

2. Why is the characteristic function important in probability theory?

The characteristic function is important because it provides a way to analyze and understand the behavior of random variables. It allows for the calculation of moments, such as the mean and variance, and can be used to prove central limit theorems and other important properties of random variables.

3. How is the characteristic function related to the probability density function?

The characteristic function and the probability density function are related through the inverse Fourier transform. The probability density function can be obtained by taking the inverse Fourier transform of the characteristic function.

4. Can the characteristic function be used to calculate probabilities?

Yes, the characteristic function can be used to calculate probabilities through the inverse Fourier transform. It provides a way to calculate the probability of a random variable falling within a certain range or satisfying a certain condition.

5. Can the characteristic function be used for non-independent random variables?

Yes, the characteristic function can be used for non-independent random variables. However, in this case, the characteristic function of the sum of random variables is not simply the product of their individual characteristic functions. More complex methods, such as the convolution of characteristic functions, must be used to calculate the characteristic function of the sum.

Similar threads

  • Calculus and Beyond Homework Help
Replies
8
Views
649
  • Advanced Physics Homework Help
Replies
1
Views
795
  • Calculus and Beyond Homework Help
Replies
1
Views
1K
  • Calculus and Beyond Homework Help
Replies
7
Views
1K
  • Calculus and Beyond Homework Help
Replies
10
Views
1K
  • Differential Geometry
Replies
10
Views
4K
Replies
10
Views
1K
  • Advanced Physics Homework Help
Replies
1
Views
1K
Replies
12
Views
722
  • Calculus and Beyond Homework Help
Replies
3
Views
2K
Back
Top