How do I make this integral substitution

In summary: The Gaussian form appears because of the quadratic form in the exponent. If you diagonalize it, you will get a product of one-dimensional Gaussian integrals. If you don't diagonalize it, you can "complete the square" by shifting the integration variable by a linear function, and get a product of one-dimensional integrals with constants added to the argument of the exponent. This is the same idea as when you complete the square in algebra: you put the quadratic expression into a sum of squares and a constant. The sum of squares is the important part; it is always positive, so the exponential is always positive. The constant just multiplies the integrand by a constant factor, and it is usually easy to do
  • #1
mjordan2nd
177
1
If I want to find [itex] \int f(x) dx [/itex] where [itex] x=a+b [/itex] and a and b are both variables how would I do this in terms of a and b. Let me give you some context about this question.

I'm trying to understand the following in my statistical mechanics class. My book states that if I have a random variable x that follows a gaussian distribution then

[tex]
<e^{iqx}> = \int_\infty^\infty dx \frac{ e^{ - \frac{ \beta (x-<x>)^2}{2}}} { \sqrt{ 2 \pi / \beta}} e^{iqx} = e^{iq<x> - \frac{q^2}{2 \beta}} .
[/tex]

where q is real. It then seems to pull the following out from the deepest regions of the netherworld using sorcery and other very, very dark arts:

[tex]
\langle e^{i \sum_{j=1}^N q_j x_j} \rangle = exp \left( i \sum_{j=1}^N q_j <x_j> - \sum_{j,k=1}^N q_j G_{jk} \frac{q_k}{2} \right).
[/tex]

In order to combat the dark arts, I'm trying to prove this to myself. That is the context that the integral I originally asked about came up.

Any help is appreciated. Thanks.
 
Last edited:
Physics news on Phys.org
  • #2
mjordan2nd said:
If I want to find [itex] \int f(x) dx [/itex] where [itex] x=a+b [/itex] and a and b are both variables how would I do this in terms of a and b.

I think what you really want is to find the expected value of f(x) where x is the sum of two random variables a and b (As the saying goes, "Random variables aren't random and they aren't variables")

[tex]
\langle e^{i \sum_{j=1}^N q_j x_j} \rangle = exp \left( i \sum_{j=1}^N q_j <x_j> - \sum_{j,k=1}^N q_j G_{jk} \frac{q_k}{2} \right).
[/tex]

You didn't explain the [itex] x_i [/itex] and the [itex] G_{jk} [/itex]. From the appearance of the equation, it looks like the basic pattern involves finding the expected value of the product of random variables. In the case of two random variables X,Y we have

[itex] <XY>\ =\ <X><Y>\ +\ COV(X,Y) [/itex]

In your case you have something like [itex] X = e^{i q_1 x_1} [/itex] and [itex] Y = e^{i q_2 x_2} [/itex]. The [itex] G_{jk} [/itex] might be relevant to covariances. That's my guess.

Anyway, what you need is results from probability theory, not merely results from calculus.
 
Last edited:
  • #3
mjordan2nd said:
If I want to find [itex] \int f(x) dx [/itex] where [itex] x=a+b [/itex] and a and b are both variables how would I do this in terms of a and b. Let me give you some context about this question.

I'm trying to understand the following in my statistical mechanics class. My book states that if I have a random variable x that follows a gaussian distribution then

[tex]
<e^{iqx}> = \int_\infty^\infty dx \frac{ e^{ - \frac{ \beta (x-<x>)^2}{2}}} { \sqrt{ 2 \pi / \beta}} e^{iqx} = e^{iq<x> - \frac{q^2}{2 \beta}} .
[/tex]

where q is real.

We can do integrals like this by "completing the square." Group the terms in the exponent together as

$$- \frac{\beta (x-\langle x\rangle)^2}{2} + iqx
= - \frac{\beta }{2} \left(x-\langle x\rangle-i\frac{q}{\beta}\right)^2
-\frac{q^2}{2\beta} + i q\langle x\rangle .$$

This let's us rewrite

$$\langle e^{iqx}\rangle = \exp \left(-\frac{q^2}{2\beta} + i q\langle x\rangle\right) \int_{-\infty}^\infty \frac{dx}{\sqrt{ 2 \pi / \beta}} \exp\left( - \frac{\beta }{2} \left(x-\langle x\rangle-i\frac{q}{\beta}\right)^2\right),$$

where we pulled out the terms in front that did not depend on ##x##. Now we would make a change of variables in the integral to express it as the usual Gaussian. The change of variables is complex, so I suppose one should be a bit careful with it.

It then seems to pull the following out from the deepest regions of the netherworld using sorcery and other very, very dark arts:

[tex]
\langle e^{i \sum_{j=1}^N q_j x_j} \rangle = exp \left( i \sum_{j=1}^N q_j <x_j> - \sum_{j,k=1}^N q_j G_{jk} \frac{q_k}{2} \right).
[/tex]

In order to combat the dark arts, I'm trying to prove this to myself. That is the context that the integral I originally asked about came up.

This integral can be done by completing the square as well. You will want to first work out the multivariable Gaussian

$$\int_{-\infty}^\infty dx_1 \cdots \int_{-\infty}^\infty dx_N \exp \left( - \frac{1}{2} \sum_{i,j=1}^N x_i A_{ij} x_j \right) = \sqrt{\frac{(2\pi)^N}{\det A}}.$$

The simplest way to see this is to change variables via the matrix that diagonalizes ##A##. When you then consider the expectation value you quoted, you should find that the matrix ##G = A^{-1}##.
 
  • #4
Thank you both! The xs are random variables, and the G matrix is indeed a correlation matrix.

Fzero, thank you very much for your response. I computed the first integral as you indicated (well, one very similar for a homework problem, anyway). Your advice to do it as a completing the square problem saved me a lot of time. In the multivariable Gaussian you asked me to compute, I don't really understand why the Gaussian has that form -- I will look this up. I suppose the A matrix is also for correlation. If the random variables were independent would the matrix be equal to the Kronecker-delta?
 
  • #5
mjordan2nd said:
In the multivariable Gaussian you asked me to compute, I don't really understand why the Gaussian has that form -- I will look this up. I suppose the A matrix is also for correlation. If the random variables were independent would the matrix be equal to the Kronecker-delta?

You could compute something like ##\langle x_i x_j\rangle## and you should find, using Stephan's expression, that ##A^{-1}## is related to the covariance matrix. Just roughly looking at the integral it seems that ##\mathrm{Cov} = A^{-1} - I##. This does seem to imply that ##A=I## if the variables are independent, but it would seem that any diagonal ##A## should correspond to independent variables as well. So perhaps my argument is a bit too fast and you can find the correct condition by doing the a more careful calculation.
 
  • #6
I finally got around to trying to looking at what that G matrix really represents. According to my book

[tex]G_{jk} = \langle (X_j - <X_j>)(X_k-<X_k>) \rangle.[/tex]

I wanted to prove this, but my derivation is off by a factor of two, and I'm not really sure why. It may be something as simple as misunderstanding the multivariable Taylor expansion, but if that's the case I can't figure out why. I can't really find a good reference for the multivariable Taylor expansion so I suspect that is the issue or that the book is incorrect. Fzero, I had a bit of trouble with your method so instead I tried to Taylor expand both sides in terms of the qs and then equate the two sides term by term. First I rewrote the second equation in my original post as

[tex]
\frac{1}{Z} \int_{\Omega} e^{- \beta H} e^{i \sum_{j=1}^N q_j (x_j-<x_j>)} = exp \left( - \sum_{j,k=1}^N q_j G_{jk} \frac{q_k}{2} \right).
[/tex]

[tex]
y_j = x_j-<x_j>
[/tex]

[tex]
\frac{1}{Z} \int_{\Omega} e^{- \beta H} e^{i \sum_{j=1}^N q_j y_j} = <e^{i \sum_{j=1}^N q_j y_j}> = exp \left( - \sum_{j,k=1}^N q_j G_{jk} \frac{q_k}{2} \right).
[/tex]

Taylor expanding the left hand side in terms of the q's around q=0 I got (to second order)

[tex]
<e^{i \sum_{j=1}^N q_j y_j}> = \langle 1 + i \sum_j y_j q_j - \frac{1}{2} \sum_{ij} y_i y_j q_i q_j + ... \rangle,
[/tex]

and expanding the right-hand side

[tex]
1+\sum_i \frac{\partial}{\partial q_i} \left[ exp \left( - \sum_{j,k=1}^N q_j G_{jk} \frac{q_k}{2} \right) \right] = 1 + \sum_{ij} \frac{1}{2} G_{ji} q_j q_i + \sum_{ik} \frac{1}{2} G_{ik}q_i q_k = 1 + \sum_{jk} G_{jk} q_j q_k
[/tex]

Where in the last step I combined the two terms due to the symmetry of the matrix G (which the books states is required, though this is not clear to me). Based on this it appears that [itex]<y_i y_j> = 2 G_{ij}.[/itex] However, while typing this it occurred to me that this discrepancy between my calculation and the books calculation could be from the fact that you have to count [itex]\frac{1}{2} \sum_{ij} y_i y_j q_i q_j[/itex] basically twice because interchanging the indices pretty much gives you the same thing. Is this the correct explanation for the difference between my calculation and the text's? I'm sorry -- my (the book's...?) notation is awfully difficult to keep up with, but could anyone confirm my suspicion? Thanks!

Edit: In fact, now, looking at it in terms of my explanation for the discrepancy, the fact that G is symmetric seems to be becoming clear. It's finally all coming together, I think.
 
Last edited:
  • #7
mjordan2nd said:
I finally got around to trying to looking at what that G matrix really represents. According to my book

[tex]G_{jk} = \langle (X_j - <X_j>)(X_k-<X_k>) \rangle.[/tex]

I wanted to prove this, but my derivation is off by a factor of two, and I'm not really sure why. It may be something as simple as misunderstanding the multivariable Taylor expansion, but if that's the case I can't figure out why. I can't really find a good reference for the multivariable Taylor expansion so I suspect that is the issue or that the book is incorrect. Fzero, I had a bit of trouble with your method so instead I tried to Taylor expand both sides in terms of the qs and then equate the two sides term by term. First I rewrote the second equation in my original post as

[tex]
\frac{1}{Z} \int_{\Omega} e^{- \beta H} e^{i \sum_{j=1}^N q_j (x_j-<x_j>)} = exp \left( - \sum_{j,k=1}^N q_j G_{jk} \frac{q_k}{2} \right).
[/tex]

The Taylor expansion method should work, but you made a mistake doing the expansion of the right-hand side that I'll point out in a bit. In the meantime, I can't help but point out something very closely related that turns out to be extremely useful in computing correlation functions in practice. The expression you have above is an example of something called a "generating function." This means that we can actually compute the expected value of any function of the ##x_i## by taking appropriate derivatives of the generating function. We will call the generating function ##W[q_j]##, then copying your formula, we have

$$ W[q_j]= \frac{1}{Z} \int_{\Omega} e^{- \beta H} e^{i \sum_{j=1}^N q_j (x_j-<x_j>)} = \exp \left( - \sum_{j,k=1}^N q_j G_{jk} \frac{q_k}{2} \right).$$

Note that taking the derivative with respect to a ##q_i## brings down a factor of ##x_i - \langle x_i \rangle## inside the integral:

$$ - i \frac{\partial}{\partial q_i} W[q_i] = \left\langle (x_i - \langle x_i \rangle) e^{i \sum_{j=1}^N q_j (x_j-<x_j>)} \right\rangle = i \sum_{k} G_{ik} q_k \exp \left( - \sum_{j,k=1}^N q_j G_{jk} \frac{q_k}{2} \right).$$

I used the fact that ##G_{ij}## can be taken to be symmetric. Since you had a question about that below, the reason is that it appears in the sum

$$ \sum_{ij} G_{ij} q_i q_j .$$

Since the product ##q_i q_j## is symmetric under ##i\leftrightarrow j##, if G had an antisymmetric part, it would simply drop out in the sum. So we may restrict to symmetric G without loss of generality.

That aside, the expression I got by taking one derivative isn't so interesting, but if we differentiate again, we can get an expression for the correlator that you want

$$ - \frac{\partial^2 }{\partial q_i \partial q_j} W[q_i] = \left\langle (x_i - \langle x_i \rangle) (x_j - \langle x_j \rangle)e^{i \sum_{j=1}^N q_j (x_j-<x_j>)} \right\rangle $$
$$= \left[ G_{ij} - \left( \sum_{k} G_{ik} q_k\right) \left( \sum_{m} G_{jm} q_m \right)\right] \exp \left( - \sum_{j,k=1}^N q_j G_{jk} \frac{q_k}{2} \right).$$

We can then set all the ##q_i = 0 ## to get an expression for the correlation function. Written in terms of your ## y_j = x_j-\langle x_j\rangle##, we indeed find that

$$ \langle y_i y_j\rangle = G_{ij}. $$

More generally, for any function of the ##y_i##, we have an expression that is symbolically

$$ \langle f(y_i)\rangle = \left[ f(-i \partial/\partial q_i ) W[q_i] \right]_{q_i=0}. $$

That is, we replace every appearance of a ##y_i## in the expression with the appropriate derivative acting on the generating function, then set all the ##q_i = 0 ##.

I note that my notation seems a bit condensed. By ##f(y_i)##, I'm including functions of different ##y_i##s, like ## y_1 y_2^2 y_9##, or more complicated expressions.
[tex]
y_j = x_j-<x_j>
[/tex]

[tex]
\frac{1}{Z} \int_{\Omega} e^{- \beta H} e^{i \sum_{j=1}^N q_j y_j} = <e^{i \sum_{j=1}^N q_j y_j}> = exp \left( - \sum_{j,k=1}^N q_j G_{jk} \frac{q_k}{2} \right).
[/tex]

Taylor expanding the left hand side in terms of the q's around q=0 I got (to second order)

[tex]
<e^{i \sum_{j=1}^N q_j y_j}> = \langle 1 + i \sum_j y_j q_j - \frac{1}{2} \sum_{ij} y_i y_j q_i q_j + ... \rangle,
[/tex]

and expanding the right-hand side

[tex]
1+\sum_i \frac{\partial}{\partial q_i} \left[ exp \left( - \sum_{j,k=1}^N q_j G_{jk} \frac{q_k}{2} \right) \right] = 1 + \sum_{ij} \frac{1}{2} G_{ji} q_j q_i + \sum_{ik} \frac{1}{2} G_{ik}q_i q_k = 1 + \sum_{jk} G_{jk} q_j q_k
[/tex]

You did the expansion a bit too quickly. Remember that for an ordinary function, the Taylor expansion is

$$f(x) = f(0) + f'(0) x + \frac{1}{2} f''(0) x^2 + \cdots .$$

In the expression above, you should really have

$$ W[q_i] = \left. W[q_]\right|_{q_i=0} + \sum_i q_i \left[ \frac{\partial }{\partial q_i}W[q_i] \right]_{q_i=0} +\frac{1}{2} \sum_{ij} q_i q_j \left[ \frac{\partial^2 }{\partial q_i\partial q_j}W[q_i] \right]_{q_i=0}+ \cdots .$$

When you compare the resulting terms, the 1/2 coefficient in this expansion makes the factors of 2 work out, along with your argument about the symmetry of G.

We can also see why the method I described above works out, since the derivatives I was taking match up with what you do here in the Taylor expansion.

Where in the last step I combined the two terms due to the symmetry of the matrix G (which the books states is required, though this is not clear to me). Based on this it appears that [itex]<y_i y_j> = 2 G_{ij}.[/itex] However, while typing this it occurred to me that this discrepancy between my calculation and the books calculation could be from the fact that you have to count [itex]\frac{1}{2} \sum_{ij} y_i y_j q_i q_j[/itex] basically twice because interchanging the indices pretty much gives you the same thing. Is this the correct explanation for the difference between my calculation and the text's? I'm sorry -- my (the book's...?) notation is awfully difficult to keep up with, but could anyone confirm my suspicion? Thanks!

Edit: In fact, now, looking at it in terms of my explanation for the discrepancy, the fact that G is symmetric seems to be becoming clear. It's finally all coming together, I think.
 
  • #8
I'm sorry to be so delayed in my responses. I appreciate your responses, but I've been quite bogged down in homework/tests and studying with finals coming up, and this problem is quite algebraically involved.

I want to thank you for your explanation about these moment generating functions. My book mentioned this term in passing when discussing this equation, but I was not aware that this was the best method to do this problem.

However, I still wanted to do this as a Taylor series problem because I don't believe I've ever tried doing a multivariable Taylor series, and I figured it was a good skill to learn. I was able to prove that the G matrix was equal to the correlation coefficients as desired using the Taylor series. I missed the fact that the third term in the Taylor expansion could have a quadratic term, and I believe I miscalculated the second term when I initially did it, which made it seem like I was close but not quite there. I did have one additional question about the Taylor series, though: it seems that the second term in the expansion of the right hand side went to 0. The left hand side had an imaginary part for that term... I'm wondering if this is correct. Based on my (possibly flawed calculations) the real parts match for both sides, but not the imaginary parts.

I did the calculation by taking derivatives as well, and the answer popped out as you indicate quite easily. I also calculated the [itex]<Y_iY_jY_kY_m>[/itex] using your method (which was a suggested homework problem in the book) and everything seemed to come out quite nicely.

Thanks again for your help.
 
  • #9
mjordan2nd said:
I did have one additional question about the Taylor series, though: it seems that the second term in the expansion of the right hand side went to 0. The left hand side had an imaginary part for that term... I'm wondering if this is correct. Based on my (possibly flawed calculations) the real parts match for both sides, but not the imaginary parts.

The linear term (actually all odd-order terms) vanish in the expectation value, since

$$\langle y_i\rangle = \langle ( x_i - \langle x_i\rangle) \rangle = \langle x_i\rangle -\langle x_i\rangle=0.$$

This takes care of that ##i y_i q_i## term on the LHS.
 

1. How do I know when to use a substitution for an integral?

Substitution is typically used when the integrand contains a function within another function, such as an inner function raised to a power or inside a trigonometric function. It can also be helpful when the integrand contains a complicated expression that can be simplified with a substitution.

2. What is the process for making an integral substitution?

The first step is to identify the function to be substituted and its corresponding differential. Then, rewrite the integral in terms of the new variable. This may require some manipulation of the integrand to make it fit the form of the substitution. Finally, evaluate the integral in terms of the new variable and then substitute back in the original variable at the end.

3. Can I use any substitution for an integral?

No, substitutions must be chosen carefully and may not always be possible. The substitution must simplify the integrand and result in an easier integral to evaluate. Additionally, the substitution must also be able to be easily reversed to substitute back in the original variable at the end.

4. Can I use multiple substitutions for one integral?

Yes, it is possible to use multiple substitutions for one integral. This may be necessary if the integrand contains multiple functions that can be simplified with different substitutions. However, it is important to keep track of the different variables and make sure to substitute back in the correct one at the end.

5. Are there any common mistakes to avoid when using substitutions for integrals?

One common mistake is forgetting to substitute back in the original variable at the end. It is also important to choose a substitution that simplifies the integrand, as some substitutions may make the integral more complicated. Additionally, it is important to carefully keep track of the different variables and their corresponding differentials when using multiple substitutions.

Similar threads

  • Calculus
Replies
6
Views
1K
Replies
3
Views
1K
Replies
3
Views
1K
Replies
5
Views
1K
Replies
4
Views
609
Replies
3
Views
900
Replies
16
Views
2K
Replies
2
Views
795
  • Calculus
Replies
7
Views
1K
Back
Top