How do I make this integral substitution

1. Apr 6, 2013

mjordan2nd

If I want to find $\int f(x) dx$ where $x=a+b$ and a and b are both variables how would I do this in terms of a and b. Let me give you some context about this question.

I'm trying to understand the following in my statistical mechanics class. My book states that if I have a random variable x that follows a gaussian distribution then

$$<e^{iqx}> = \int_\infty^\infty dx \frac{ e^{ - \frac{ \beta (x-<x>)^2}{2}}} { \sqrt{ 2 \pi / \beta}} e^{iqx} = e^{iq<x> - \frac{q^2}{2 \beta}} .$$

where q is real. It then seems to pull the following out from the deepest regions of the netherworld using sorcery and other very, very dark arts:

$$\langle e^{i \sum_{j=1}^N q_j x_j} \rangle = exp \left( i \sum_{j=1}^N q_j <x_j> - \sum_{j,k=1}^N q_j G_{jk} \frac{q_k}{2} \right).$$

In order to combat the dark arts, I'm trying to prove this to myself. That is the context that the integral I originally asked about came up.

Any help is appreciated. Thanks.

Last edited: Apr 6, 2013
2. Apr 6, 2013

Stephen Tashi

I think what you really want is to find the expected value of f(x) where x is the sum of two random variables a and b (As the saying goes, "Random variables aren't random and they aren't variables")

You didn't explain the $x_i$ and the $G_{jk}$. From the appearance of the equation, it looks like the basic pattern involves finding the expected value of the product of random variables. In the case of two random variables X,Y we have

$<XY>\ =\ <X><Y>\ +\ COV(X,Y)$

In your case you have something like $X = e^{i q_1 x_1}$ and $Y = e^{i q_2 x_2}$. The $G_{jk}$ might be relevant to covariances. That's my guess.

Anyway, what you need is results from probability theory, not merely results from calculus.

Last edited: Apr 6, 2013
3. Apr 7, 2013

fzero

We can do integrals like this by "completing the square." Group the terms in the exponent together as

$$- \frac{\beta (x-\langle x\rangle)^2}{2} + iqx = - \frac{\beta }{2} \left(x-\langle x\rangle-i\frac{q}{\beta}\right)^2 -\frac{q^2}{2\beta} + i q\langle x\rangle .$$

This lets us rewrite

$$\langle e^{iqx}\rangle = \exp \left(-\frac{q^2}{2\beta} + i q\langle x\rangle\right) \int_{-\infty}^\infty \frac{dx}{\sqrt{ 2 \pi / \beta}} \exp\left( - \frac{\beta }{2} \left(x-\langle x\rangle-i\frac{q}{\beta}\right)^2\right),$$

where we pulled out the terms in front that did not depend on $x$. Now we would make a change of variables in the integral to express it as the usual Gaussian. The change of variables is complex, so I suppose one should be a bit careful with it.

This integral can be done by completing the square as well. You will want to first work out the multivariable Gaussian

$$\int_{-\infty}^\infty dx_1 \cdots \int_{-\infty}^\infty dx_N \exp \left( - \frac{1}{2} \sum_{i,j=1}^N x_i A_{ij} x_j \right) = \sqrt{\frac{(2\pi)^N}{\det A}}.$$

The simplest way to see this is to change variables via the matrix that diagonalizes $A$. When you then consider the expectation value you quoted, you should find that the matrix $G = A^{-1}$.

4. Apr 7, 2013

mjordan2nd

Thank you both! The xs are random variables, and the G matrix is indeed a correlation matrix.

Fzero, thank you very much for your response. I computed the first integral as you indicated (well, one very similar for a homework problem, anyway). Your advice to do it as a completing the square problem saved me a lot of time. In the multivariable Gaussian you asked me to compute, I don't really understand why the Gaussian has that form -- I will look this up. I suppose the A matrix is also for correlation. If the random variables were independent would the matrix be equal to the Kronecker-delta?

5. Apr 8, 2013

fzero

You could compute something like $\langle x_i x_j\rangle$ and you should find, using Stephan's expression, that $A^{-1}$ is related to the covariance matrix. Just roughly looking at the integral it seems that $\mathrm{Cov} = A^{-1} - I$. This does seem to imply that $A=I$ if the variables are independent, but it would seem that any diagonal $A$ should correspond to independent variables as well. So perhaps my argument is a bit too fast and you can find the correct condition by doing the a more careful calculation.

6. Apr 15, 2013

mjordan2nd

I finally got around to trying to looking at what that G matrix really represents. According to my book

$$G_{jk} = \langle (X_j - <X_j>)(X_k-<X_k>) \rangle.$$

I wanted to prove this, but my derivation is off by a factor of two, and I'm not really sure why. It may be something as simple as misunderstanding the multivariable Taylor expansion, but if that's the case I can't figure out why. I can't really find a good reference for the multivariable Taylor expansion so I suspect that is the issue or that the book is incorrect. Fzero, I had a bit of trouble with your method so instead I tried to Taylor expand both sides in terms of the qs and then equate the two sides term by term. First I rewrote the second equation in my original post as

$$\frac{1}{Z} \int_{\Omega} e^{- \beta H} e^{i \sum_{j=1}^N q_j (x_j-<x_j>)} = exp \left( - \sum_{j,k=1}^N q_j G_{jk} \frac{q_k}{2} \right).$$

$$y_j = x_j-<x_j>$$

$$\frac{1}{Z} \int_{\Omega} e^{- \beta H} e^{i \sum_{j=1}^N q_j y_j} = <e^{i \sum_{j=1}^N q_j y_j}> = exp \left( - \sum_{j,k=1}^N q_j G_{jk} \frac{q_k}{2} \right).$$

Taylor expanding the left hand side in terms of the q's around q=0 I got (to second order)

$$<e^{i \sum_{j=1}^N q_j y_j}> = \langle 1 + i \sum_j y_j q_j - \frac{1}{2} \sum_{ij} y_i y_j q_i q_j + ... \rangle,$$

and expanding the right-hand side

$$1+\sum_i \frac{\partial}{\partial q_i} \left[ exp \left( - \sum_{j,k=1}^N q_j G_{jk} \frac{q_k}{2} \right) \right] = 1 + \sum_{ij} \frac{1}{2} G_{ji} q_j q_i + \sum_{ik} \frac{1}{2} G_{ik}q_i q_k = 1 + \sum_{jk} G_{jk} q_j q_k$$

Where in the last step I combined the two terms due to the symmetry of the matrix G (which the books states is required, though this is not clear to me). Based on this it appears that $<y_i y_j> = 2 G_{ij}.$ However, while typing this it occurred to me that this discrepancy between my calculation and the books calculation could be from the fact that you have to count $\frac{1}{2} \sum_{ij} y_i y_j q_i q_j$ basically twice because interchanging the indices pretty much gives you the same thing. Is this the correct explanation for the difference between my calculation and the text's? I'm sorry -- my (the book's...?) notation is awfully difficult to keep up with, but could anyone confirm my suspicion? Thanks!

Edit: In fact, now, looking at it in terms of my explanation for the discrepancy, the fact that G is symmetric seems to be becoming clear. It's finally all coming together, I think.

Last edited: Apr 15, 2013
7. Apr 16, 2013

fzero

The Taylor expansion method should work, but you made a mistake doing the expansion of the right-hand side that I'll point out in a bit. In the meantime, I can't help but point out something very closely related that turns out to be extremely useful in computing correlation functions in practice. The expression you have above is an example of something called a "generating function." This means that we can actually compute the expected value of any function of the $x_i$ by taking appropriate derivatives of the generating function. We will call the generating function $W[q_j]$, then copying your formula, we have

$$W[q_j]= \frac{1}{Z} \int_{\Omega} e^{- \beta H} e^{i \sum_{j=1}^N q_j (x_j-<x_j>)} = \exp \left( - \sum_{j,k=1}^N q_j G_{jk} \frac{q_k}{2} \right).$$

Note that taking the derivative with respect to a $q_i$ brings down a factor of $x_i - \langle x_i \rangle$ inside the integral:

$$- i \frac{\partial}{\partial q_i} W[q_i] = \left\langle (x_i - \langle x_i \rangle) e^{i \sum_{j=1}^N q_j (x_j-<x_j>)} \right\rangle = i \sum_{k} G_{ik} q_k \exp \left( - \sum_{j,k=1}^N q_j G_{jk} \frac{q_k}{2} \right).$$

I used the fact that $G_{ij}$ can be taken to be symmetric. Since you had a question about that below, the reason is that it appears in the sum

$$\sum_{ij} G_{ij} q_i q_j .$$

Since the product $q_i q_j$ is symmetric under $i\leftrightarrow j$, if G had an antisymmetric part, it would simply drop out in the sum. So we may restrict to symmetric G without loss of generality.

That aside, the expression I got by taking one derivative isn't so interesting, but if we differentiate again, we can get an expression for the correlator that you want

$$- \frac{\partial^2 }{\partial q_i \partial q_j} W[q_i] = \left\langle (x_i - \langle x_i \rangle) (x_j - \langle x_j \rangle)e^{i \sum_{j=1}^N q_j (x_j-<x_j>)} \right\rangle$$
$$= \left[ G_{ij} - \left( \sum_{k} G_{ik} q_k\right) \left( \sum_{m} G_{jm} q_m \right)\right] \exp \left( - \sum_{j,k=1}^N q_j G_{jk} \frac{q_k}{2} \right).$$

We can then set all the $q_i = 0$ to get an expression for the correlation function. Written in terms of your $y_j = x_j-\langle x_j\rangle$, we indeed find that

$$\langle y_i y_j\rangle = G_{ij}.$$

More generally, for any function of the $y_i$, we have an expression that is symbolically

$$\langle f(y_i)\rangle = \left[ f(-i \partial/\partial q_i ) W[q_i] \right]_{q_i=0}.$$

That is, we replace every appearance of a $y_i$ in the expression with the appropriate derivative acting on the generating function, then set all the $q_i = 0$.

I note that my notation seems a bit condensed. By $f(y_i)$, I'm including functions of different $y_i$s, like $y_1 y_2^2 y_9$, or more complicated expressions.

You did the expansion a bit too quickly. Remember that for an ordinary function, the Taylor expansion is

$$f(x) = f(0) + f'(0) x + \frac{1}{2} f''(0) x^2 + \cdots .$$

In the expression above, you should really have

$$W[q_i] = \left. W[q_]\right|_{q_i=0} + \sum_i q_i \left[ \frac{\partial }{\partial q_i}W[q_i] \right]_{q_i=0} +\frac{1}{2} \sum_{ij} q_i q_j \left[ \frac{\partial^2 }{\partial q_i\partial q_j}W[q_i] \right]_{q_i=0}+ \cdots .$$

When you compare the resulting terms, the 1/2 coefficient in this expansion makes the factors of 2 work out, along with your argument about the symmetry of G.

We can also see why the method I described above works out, since the derivatives I was taking match up with what you do here in the Taylor expansion.

8. Apr 21, 2013

mjordan2nd

I'm sorry to be so delayed in my responses. I appreciate your responses, but I've been quite bogged down in homework/tests and studying with finals coming up, and this problem is quite algebraically involved.

I want to thank you for your explanation about these moment generating functions. My book mentioned this term in passing when discussing this equation, but I was not aware that this was the best method to do this problem.

However, I still wanted to do this as a Taylor series problem because I don't believe I've ever tried doing a multivariable Taylor series, and I figured it was a good skill to learn. I was able to prove that the G matrix was equal to the correlation coefficients as desired using the Taylor series. I missed the fact that the third term in the Taylor expansion could have a quadratic term, and I believe I miscalculated the second term when I initially did it, which made it seem like I was close but not quite there. I did have one additional question about the Taylor series, though: it seems that the second term in the expansion of the right hand side went to 0. The left hand side had an imaginary part for that term... I'm wondering if this is correct. Based on my (possibly flawed calculations) the real parts match for both sides, but not the imaginary parts.

I did the calculation by taking derivatives as well, and the answer popped out as you indicate quite easily. I also calculated the $<Y_iY_jY_kY_m>$ using your method (which was a suggested homework problem in the book) and everything seemed to come out quite nicely.

$$\langle y_i\rangle = \langle ( x_i - \langle x_i\rangle) \rangle = \langle x_i\rangle -\langle x_i\rangle=0.$$
This takes care of that $i y_i q_i$ term on the LHS.