Gaussian process with linear correlation

[gordonc4513]

Hi
I have a stationary gaussian process { X_t } and I have the correlation function p(t) = Corr(X_s, X_(s+t)) = 1 - t/m, where X_t is in {1,2,..,m}, and I want to simulate this process.
The main idea is to write X_t as sum from 1 to N of V_i, X_t+1 as sum from p+1 to N+p V_i and so on, where V_i are uniform variables in [-a, a]. From this point I will use the central limit theorem to prove that X_t is a normal variable. My question is: how can I find the correlation, covariance of X_T and X_t+k for example, a and p using this notification?

 

[Stephen Tashi]

I don't know why you chose sums of independent uniform random variables. Why not use sums of independent normal random variables? Then you know the sums are normally distributed instead of approximately normally distributed.

A specific example of your question is:
Let $u_1,u_2,u_3, u_4$ each be independent uniformly distributed random variables on [0,1].

Let $S_1 = u_1 + u_2 + u_3$
Let $S_2 = u_2 + u_3 + u_4$

Find $COV(S_1,S_2)$.
Find the correlation of $S_1$ with $S_2$.

Well, the covariance of two sums of random variables should be easy enough. See
http://mathworld.wolfram.com/Covariance.html equation 21.

To force the autocorrelation to have a particular shape, you could use weighted sums . That would be another argument for using normal random variables for $V_i$.

 

[gordonc4513]

Thank you for the answer.

Can you explain in detail how can I use the weighted sums to force the autocorrelation and how can I find all the parameters I need?

 

[Stephen Tashi]

Whether I can explain it depends on you background in mathematics, including probability theory - and whether I have time! It isn't a good idea to write computer simulations for any serious purpose without having the mathematical background to understand how they work. If your are doing this for recreation, a board game etc. then perhaps it's Ok.

To start with, I suggest that you consider the weights represented by constants $k_1, k_2, k_3$.

Let $S_j = k_1 x_j + k_2 x_{j+1} + k_3 x_{j+2}$

Compute $Cov(S_j, S_{j+1}), Cov(S_j,S_{j+2})$ etc. and see how the constants determine those covariances.

 

[gordonc4513]

Yes, I understand what I have to calculate. Actually , what I don't understand is a result from a book I read. The idea is
$X_t$ is a stationary gaussian process. I have m states.
$p(t) = 1 - t/m$ the autocorrelation function.
So, $p(t) = Cov(X_s, X_{s+t}) / Var(X_t)$.
Now, to simulate it, are generated some uniform random values $V_i$ in $[-a,a]$.
Now $X_t = V_1 + V_2 + .. +V_N, X_{t+1} = V_{p+1} + V_{p+2} + .. + V_{N+p}$ and so on.N is enough large to use the central limit theorem.
My real problem is that there is stated that $Cov[X_t , X_{t+k}] = E[(X_t - X_{t+k})^2] = 2No -Cov(X_t,X_{t+k})$, where o is the variance for $V_t$. Next is stated that for $k < N/p, Cov[X_t , X_{t+k}] = E[(X_t - X_{t+k})^2] = 2kp^2$. My problem is that I can't reach to this results, I can't prove them. If you can give me some ideas or links to read about this idea to aproximate, it would be great. Thanks again.

 

[Stephen Tashi]

$Cov[X_t , X_{t+k}] = E[(X_t - X_{t+k})^2] = 2No -Cov(X_t,X_{t+k})$, where o is the variance for $V_t$.

$$E[(X_t - X_{t+k})^2] = E( X_t^2 - 2 X_t X_{t+k} + X_{t+k}^2) = E(X_t^2) - 2 E(X_t X_{t_k}) + E(X_{t+k}^2) = N_o - 2 E(X_t X_{t+k}) + N_o$$

Next is stated that for $k < N/p, Cov[X_t , X_{t+k}] = E[(X_t - X_{t+k})^2] = 2kp^2$.

That could be a mess to write out, but the way I would start it is:

$$COV( X_t, X_{t+k}) = COV( (\sum_A V_i + \sum_B V_i) (\sum_B V_i + \sum_C V_i) )$$

Where $A$ are the $Vi$ unique to $X_t$, $B$ are the $V_i$ common to both $X_t$ and $X_{t+k}$ and $C$ are the $V_i$ unique to $X_{t+k}$.

As I recall, the covariance function obeys a distributive law that would give:

$$= COV( \sum_A V_i ,\sum_B V_i) + COV(\sum_A V_i ,\sum_C V_i) + COV(\sum_B V_i ,\sum_B V_i) + COV(\sum_B V_i ,\sum_C V_i)$$

$$= 0 + 0 + COV(\sum_B V_i, \sum_B V_i) + 0$$

So you must comput the variance of $\sum_B V_i$, which will be a function of the variance of one particular $V_i$ and the number of the $V_i$ in the set $B$.

 

[gordonc4513]

I have one more question. Maybe is obvious, but I don't see it. Why is the relation $Cov(X_t,X_{t+k}) = E[(X_t - X_{t+k})^2]$ true?

 

[Stephen Tashi]

I have one more question. Maybe is obvious, but I don't see it. Why is the relation $Cov(X_t,X_{t+k}) = E[(X_t - X_{t+k})^2]$ true?

I don't have time to think about that now.

Is $X_t$ assumed to have mean = 0 as part of the terminology "gaussian"?

Also, I notice that I got $2 N_0 - 2 E(X_t,X_{t+k})$ instead of what you wrote.

I'll be back this evening.

 

[gordonc4513]

Yes, the mean is 0. Also, from what I calculated, there it should be [itex]kpo[\itex] instead of [itex]kp^2[\itex].

 

[Stephen Tashi]

Also, from what I calculated, there it should be [itex]kpo[\itex] instead of [itex]kp^2[\itex].

The notation is getting confusing. We have p() for the correlation function and p in the index that defines $X_t$ as a sum of the $V_i$. What is "po"?

Anyway, let's say that the cardinality of the set $B$ is b.

$$COV( \sum_B V_i, \sum_B V_i) = VAR( \sum_B V_i)$$

$$=\sum_B ( VAR(V_i)) = b (VAR(V_i))$$

versus:

$$E( (X_t - X_{t+k})^2) = E ( ( \sum_A V_i - \sum_C V_i)^2)$$

$$= E( (\sum_A V_i)^2 + (\sum_C V_i)^2 - 2 \sum_A V_i \sum_C V_i )$$

$$= E( (\sum_A V_i)^2 + E (\sum_C V_i)^2 - 2 E( \sum_A V_i)E(\sum_C V_i)$$

I gather we are assuming $E(V_i) = 0$ so $E( \sum_A V_i)^2 = VAR( \sum_A V_i)$ etc.

So the above is

$$= VAR( \sum_A V_i) + VAR(\sum_C V_i) - 2 (0)(0)$$

Let a = the cardinality of $A$ and c = the cardinality of $C$

$$= (a + c) VAR(V_i)$$

It there an argument that (a+c) = b ?

 

[gordonc4513]

Thank you for the help, I figured it out finally. I have one more question. Now I have the algorithm for simulating the process and I want to validate it. Can you give some hints how it must be done?

 

[Stephen Tashi]

Do you mean you want to validate it as a computer program by running it and looking at the data? Or do you mean you want to validate whether the algorithm implements the procedure in the book by reading the code of the algorithm?

 

[gordonc4513]

I meant if I see the values the computer program generated, what tests could I use to see if it is working correctly? I am thinking to check if the average of the values estimates the mean, the values respect the autocorrelation function rules and I was wondering if there are more things I could test.

 

[Stephen Tashi]

I'd have to think about this in detail to give good advice. On the spur of the moment, I'd say to also check whether the X_i are normally distributed.

In the history of simulations, there are examples where the random number generating functions were flawed in certain software packages. Don't assume that the random number generator in your software really works. Test it or at least find some documentation that someone else did. A quick test is to do a 2D plot of (X,Y,color) where each component is selected by a uniform distribution on some scale. See if any pronounced visual patterns appear. (Some random number generators misbehave only on particular seeds.)