Time dependent correlation/dependence

[Whenry]

How do I handle the following situation.

I have two time series, A and B. For the entire 100 yrs, we have P(A|B) ≠P(A), and Chi squared test leads us to reject the hypothesis of independence.

But, If I break the data down into 10 yr chucks, I find that for some 10 yr chucks P(A|B) = P(A).

So it seems maybe there is a third variable C, and I should be looking at P(A|B,C).

Similarly,

if I use regression on some data and find a significan relationship, for 100 yrs, of

Y = b_0 + b_1*x_1 + ε

but when I break down into 10 yr chucks, I find some years where we have

Y = b_0 + ε.

So it seems I should be looking at some interaction term :

Y = b_0 + b_1*x_1*α_1

but I have no idea what α_1 is...

Any suggestions as to how to handle this? resources that help explain how to proceed?

thank you,

Will

 

[chiro]

Hey Whenry and welcome to the forums.

What statistical knowledge or experience do you have?

You could put your data into a statistical software package and calculate interaction effects.

From what you have described, you would add another variable which would correspond to your different intervals and then get some output on the interaction effects which you could use provided all the relevant assumptions are met.

How are you analyzing your data currently? What packages (if you are using any) are you using?

 

[Whenry]

I am using MatLab. But I have used R as well and could use that if it would be a good tool.

I have a fairly good amount of numerical experience and dynamical systems experience, but my experience with statistics is not as strong...although I did have a year of grad level...I need practice, and stats vocab practice hah :-)

The variable A is categorical, 0 or 1, and so is B.

The linear model is similar, we are actually using a logistic function that gives us statistically significant b_0 and b_1 when calculated for all time. However, when I look at 10 yr chunks, and run a logistic fit, I will find y = b_0 + b_1*x, but b_1 is not significant and is near 0.

y = b_0 + b_1*x

My thought was, I could almost determine the interaction term numerically. For example, if i took the all time b_1, as the 'correct' or baseline b_1, I could then calculate b_1 for each 10 or 5 or 2 yr chunk, and then calculate my a_1...I could have a hint at least of what the a_1 term might look like.

y = b_0 + b_1*a_1*x1

If there are anythings you would recommend I do in MatLab and R, I would be very thankful

Will

 

[chiro]

If you are using R you could create a new dataset with the new "C" variable that sets the appropriate value for it depending on the time period.

Using this new data-set you can do some standard analysis to gauge interaction effects if any exist.

To me, that seems like the easiest way to do it. I am not familiar with other ways to do this. Maybe you could tell me specifically how you intended (or tried) to gauge the interaction effect numerically.

 

[Whenry]

chiro, thank for your time and advice...

could you explain this a little more; especially in the context of logistic regression.

I have 50 yr time series where b_1 > 0 and statistically sig, p < 0.05

but for years 40 to 50 b_1 < 0 and stat sig.

So if I am trying to add an interaction term like so ; b_1*x_1(t)*C(t).

Now... to gauge interaction effects to assign values to C?

Will

 

[chiro]

chiro, thank for your time and advice...

could you explain this a little more; especially in the context of logistic regression.

I have 50 yr time series where b_1 > 0 and statistically sig, p < 0.05

but for years 40 to 50 b_1 < 0 and stat sig.

So if I am trying to add an interaction term like so ; b_1*x_1(t)*C(t).

Now... to gauge interaction effects to assign values to C?

Will

I'm not exactly sure for the command to use in R for fitting a logistic regression model so maybe you can tell me what the command is for that. If it uses the lm command, then show me the syntax if you could please.

What you will have to do is after you create a new dataset with the right values for your C variable, is to simply modify the model to add your term involving C and the model fitting algorithm should take care of the rest as well as calculate all the interaction effects.

I know WinBUGS also allows you to state very complex models quite easily and let's you specify logistic regression very easily in the format you are talking about, so if you can't use R, you could end up using WinBUGS which is free to download.