Paired samples-equality of variance and 95% CI around difference in variances

[wvguy8258]

Hi,

Can a few of you please review the approach I plan to take for obvious errors?

I have 50 subjects and each have a measure taken on the same variable before and after treatment. So, this is standard paired t-test time, but what I am actually interested in is the variance of the treatment versus the control. I would like to test the equality of variance for these two groups of values (treatment and control) and also place a 95% confidence interval around the difference of these two variances. I would prefer randomization/resampling methods to be used for each as normality assumptions do not hold and I would like a robust result. I have not found any routines specifically like what I would want, so I think I may have to do the following in R. Any advice on an easier or better approach is welcome.

I know that equality of variance for paired data can be tested using the pitman-morgan statistic. I was planning on calculating this for the original data and then randomly switching the values within pairs the the pre-treatment and post-treatment measures in order to achieve randomization that respects the paired nature of the data. I could then extract p-values based upon the percent of randomizations with more extreme pitman-morgan statistic.

For the 95% CI interval around the differences, I thought I would resample pairs of values with replacement. So, I would select among the 50 subjects 50 times with replacement. I would then calculate the variance for the pre-treatment measures and for the post-treatment measures and I would then take the difference and store this value. I would do this many times and then determine the 95% confidence interval by ordering my resamples and simply taking the 2.5% and 97.5% percentiles.

Does this make sense at all?

NOTE: Since posting I have been advised elsewhere that a ratio of the variances would be better than the difference.

Thanks,
Seth

 

[mmwave]

Dear Seth,

Thank you for sharing your approach with us. It seems like you have put a lot of thought into your methodology, and I appreciate your desire for a robust result. However, I do have a few suggestions for your approach.

Firstly, I would recommend using the Levene's test for equality of variances instead of the Pitman-Morgan statistic. This test is specifically designed for comparing variances between two groups and does not require normality assumptions. Additionally, it is available in most statistical software packages, including R.

Secondly, instead of randomly switching the values within pairs, I would suggest using a permutation test. This involves randomly shuffling the labels of your treatment and control groups and calculating the test statistic (in this case, the Levene's test) for each permutation. This will give you a distribution of the test statistic under the null hypothesis of equal variances. You can then compare your observed test statistic to this distribution to obtain a p-value.

For the 95% confidence interval, I would recommend using bootstrapping instead of resampling with replacement. Bootstrapping involves randomly sampling from your data with replacement to create new datasets, and then calculating the test statistic for each new dataset. This will give you a distribution of the test statistic, from which you can obtain confidence intervals. Bootstrapping is a more robust method for obtaining confidence intervals and does not rely on assumptions about the underlying distribution of the data.

Finally, I agree with the advice you received about using a ratio of variances instead of the difference. This is a more commonly used approach and has been shown to be more robust in various scenarios.

I hope this helps and I wish you all the best with your analysis.