How to take into consideration errors in the Mann-Whitney test?

[Leonid92]

I have 2 independent samples (groups), each group contains 10 experimental values of variable T and their standard deviations (errors). I need to do intergroup statistical analysis, i.e. to elucidate whether there is significant difference between 2 groups in terms of variable T. I chose non-parametric Mann-Whitney test for doing such analysis. The question is: should I take into consideration standard deviations of my experimental data? If yes, how can I take into consideration the standard deviations in Mann-Whitney test?

 

[gleem]

I do not know a great deal about statistics but isn't the Mann-Whitney test an rank test and as such does not use experimental values or standard deviations?

 

[Leonid92]

I do not know a great deal about statistics but isn't the Mann-Whitney test an rank test and as such does not use experimental values or standard deviations?

Mann-Whitney test definitely uses experimental values, you can see the following source: https://www.statsdirect.co.uk/help/nonparametric_methods/mann_whitney.htm

 

[gleem]

Yes the experimental values are used but to determine the rank of the measurements which are then used in determining the Whitney-Manning U statistic. The standard deviations do not enter the calculation. The ranking of the data must be carried out in the StatsDirect software.

If you are not familiar with the workings of this statistic it might be valuable to make up some data and work out a simple situation by hand to see how the calculation is performed.

Also you may want to look at the original paper by Mann and Whitney.

 

[Leonid92]

Yes the experimental values are used but to determine the rank of the measurements which are then used in determining the Whitney-Manning U statistic. The standard deviations do not enter the calculation. The ranking of the data must be carried out in the StatsDirect software.

If you are not familiar with the workings of this statistic it might be valuable to make up some data and work out a simple situation by hand to see how the calculation is performed.

Also you may want to look at the original paper by Mann and Whitney.

Thank you!

 

[Ray Vickson]

I do not know a great deal about statistics but isn't the Mann-Whitney test an rank test and as such does not use experimental values or standard deviations?

The numerical values of the data don't matter---only their ranks are used. So, if the standard deviations are all "small" they likely will not give much of a chance of a change of rank. However, if they are large enough, they could lead to a credible possibility of a change of rank, and that could lead to some tricky and interesting issues.

 

[gleem]

Yes. So you would not want to use a rank test on any data whose uncertainty would change the ranks.

 

[gleem]

Leonid,

When I asked you about the nature of your data in https://www.physicsforums.com/threa...o-experimental-data-good.966073/#post-6140610 I wasn't paying attention and I thought I was in this thread. sorry. Are you talking about the same data in this thread? If so can you tell me how the variable T is related to the spectra ? Does @Ray Vickson's comment above help you how to consider the uncertainties in the Mann-Whitney test?

 

[WWGD]

Maybe you can do separately tests for equality of means ( t-test) and variance ( F-test)? Are you interested in any soecific parameters from your population?

 

[Leonid92]

Leonid,

When I asked you about the nature of your data in https://www.physicsforums.com/threa...o-experimental-data-good.966073/#post-6140610 I wasn't paying attention and I thought I was in this thread. sorry. Are you talking about the same data in this thread? If so can you tell me how the variable T is related to the spectra ? Does @Ray Vickson's comment above help you how to consider the uncertainties in the Mann-Whitney test?

In this thread, I'm talking about magnetic resonance data as well, but in this case it is T value which is retrieved from exponential curve. Yet I don't unerstand, how standard deviations should be considered in Mann-Whitney test if we assume that standard deviations are not small. What formula should I use to take into account standard deviations when performing Mann-Whitney test? I didn't find any formula for this.

 

[Leonid92]

Maybe you can do separately tests for equality of means ( t-test) and variance ( F-test)? Are you interested in any soecific parameters from your population?

Thank you! I will look at t-test and F-test

 

[WWGD]

In this thread, I'm talking about magnetic resonance data as well, but in this case it is T value which is retrieved from exponential curve. Yet I don't unerstand, how standard deviations should be considered in Mann-Whitney test if we assume that standard deviations are not small. What formula should I use to take into account standard deviations when performing Mann-Whitney test? I didn't find any formula for this.

But please remember, as someone else stated, that the Mann-Whitney only makes use of the ranks; by design it is non-parametric, i.e., it makes no assumptions about population parameters and their respective properties.

 

[gleem]

What formula should I use to take into account standard deviations when performing Mann-Whitney test? I didn't find any formula for this.

Being a non parametric you do not need to determine any other information from the data as the sample mean. Let me give an illustration of ranking for a simple example using the Wilcoxon rank sum test which is very similar to the Mann Whitney test. You have two situations which you wish to compare for statistically significant difference in results , A and B. You repeat the experiment seven times.

The largest difference (disregarding sign) is given the rank of the number of experiments i.e. 7 and the smallest difference (disregarding sign) is ranked one. Sum the negative valued numbers and compare to the sum of the positive numbers. Take the lower number of the two and go to the Table of the the statistic for that test. The table will have been computed for the number of pairs and for the confidence level. If the number is less than the number associated with a given confidence level for the given number pairs then the difference data is statistically significant at that level.

In the example the sum of the positive numbers is 8 and the sum of the negative ranks is 20 from the appropriate table the expected value at the 5% level for seven pairs is 2. Since the value above is 8 the data for the two situations is not significantly different at the 5% level. You might have suspected it anyway.by just looking at the data.

The rank sum tests are often used in comparing data that is more qualitative. For example compare two different implementations of a service using a questionnaire which uses the questions do you; strongly agree, agree, somewhat agree, neutral, somewhat, disagree, disagree, and strongly disagree. You would assign numerical values for these answers from 1 to 8.

Advantage of the rank sum tests is that the population distribution of the data does not need to be normal and the test is not sensitive to outliers.

As far as the uncertainties are concerned If they are larger than the separations of the measurements on average you can see a problems for the ranks and therefore the sums can be significantly different if you where to repeat the series of experiments. Your only job would be to assure this is not the case and forget about them in this test.

 

[Leonid92]

Being a non parametric you do not need to determine any other information from the data as the sample mean. Let me give an illustration of ranking for a simple example using the Wilcoxon rank sum test which is very similar to the Mann Whitney test. You have two situations which you wish to compare for statistically significant difference in results , A and B. You repeat the experiment seven times.
View attachment 239609The largest difference (disregarding sign) is given the rank of the number of experiments i.e. 7 and the smallest difference (disregarding sign) is ranked one. Sum the negative valued numbers and compare to the sum of the positive numbers. Take the lower number of the two and go to the Table of the the statistic for that test. The table will have been computed for the number of pairs and for the confidence level. If the number is less than the number associated with a given confidence level for the given number pairs then the difference data is statistically significant at that level.

In the example the sum of the positive numbers is 8 and the sum of the negative ranks is 20 from the appropriate table the expected value at the 5% level for seven pairs is 2. Since the value above is 8 the data for the two situations is not significantly different at the 5% level. You might have suspected it anyway.by just looking at the data.

The rank sum tests are often used in comparing data that is more qualitative. For example compare two different implementations of a service using a questionnaire which uses the questions do you; strongly agree, agree, somewhat agree, neutral, somewhat, disagree, disagree, and strongly disagree. You would assign numerical values for these answers from 1 to 8.

Advantage of the rank sum tests is that the population distribution of the data does not need to be normal and the test is not sensitive to outliers.

As far as the uncertainties are concerned If they are larger than the separations of the measurements on average you can see a problems for the ranks and therefore the sums can be significantly different if you where to repeat the series of experiments. Your only job would be to assure this is not the case and forget about them in this test.

Thanks a lot for this explanation!