Average of standard deviation makes sense?

Click For Summary

Discussion Overview

The discussion revolves around the concept of averaging standard deviations in the context of test results with varying sample sizes. Participants explore whether calculating the mean of standard deviations can provide insights into the consistency of test results across different groups.

Discussion Character

  • Exploratory
  • Technical explanation
  • Debate/contested

Main Points Raised

  • One participant questions the validity of averaging standard deviations, suggesting it may indicate that test results are similar within each group.
  • Another participant notes that traditional methods for testing the equality of variances are unreliable with small sample sizes, indicating that confidence in conclusions drawn from the data may be limited.
  • Some participants argue that when combining data sets, variances should be used instead of standard deviations, proposing formulas for calculating total and mean standard deviations.
  • There is a suggestion that understanding total standard deviation can be beneficial for process improvement, even if the mean of standard deviations is not particularly useful.
  • A participant expresses uncertainty about whether the audience is familiar with the concepts of variance and standard deviation, implying a need for clarity in the discussion.

Areas of Agreement / Disagreement

Participants express differing views on the appropriateness of averaging standard deviations and whether variances should be used instead. The discussion remains unresolved with multiple competing perspectives presented.

Contextual Notes

Limitations include the small sample sizes discussed, which may affect the reliability of statistical conclusions. There is also a lack of consensus on the best approach to analyze the data.

xeon123
Messages
90
Reaction score
0
I've a set of tests, and for each test, they have different size.

I'll show an example.

Code:
Test1
size - Time(seconds)
100 - 10
100 - 23
100 - 17
200 - 37
200 - 42
200 - 47
300 - 53
300 - 53
300 - 53

For each test, I took the average.
Code:
Average1
size - average
100 - 16
200 - 42
300 - 53

And for each entry of the Test1, I took the standard deviation. Does't in make sense calculate the mean of the standard deviation?

Doing an average of standard deviation can proof that all examples ran at similar time? For example, having low average in the standard deviation means that all the 3 results of 100 were similar among themselves, the 3 results of 200 were similar results among themselves, and the 3 results of 300 were similar among themselves?
 
Physics news on Phys.org
There are a few procedures used to test the hypothesis that the variances of two groups are equal, but they tend to be extremely unreliable (especially for your small N).

The standard deviation has a distribution, of course, and the distribution tends to be pretty normal for large N (which you don't have, so your distribution is going to be quite skewed). You're not going to be able to say much with confidence; the best thing to do may be to just point to the standard deviations of the three groups and note that they're pretty close to each other.
 
I just showed here a small example. I've a set with N=100.
 
When combining data sets you should work with the variances and not the standard deviations.
 
mathman said:
When combining data sets you should work with the variances and not the standard deviations.

Where variance is SD2.

So I presume that SD(total) = √(SD12 + SD22...SDn2)

And SD(mean) = SD(total)/n

While I can't see a good use for SD(mean), being able to understand SD(total) is a powerful tool in process improvement. Often you can't reduce all variation in a process but you can sure tackle some of them individually and reduce the total that way.
 
Bandit127 said:
Where variance is SD2.

So I presume that SD(total) = √(SD12 + SD22...SDn2)

And SD(mean) = SD(total)/n

While I can't see a good use for SD(mean), being able to understand SD(total) is a powerful tool in process improvement. Often you can't reduce all variation in a process but you can sure tackle some of them individually and reduce the total that way.
Do you have a further question?
 
Since you have not taken issue with it, I presume the workings are correct.

Should I take for granted that all the readers of this thread know what variance is and how to sum and take the average of standard deviations? If so my post is redundant.

If not you could have prevented me posting by explaining it yourself.
 

Similar threads

  • · Replies 5 ·
Replies
5
Views
2K
  • · Replies 2 ·
Replies
2
Views
2K
  • · Replies 9 ·
Replies
9
Views
6K
  • · Replies 9 ·
Replies
9
Views
3K
  • · Replies 6 ·
Replies
6
Views
2K
  • · Replies 6 ·
Replies
6
Views
4K
  • · Replies 1 ·
Replies
1
Views
2K
  • · Replies 5 ·
Replies
5
Views
2K
  • · Replies 1 ·
Replies
1
Views
2K
  • · Replies 3 ·
Replies
3
Views
2K