How to measure if a statistic is accurate?

  • Context: Undergrad 
  • Thread starter Thread starter chingkui
  • Start date Start date
  • Tags Tags
    Measure Statistic
Click For Summary
SUMMARY

This discussion focuses on measuring the accuracy of performance statistics derived from coding techniques applied to 380-bit long messages. The user generates random bit sequences and tests approximately 10^5 to 10^6 samples, seeking methods to assess the accuracy of the resulting statistics. The key conclusion is that one can estimate a confidence interval around any calculated statistic, which is influenced by the sample size of 10^6 independent random variables. Utilizing statistical theories related to confidence intervals is essential for evaluating the accuracy of the performance statistics.

PREREQUISITES
  • Understanding of confidence intervals in statistics
  • Knowledge of random sampling techniques
  • Familiarity with performance metrics in coding
  • Basic principles of statistical inference
NEXT STEPS
  • Research how to calculate confidence intervals for large sample sizes
  • Learn about the Central Limit Theorem and its implications for statistics
  • Explore statistical software tools like R or Python's SciPy for statistical analysis
  • Investigate methods for random sampling in large datasets
USEFUL FOR

Data scientists, statisticians, software engineers, and anyone involved in performance analysis of coding techniques will benefit from this discussion.

chingkui
Messages
178
Reaction score
2
I am studying the performance statistic of some coding techniques on messages of 380 bits long. I studied it by generating bit sequences randomly and test the performance on each sample. But the sample space is so large (2^380) and practically I can only randomly test about 10^5-10^6 samples, how can I know if the statistic I get is accurate enough? Is there any mathematical theory that could be used to estimate how accurate it is?
 
Physics news on Phys.org
Start by assuming you have 106 independent random variables. Although this is a finite number and is much smaller than 2380 (my guess), it is a sizeable sample size by any standard. You should be able to estimate a confidence interval around any statistic you have happened to calculate; the conf. int. is also a function of the sample size.
 

Similar threads

  • · Replies 5 ·
Replies
5
Views
3K
  • · Replies 5 ·
Replies
5
Views
2K
  • · Replies 7 ·
Replies
7
Views
3K
  • · Replies 24 ·
Replies
24
Views
7K
  • · Replies 7 ·
Replies
7
Views
3K
  • · Replies 30 ·
2
Replies
30
Views
5K
  • · Replies 30 ·
2
Replies
30
Views
4K
  • · Replies 9 ·
Replies
9
Views
3K
  • · Replies 7 ·
Replies
7
Views
2K
  • · Replies 3 ·
Replies
3
Views
3K