Need statistics help working with normal distribution

Click For Summary

Discussion Overview

The discussion revolves around solving for the variable Qc in the context of a normal distribution problem, specifically related to determining the range of values from repeated experiments involving normally distributed random variables. The focus is on deriving Qc as a function of other variables (N and C) using the complementary error function (erfc) for large values of N.

Discussion Character

  • Technical explanation
  • Mathematical reasoning
  • Exploratory

Main Points Raised

  • Tim seeks assistance in solving for Qc in equation form, particularly using erfc for large N (>1E+16).
  • One participant rephrases the problem to determine Q such that the probability of the maximum and minimum of iid normal variables falls within a specified range.
  • The left-hand side of the probability expression can be represented as an integral involving the normal cumulative distribution function (CDF) and probability density function (PDF), though it may not have a closed form.
  • Another approach is proposed to solve for Q based on the probability that the minimum and maximum of the normal variables fall within a certain range, leading to a specific formula involving the normal quantile function.
  • Tim acknowledges the proposed solution but questions how to account for multiple experiments and the relationship between Q and Qc, suggesting that Qc should differ from Q.

Areas of Agreement / Disagreement

Participants have not reached a consensus on the relationship between Q and Qc, and the discussion remains unresolved regarding the correct approach to derive Qc from repeated experiments.

Contextual Notes

The discussion includes assumptions about the normal distribution and the behavior of the variables involved, which may not be fully articulated or agreed upon by all participants.

tim8691
Messages
9
Reaction score
0
Hello experts,

Thanks to discussions with Stephen Tashi for getting me this far.

See the problem statement in the attached PDF page 1. I need help solving for Qc in equation form, as a function of the other variables (N and C), preferably using erfc so I can program an accurate algorithm for very large values of N (>1E+16).

Pages 2 and 3 attempt to solve this problem, but only take me so far. Not sure if this is the right approach, of it there's just another step or two needed from what's presented there.

Looking forward if anyone can help me figure this out.

Tim
 
Physics news on Phys.org
Attached pdf?
 
Hmm, not sure why it didn't take. I'll try again.
 

Attachments

To rephrase, the problem would be to determine the Q such that
P[\max(X_1,...,X_N)-\min(X_1,...,X_N) \le 2Q]=C
where the X_i are iid N(0,1). The LHS can be written as
\int_{-\infty}^{\infty}N(F(x+2Q)-F(x))^{N-1}f(x) dx
where F(x) and f(x) are the Normal CDF and PDF respectively, though the integral may not have a closed form.

If instead you solve for the Q such that
P[-Q\le \min(X_1,...,X_N) \le \max(X_1,...,X_N) \le Q] = C
then the left hand side is
P[-Q\le X_1 \le Q]^N = (F(Q)-F(-Q))^N = (2F(Q)-1)^N
The solution is
Q = F^{-1}((1+C^{1/N})/2)
The normal quantile function is implemented in many computer languages, e.g. with C=0.95 and N=1e6 the Excel formula "=NORMSINV((1+0.95^(1/1e6))/2)" returns 5.446768 which agrees with R and MATLAB.

Edit: if you must use erf, use F(x)=(erf(x/\sqrt{2})+1)/2 so
Q=\sqrt{2}erf^{-1}(C^{1/N})
 
Last edited:
Thanks so much bpet, I believe your solution above solves the equation:

(p(Q))N = C

as I've defined on page 2 of the attached document. This Q is the probability of running one experiment of population N and computing the range (e.g. 2Q) of the normally distributed random variable x.

But after that, how do we then account for running that experiment many times and solving for Qc? That is, "If we repeat the above experiment an infinite (or, very large) number of times, and create a histogram from all the values of 2Q measured, how far (e.g. 2Qc) into this new histogram contains C percent of the population?"

I think Qc should differ from Q, is that right?
 

Similar threads

  • · Replies 1 ·
Replies
1
Views
2K
Replies
1
Views
4K
  • · Replies 6 ·
Replies
6
Views
1K
Replies
2
Views
2K
  • · Replies 1 ·
Replies
1
Views
5K
  • · Replies 15 ·
Replies
15
Views
2K
  • · Replies 4 ·
Replies
4
Views
2K
  • · Replies 6 ·
Replies
6
Views
3K
  • · Replies 3 ·
Replies
3
Views
2K
  • · Replies 3 ·
Replies
3
Views
5K