T-distribution, standard error - how to interpret

[Vital]

Hello.
Math is not my strongest part, yet, so I will be very grateful for your help and for giving the answer in sort of "plain" English, meaning that it will be great if you don't go too deep into the rabbit hole using difficult terminology, as these distribution issues are truly difficult for me

Question:
Below I show formulas, whose meanings I would like to understand, i.e. to understand the logic of each part - why we do this or that computation, the logic, intuition and the the result. I will, of course, show how I read them, thus you will see what I don't understand (hopefully).

To compute the confidence interval using t-distribution we use the following formula:

First the formulas and an example:

X ± t_α/2 x [s / √n]
where X is the mean, s is the sample standard deviation (as we don't know the population standard devitation), n is the population size

δ = s / √n is the formula for the standard error.

For example, find the 95 percent confidence interval:
X = 3%, s = 6%, n = 10
Then, 3% ± t_0.025 x [6% / √10], hence, as t_0.025 = 2.262 (I took that from the table), the interval is from 1.64 to 4.36

Questions:

1) What is the meaning and how to read the standard error part s / √n ?
What are we trying to do here? We take standard deviation and divide it by the square root of the size, and thus we "break" the standard deviation into √n to get the value of one that part. What does it give us? And also why the square root?

2) Upon finding that number, we multiply it by the value of t_0.025 found for the degree of freedom of n - 1, that is 9. What does this mean?

Thank you very much.

 

[Dale]

The standard deviation is a measure of how much the data varies. The standard error is a measure of how much the mean varies.

Suppose you have some random variable, say the weight of knee replacement patients, and suppose that it is normally distributed with mean 70 and SD 10. If you take one random patient, then they are probably (95%) between 50 and 90 kg. You can confirm that by repeatedly randomly selecting one patient and recording their weight.

Now, instead of taking one patient, suppose you took a sample of 4 patients and took the mean. The mean of a sample of four patient weights would itself be a random variable with a mean of 70 and a SD of 5. It turns out that this value is important enough that it is given the name “Standard error”. If you take four random patients then their mean weight is probably between 60 and 80. You could confirm this by repeatedly randomly selecting four patients and recording their mean weight.

The larger your sample, the closer the sample mean will be to the population mean. This is why the SE gets smaller as n gets larger. In this example a sample size of 4 gives an SE of 5, a sample size of 25 gives an SE of 2, and a sample size of 100 gives an SE of 1.

 

[Vital]

Dale, Thank you very much for your answer. It is very helpful as I finally understood the meaning of the SE concept. Thank you. But I still have some issues I would like to clarify. And also what is very important I was asking about how to interpret (read) the math we are performing in those formulas. Your answer helped me to see why we have the square root of n in the Standard Error formula (because the variance of the mean = δ² / n, hence δ / √n), but I still don't understand what this math function means and does. We take the standard deviation of the data (if I correctly understand, we take the population's SD, not the sample's one) and divide it by the square root of the number of observations, thus, if I understand correctly the logic of this, computing how much of observations fit into one standard deviation. Why are we doing this and what does it tell us, mathematically and logically?

This is why the SE gets smaller as n gets larger. In this example a sample size of 4 gives an SE of 5

How did you get to SE = 5? If SD = 5, n = 4, then SE = 5/2 = 2.5. Have I incorrectly understood how to use the formula?

Thank you very much.

 

[Dale]

How did you get to SE = 5? If SD = 5, n = 4, then SE = 5/2 = 2.5. Have I incorrectly understood how to use the formula?

I was continuing the same example so SD=10.

I was asking about how to interpret (read) the math we are performing in those formulas

I am not sure how to answer this. It seems that you already understand that part perfectly. To calculate a confidence interval for the sample mean you take the standard deviation, divide it by the square root of the sample size, multiply by the critical value from the distribution, and then both add and subtract it from the mean. That is it and you already seem to understand that. I will need more guidance on what you feel is missing because from my perspective it seems like you already understand that.

Why are we doing this and what does it tell us, mathematically and logically?

Continuing with my previous example, each time you select a random patient and measure their weight the answer you get will vary. Therefore, the weight is a random variable, and we would like to characterize it, eg with the mean and standard deviation.

Each time we select four random patients and take the mean of their weights then the answer we get will vary. Therefore, the sample mean is also a random variable with its own distribution, which we would also like to characterize. The standard error is the standard deviation of the distribution of the sample mean.

 

[Vital]

I was continuing the same example so SD=10.

I am not sure how to answer this. It seems that you already understand that part perfectly. To calculate a confidence interval for the sample mean you take the standard deviation, divide it by the square root of the sample size, multiply by the critical value from the distribution, and then both add and subtract it from the mean. That is it and you already seem to understand that. I will need more guidance on what you feel is missing because from my perspective it seems like you already understand that.

Upon reading your answer few times, I think I now understand what I was asking about)) I will try to explain myself. I truly didn't understand the mathematical meaning of that computation and the purpose of dividing the population's standard deviation by the square root of the number of observations. But now I see to understand what we are doing here. Please, correct me if I am wrong. By computing the standard error we are finding the standard deviation of the sample mean, as you pointed, and this [s / √n] means that we are finding the mean of the population's standard deviation. Is this correct?

One more question here, if I may, about the value of t, which you named the critical value.
t_0.025 = 2.262 for 95 percent interval. How to interpret this number? How is it computed? I took the value from the table. And what does it mean when we take the t and multiply it by the standard error - if the standard error is the averaged population's standard deviation as I assumed above?

Thank you very much.

 

[Dale]

By computing the standard error we are finding the standard deviation of the sample mean, as you pointed, and this [s / √n] means that we are finding the mean of the population's standard deviation. Is this correct?

Not really. The SD of the mean is not the same as the mean of the SD. And generally a sample mean or SD will not be the same as a population mean or SD.

I think of it more like this. We have some random variable. My example was the weight of a patient, but it could be anything. That random variable has some statistical distribution, with an associated mean and SD. From that random variable we can generate new random variables in a number of ways. For example, if our patient weights, X, are normally distributed with mean 70 and SD 10 then we can compute a new random variable as Y=(X-70)/10. Now, suppose we want to know the distribution of this new random variable, it turns out that we can calculate the distribution of Y from the distribution of X, and it is also normally distributed, but with a mean of 0 and a SD of 1.

Now, we can also generate a new random variable from X as follows: $Z=\Sigma_n X/n$ where each of the terms in the sum is one of n random samples from X. Now, we ask what is the distribution of Z. It turns out that this too has a distribution that can be determined from X, and that is that Z is normally distributed with the same mean as X but with a SD of $1/\sqrt{n}$ times the SD of X.

Note, X and Z are different random variables. They are related to each other, but they are different things.

 

[haushofer]

Just my 2 cents to add to Dale's response: I like the intuitive explanation of Khan academy,

https://www.khanacademy.org/math/ap...ivide-by-n-1-for-the-unbiased-sample-variance

why you divide by n-1 in the sample standard deviation s. Maybe this helps; for me, it was a bit confusing when I first encountered this.

 

[Tosh5457]

It can be proven why that makes for a $$1-\alpha$$ confidence interval for X. This involves:
1) Proving that $$(n-1)s/n$$ is a chi-squared random variable (this is because s is a random variable where normal random variables are squared)
2) Proving $$\sqrt{n}X/s$$ is a t-student random variable (this can be easily proven using a result that relates normal and chi-squared random variables)
3) $$P(-a<\sqrt{n}X/s < a) = 1-\alpha => P(-as/\sqrt{n} < X < as/\sqrt{n}) = 1-\alpha => $$ X ± tα/2 x [s / √n] is an equal-tailed $$1-\alpha$$ confidence interval, since t-student is a symmetric distribution. This is just the basic sketch. This proof, along with many other ones, are usually skipped in basic statistics courses because they would require these courses to teach a lot more to understand the proofs properly.