How can I test the difference between two sample means from SAME population?

[adgalo]

Here is the scenario: Let's say that two sets of samples (SET 1 and SET 2) are from the same population (N=1000). SET 1 and SET 2 has the same sample size (n=500). SET 1 is composed of doctors only, and their average (mean) price that they are willing to pay for a certain product (a new drug) is $20. While SET 2 is composed of doctors, nurses, patients, etc, and their average (mean) price that they are willing to pay for a certain drug is $40.

My questions are the following:

1.How can I test if there is a difference between two sample means from SAME population?
2. If I run another survey for NURSES only, with the same sample size (n=500), will I get a sample mean price lower than the sample mean price of SET 2 (which is $40)?
3. Does the sample mean of SET 1, which is just a subset of SET 2 in terms of composition, is always lower than the sample mean of SET 2?

Thank you very much

 

[chiro]

Hey adgalo and welcome to the forums.

For your problem as you have described it, I would argue that these populations are not the same but different in terms of their characteristics.

Also you have to be careful about your terminology and I think it's a good idea I said a few words so that you don't misuse the statistical terminology in the future.

The first thing is that you don't have one population with 1000 data points: you have two. It's like saying that if you sampled the country of 20 million with 10 million men and women, that the men and women reflect the same population.

This is not correct. While they the men and women are indeed subsets of the entire population, their characteristics are not the same and thus for that reason you have two populations. If the characteristics were exactly the same, then yes they would be part of the same but since they are not, they are not. It's important you realize this because making this error could be very costly for you in the future.

If you want to show evidence that these two distributions are from the same underlying population distribution, then you need to use a two sample t-test. If you had more than two groups you use what is called an ANOVA.

What you have to do is show that there is evidence either that the two samples come from the same population, or that there is evidence that they don't come from the same population under some kind of credibility or 'confidence' constraint. You can't just say they come from the same distribution: it doesn't work like that. You have to show enough evidence that they 'may' come from the same distribution because even though you may get evidence, it doesn't mean that they do. If you don't understand this, then you need to in order to understand what statistics is all about.

You may now ask what characteristics have to be the same? The answer depends on your question. You will have variation in some way and the question will determine what you will be actually be comparing and analyzing. As long as you have variation in some sense between variables, you will always have a hypothesis to test: again you can't just say things come from the same distribution if there is variation in the characteristics! You have to show evidence for it! Can't just assume it! I stress this point because it is fundamental to understanding what statistics can and can't do and why we even use it in the first place.

With that said, let's go to the questions.

For 1. You need to treat them as two different populations and then do a two-sampled t-test. There are variations of the test depending on 'equal variances' and whether data is paired (each element at index i in set A is 'linked' to element at same index in set B).

For 2. You have to actually do the experiment to find out! You can also test if these are likely to be from the same population using the kinds of procedures above, but if you have more than two groups I would recommend an ANOVA procedure for testing means.

For 3. Set 1 is not a subset of Set 2. They are independent samples and they refer to different sets of data. You can't just say because doctors are in set 1 and partially in set 2 that they are subsets: it's not true.

There's a lot more to this when you are doing sampling and I'm not going to get into right now because at the moment, I see that you are having a bit of confusion with statistics and how you should think about it.

If you are doing an actual course, I would really talk with your teacher about these issues because these are really important.

If you are a researcher, scientist, engineer or some analyst trying to analyze data then you should talk to a statistician before you do anything else.

If you have specific questions, I will do my best to answer them but again your understanding of statistics, how it works, and how its used needs to be addressed.

 

[adgalo]

Thank you very much chiro. I guess my problem boils down on how do i treat them as separate populations. Lately, I realized that indeed SET 1 and SET 2 are from different populations based on their characteristics. Before I posted my problem here in the forum, I was thinking that ANOVA should be the method that I will use to test the difference between two means, but then my first assumption was that these two means came from the SAME population, and so I decided that ANOVA is not a good idea. Later you mentioned that these sets should be treated as separate populations. I realized that my assumption was wrong.^_^

 

[chiro]

The ANOVA technique is used to generalize the t-test to more than two comparisons. If you are only comparing two groups then the t-test is suitable in its current form.

 

[adgalo]

As what you have said earlier, two-sample t-test can show evidence that two samples are from the same underlying population. What should be my assumptions here? Do I have to assume that my sample sizes (SET 1 and SET 2) are equal and my population variances are unknown?