Experiment to test for causality

[Agent Smith]

TL;DR Summary

An experiment to check for causality

We want to check whether specially treated bitter gourd is effective, marketed as BitterHeal, in lowering blood sugar levels in diabetics. They take a random sample of diabetics and assign them randomly to $3$ groups:
Group A: Are given BitterHeal
Group B: Are given untreated bitter gourd
Group C: Are not given either untreated bitter gourd or BitterHeal

A reference/baseline is collected: Blood sugar level pre-experiment of all participant diabetics in the experiment.
After 3 weeks, blood sugar levels are measured for all $3$ groups.
Response variable: Blood sugar level
Explanatory variable: BitterHeal

The Null Hypothesis $H_o$: There is no (statistically significant) difference between the $3$ groups.
The alternative hypothesis $H_1$: There is a statistically significant difference between the $3$ groups, i.e. blood sugar levels are lowered by BitterHeal.
Ab hinc difference is equivalent to statistically significant difference.

Assume all conditions for a statistical experiment have been met adequately.

Possible outcomes:
1. No difference between B and C
2. Difference between B and C
3. No difference between A and B
4. Difference between A and B

My question is the number of groups being experimented upon. There are $3$ (A, B, C). Couldn't we achieve the same thing by using only $2$ groups (a control group B and a test/treatment group A)? How does having group C help? Are we trying to control for the placebo effect? I think not because both groups A and B receive bitter gourds (and they don't know whether it's treated/untreated gourds).

I know that if outcome $2$ occurs, we can't rule out the placebo effect because group C knows they didn't get any treatment. Is this why we need group C, because if outcome $2$ happens, it (does it?) somehow lowers our confidence in outcome $4$. My best guess is the outcome combination $1$ (does this mean the placebo effect is nonexistent or negligible?) and $4$ is best with respect to inferring causality (that BitterHeal is antidiabetic).

One of the main worries of modern medicine seems to be the placebo effect (a psychosomatic phenomenon instead of a purely somatic/physical one), because then the drug is a dud.

 

[.Scott]

The combination of outcome 1 and 3 would suggest that no medical effectiveness has been detected.
The combination of outcome 1 and 4 would suggest that the Bitterheal "treatment" has a medical effect (hopefully a good one).
The combination of outcome 2 and 3 would suggest that the Bitterheal "treatment" is not important in providing the positive or negative health effect.
The combination of outcome 2 and 4 could suggest many different possibilities.

There's placebo effect here, but that may not be the best way of looking at it.
If taking the bitter gourd alone changes the blood levels (very objective measurements), then those gourds are either medically effective or induce dietary behavior that is effective. Inducing dietary changes might be "placebo", might be "I need to get that taste our of my mouth", or might be "with this much discomfort, I'm not going to jeopardize the effectiveness with bad diet".

 

[Agent Smith]

The combination of outcome 1 and 3 would suggest that no medical effectiveness has been detected.
The combination of outcome 1 and 4 would suggest that the Bitterheal "treatment" has a medical effect (hopefully a good one).
The combination of outcome 2 and 3 would suggest that the Bitterheal "treatment" is not important in providing the positive or negative health effect.
The combination of outcome 2 and 4 could suggest many different possibilities.

Gracias for the elucidation.

So, to answer your question directly, three groups allows the study to determine whether Bitterheal is effective and whether the researchers have the recipe that they can use to reproduce it.

What's not clear is the necessity for $3$ groups. Like I mentioned in the OP, we have a control group (untreated bitter gourd) and a test/treatment group (BitterHeal) that already cancels the placebo effect (both groups A and B don't know whether they're getting treatment or a placebo pill). Isn't this how you negate the placebo effect?

I can also say with an uncomfortably high degree of uncertainty that what we're attempting here is to recommend BitterHeal to diabetics (if it's shown to work, outcome combination $1$ and $4$) and for that we'd have to demonstrate:
X. Taking BitterHeal is better than not taking anything at all
Y. Taking BitterHeal is better than taking a normal bitter gourd

Yes?

If yes than $4$ goes with Y, What goes with X?

 

[.Scott]

Consider these two scenarios (high score being healthy):
BH scores 123, BG scores 20, and nothing scores 50.
BH scores 40, BG scores 20, and nothing scores 50.

Before recommending BH, you:
1) would want BH to significantly better than either of the other 2; and
2) might have a problem if the main ingredient (BG) was harmful.

 

[FactChecker]

How does having group C help? Are we trying to control for the placebo effect?

Not just the placebo effect. They must rule out that an untreated bitter gourd has a beneficial effect, whether imaginary or real.

I know that if outcome $2$ occurs, we can't rule out the placebo effect because group C knows they didn't get any treatment.

How do they know that? Do they know the nature of the test? That would seriously weaken the results. The less they know, the better.

 

[Agent Smith]

Not just the placebo effect. They must rule out that an untreated bitter gourd has a beneficial effect, whether imaginary or real.

Group B doesn't know anything except that they're being given bitter gourd. Just that fact - being given something - could trigger the placebo effect. We can compare group B to C (hasn't been given anything). Either there's a difference ($2$) or there's no difference ($1$)

If there's no difference ($1$) then we can say that untreated bitter gourd alone has no antidiabetic effect.

If there's a difference ($2$), it could mean untreated bitter gourd has an antidiabetic effect and/or the placebo effect. How would we distinguish between these two possibilities?

------------------------------------------------------------------------------------------------------------------------

If there's a difference between group A and group B ($4$) and no difference between group B and C ($1$), we can conclude BitterHeal is antidiabetic but normal. untreated bitter gourd isn't. The fact that there's no difference between group B and C ($1$) means we can also rule out the placebo effect?

 

[Agent Smith]

Consider these two scenarios (high score being healthy):
BH scores 123, BG scores 20, and nothing scores 50.

For the above score outcomes, I would recommend in order of efficacy:
1. BH
2. Nothing
3. BG

Let statistically significant S(x) = the blood sugar lowering potency of x

Scores: S(BitterHeal) = 100, S(untreated bitter gourd) = 10, S(nothing) = 10
I've established causality?

 

[gleem]

All three groups must not know what they have been given in addtion the researchers must also not kown who received what, so as reduce bias in their analysis. What is the criteria for causation? Was this a true double blind study? If not then all bets are off.

 

[Agent Smith]

Let's assume that it's impossible to conduct a completely blind experiment. Group C won't be given anything after all.

 

[gleem]

Group C won't be given anything

This escaped me but Group C must also be diabetic and they would also be on some medication. What do you want to find out?

 

[WWGD]

Let's assume that it's impossible to conduct a completely blind experiment. Group C won't be given anything after all.

How is that blinding? Blinding is if neither researchers nor patients know who's given what. You have triple blinding if data analysts don't know either. As I understand, causation is determined by correlation plus controlling for possible lurking variable.

 

[Agent Smith]

How is that blinding? Blinding is if neither researchers nor patients know who's given what. You have triple blinding if data analysts don't know either. As I understand, causation is determined by correlation plus controlling for possible lurking variable.

I did admit that it was impossible to blind all the subjects. Group C doesn't get anything and hence they would know, right? I'm trying to figure out why we need $3$ groups. As I said in my previous post, somewhere in here, that just groups A and B seem adequate (most clinical trials I've heard of consist of the test drug and a placebo, $2$ groups). Why would we need group C?

 

[Agent Smith]

This escaped me but Group C must also be diabetic and they would also be on some medication. What do you want to find out?

I want to find out of BitterHeal is an antidiabetic.

 

[gleem]

I want to find out of BitterHeal is an antidiabetic

You, want to find out. Are you planning a study or are you trying to interpret a study?

 

[Agent Smith]

@gleem , just a spinoff of a question in a course I took.

 

[gleem]

What was the course and could you state the question?

 

[pbuk]

TL;DR Summary: An experiment to check for causality

There is no statistical test for causality; statistics can only show correlation.

Group C: Are not given either untreated bitter gourd or BitterHeal

As has been mentioned, Group C should be given a placebo.

Ab hinc difference

"Ab hinc difference" is not a term used in statistics. Like most of the non-English terms you use in your posts this is semantic noise and in order to communicate more effectively you should stop doing it.

Assume all conditions for a statistical experiment have been met adequately.

Possible outcomes:
1. No difference between B and C
2. Difference between B and C
3. No difference between A and B
4. Difference between A and B

These are not possible 'outcomes' these are possible (partial) results. 'Outcome' has a specific meaning in statistics: in this experiment an example of an outcome would be 'blood sugar levels are reduced by 10% in patient A'.

$4$ is best with respect to inferring causality (that BitterHeal is antidiabetic).

No, again you are using the wrong term, 'BitterHeal is antidiabetic' is a proposition not an inference (and again causality is irrelevent).

The propositions that you should be looking at here are:

1. BitterHeal does/does not reduce blood sugar compared to a placebo.
2. BitterHeal does/does not reduce blood sugar compared to an untreated gourd.

The propositions you have stated:

X. Taking BitterHeal is better than not taking anything at all
Y. Taking BitterHeal is better than taking a normal bitter gourd

are not appropriate because other things need to be considered before saying something is 'better': what if taking BitterHeal reduces blood sugar on average by 10% but increases the risk of stroke by 20%?

 

[Agent Smith]

As has been mentioned, Group C should be given a placebo.

What exactly does that mean? If x were the placebo, what would x look like?

The propositions that you should be looking at here are:

1. BitterHeal does/does not reduce blood sugar compared to a placebo.
2. BitterHeal does/does not reduce blood sugar compared to an untreated gourd.

I agree, but these will manifest in the experimental study as statistically significant/insignificant differences in the blood sugar levels in the 3 groups, which I've listed as possible results.

Nevertheless, gracias for the terminology lesson. I appreciate it.

The question I'd like an answer to is why the need for $3$ groups? As far as I can tell, with groups A and B, we've taken adequate countermeasures (group B) against the placebo effect (neither of the groups know whether they're being given BitterHeal or just plain gourd, whether they're in the treatment group or the control group).

 

[pbuk]

What exactly does that mean?

https://en.wikipedia.org/wiki/Placebo-controlled_study

If x were the placebo, what would x look like?

Exactly the same as the drugs being trialled.

The question I'd like an answer to is why the need for $3$ groups?

To account for the possibility that blood sugar is lowered simply because patients are taking part in a trial. This effect can be psychosomatic but it can also be due to a change in behavior e.g. diet.

 

[Agent Smith]

@pbuk most drug trials consist of only $2$ groups: the treatment group and the control group. This is true for most I believe and I take that to mean the conditions for a good trial are fulfilled. Why then the need for a third group?

To account for the possibility that blood sugar is lowered simply because patients are taking part in a trial.

This difference is not asymmetrical: both the control and treatment groups are same in this regard. In short we don't have to worry about it being a confounding factor.

 

[Agent Smith]

There is no statistical test for causality; statistics can only show correlation.

Correlation may be inferred from observational studies. An experimental study can be used to establish causality. I mean when drug trials are conducted, we're trying to prove that a particular drug X causes a desired effect, no?

 

[pbuk]

@pbuk most drug trials consist of only $2$ groups: the treatment group and the control group. This is true for most I believe and I take that to mean the conditions for a good trial are fulfilled. Why then the need for a third group?

The minimum for a trial is one treatment group and one control group. Because we don't know the effectiveness of the untreated gourd it cannot act as a control group so we need a third, placebo, group.

This difference is not asymmetrical: both the control and treatment groups are same in this regard.

I don't understand what this means.

In short we don't have to worry about it being a confounding factor.

What is 'it' here? Whatever 'it' is, 'confounding factor' has a specific meaning in statistics which doesn't fit the way you are using it here.

Correlation may be inferred from observational studies.

Yes: the study you describe is an observational study.

An experimental study can be used to establish causality.

Yes: can you describe what an experimental study might look like in this example?

I mean when drug trials are conducted, we're trying to prove that a particular drug X causes a desired effect, no?

No, for causality you also need to demonstrate the mechanism.

 

[gleem]

No, for causality you also need to demonstrate the mechanism.

What if you are trying to determine the proximate cause of something?

 

[pbuk]

What if you are trying to determine the proximate cause of something?

Then I don't know how a placebo controlled trial (which is what this thread is about, or rather it is about a specific hypothetical placebo controlled trial in some course) would be relevant.

 

[gleem]

Then I don't know how a placebo controlled trial (which is what this thread is about, or rather it is about a specific hypothetical placebo controlled trial in some course) would be relevant.

One still has to rule out statistical variations.

 

[Agent Smith]

The minimum for a trial is one treatment group and one control group. Because we don't know the effectiveness of the untreated gourd it cannot act as a control group so we need a third, placebo, group.

Yes: the study you describe is an observational study.

No, for causality you also need to demonstrate the mechanism.

It's all very confusing.

 

[Agent Smith]

The minimum for a trial is one treatment group and one control group

The reason for this being ... to rule out the placebo effect?

For that, in my example: the normal bitter gourd group (group B).

If it were the case that there's a placebo effect at play, there would be no difference between groups A and B, no? If there is a difference then we can safely infer an actual positive antidiabetic effect for BitterHeal, no? We want to check if BitterHeal has an antidiabetic effect but there are the following possibilities:
1. The placebo effect
2. Normal/untreated bitter gourd too being antidiabetic

To rule out 1, group C has to be given a placebo (a normal bitter gourd). To rule out 2, we have group B. But now group B and C are identical.

 

[Agent Smith]

How about listing out the possibilities:
1. Difference(A, B) = True
2. Difference(A, B) = False

3. Difference(A, C) = True
4. Difference(A, C) = False

5. Difference(B, C) = True
6. Difference(B, C) = False

1 and 6 seems the best result for BitterHeal : BitterHeal is better than normal/untreated gourd and there's no evidence of a placebo effect in action.

2, 4, 6 is the worst-case-scenario for BitterHeal. Taking BitterHeal or taking normal bitter gourd or taking nothing at all makes no difference to a diabetic.

2 and 5 is also a bad situation for BitterHeal because there's a placebo effect.

@pbuk @others

 

[gleem]

If C knows it did not receive an antidiabetic substance there will be no placebo effect in that group. Bitter gourd is believed to have some antidiabetic effect so it might happen that in this study the effect will not be observed. Think of it as a current treatment. BitterHeal is believed to be better. You compare Groups A and B to determine the magnitude of the difference. Group C may provide some validation for BitterHeal if B is found to be ineffective.

 

[Agent Smith]

If C knows it did not receive an antidiabetic substance

Yes, C knows. It seems that when you market a drug (call it X), your claim is to say "Diabetics, drug X lowers blood sugar levels" To do that we need to show a statistically significant difference between those who take drug X and those who don't. The sensible thing to do would be to conduct a clinical trial on randomly selected volunteers randomly assigned to 2 groups (treatment/test group and control group, the former receives treatment while the latter does not). The problem is there's the placebo effect. The usual procedure is to give the control group a placebo. I suppose this is what blinding is. As is apparent, we need only $2$ groups to conduct a clinical trial. Where is the necessity for a $3$rd group, like the one in my question?

Does my question consist of $2$ trials (not $1$), with one trial to check for the placebo effect and the other trial to check for BitterHeal's antidiabetic effect?

 

[gleem]

AFAIK the placebo effect does not occur in studies involving physical changes in body chemistry like serum cholesterol, and blood sugar. It is significant in studies of the subjective effect of a medication as in pain relief or mood change.

When comparing two medications for effectiveness you compare it to a (gold)standard commonly used for the treatment. The problem as I see it with this study is that we are comparing two medications of marginal value.

I assume this problem was from a statistics course, but did they fully realize the issues with this scenario? The question is what was the intent of the problem? What did they want you to learn from it?

I think you need group C to establish the effectiveness of B as well as the effectiveness of A in lowering blood sugar. Next, you can compare A and B to determine their relative effectiveness.