Analyzing Data: Is There a Significant Difference Between Two Screening Tools?

  • Context: MHB 
  • Thread starter Thread starter mermaidrunner
  • Start date Start date
  • Tags Tags
    Data
Click For Summary

Discussion Overview

The discussion revolves around analyzing data from two screening tools used to assess children for a disorder. Participants are exploring the best statistical methods to determine if there is a significant difference in the number of referrals for further testing between the two tools, considering the nature of the data collected (pass or fail rather than numerical scores).

Discussion Character

  • Technical explanation
  • Debate/contested
  • Mathematical reasoning

Main Points Raised

  • One participant suggests using a paired t-test to compare the two tools, assuming the data is continuous.
  • Another participant raises concerns about the appropriateness of the paired t-test given that the screening results are binary (pass or fail), questioning whether a nonparametric approach would be more suitable.
  • A different participant proposes a two-proportion z-test as a potential method for analysis, referencing a specific hypothesis about the proportions of referrals.
  • Another participant points out the correlation between the samples from the same children and suggests using McNemar's test instead, which is designed for paired nominal data.
  • One participant mentions difficulties in setting up McNemar's test in SPSS and seeks suggestions for proper implementation.
  • A later reply indicates that Minitab does not support McNemar's test but offers a macro as a workaround.
  • Another participant expresses uncertainty about the suitability of the two-proportion z-test due to the correlation within the data.

Areas of Agreement / Disagreement

Participants do not reach a consensus on the best statistical method to use, with multiple competing views on whether to use a paired t-test, a two-proportion z-test, or McNemar's test. The discussion remains unresolved regarding the most appropriate analysis given the nature of the data.

Contextual Notes

Participants highlight limitations related to the binary nature of the screening results and the correlation between samples from the same subjects, which may affect the choice of statistical tests.

mermaidrunner
Messages
4
Reaction score
0
I gathered some data on kids who were screened for a disorder using two screening tools. Each child was screened using both tools. I wanted to see if one tool referred more kids for further testing than the other. I need to determine if there is a significant difference between the number of referrals of the two tools. Tool A referred 34 kids and tool B referred 16 kids. Best way to analyze in SPSS?
 
Physics news on Phys.org
If I get you right, for each kid you have two samples, one for each tool. Both numbers are continuous and you want to compare the tools ? If so, paired t test
 
Yankel said:
If I get you right, for each kid you have two samples, one for each tool. Both numbers are continuous and you want to compare the tools ? If so, paired t test

Yes, that is correct. Each kid has a result from each tool. There were 51 total kids tested. My main concern is that these screening tools aren't scored on a numerical scale. It is strictly pass or fail - is that going to have an impact on using the paired t test? Should I be looking at something more nonparametrical?
 
mermaidrunner said:
Yes, that is correct. Each kid has a result from each tool. There were 51 total kids tested. My main concern is that these screening tools aren't scored on a numerical scale. It is strictly pass or fail - is that going to have an impact on using the paired t test? Should I be looking at something more nonparametrical?

Hi mermaidrunner. Welcome to MHB!

That sounds like a two-proportion z-test, pooled for $H_0\colon p_1=p_2$.

See for instance this wiki page.
 
Oh, it ain't numerical...you still have this problem that you get pairs, 2 samples for each kid, meaning you have correlation between these samples.

What about the McNemar's test ?
 
I like Serena said:
Hi mermaidrunner. Welcome to MHB!

That sounds like a two-proportion z-test, pooled for $H_0\colon p_1=p_2$.

See for instance this wiki page.

I had some trouble setting up Mcnemars in spss.. it wasn't looking right . Any suggestions?

I actually went ahead and ran the 2 proportion z-test (in Minitab). Is that going to be effective? Just trying to cover all my bases before submitting.
 
I think that the two proportion Z test is not suitable here, you have correlation within kids.

Here is a youtube clips showing how to do McNemar's test on SPSS.

McNemar Test - SPSS (part 1) - YouTube

Minitab doesn't support it, but you can download a macro from their website which does it.
 
Can anyone spare a few minutes (with SPSS) and look over some analyses I ran related to this? Sorry, I'm out of the stats loop and need these for a project...
 

Similar threads

Graduate Mathematical Connection between Cosmic Expansion and Exponential Growth
  • · Replies 24 ·
Replies
24
Views
2K
Undergrad Observational Studies: What to Believe
  • · Replies 3 ·
Replies
3
Views
2K
Undergrad Statistical analysis of COVID reinfection
  • · Replies 13 ·
Replies
13
Views
2K
Undergrad Simple statistical analysis of a limited data set
  • · Replies 27 ·
Replies
27
Views
3K
AlphaEvolve - The Future of Math?
  • · Replies 5 ·
Replies
5
Views
1K
MHB Help in identifying why these two calculations have the same result?
  • · Replies 1 ·
Replies
1
Views
2K
Undergrad Significant correlation, not significant coefficient
  • · Replies 4 ·
Replies
4
Views
2K
How Do Cryo-EM Images Enhance Our Understanding of the Coronavirus?
  • · Replies 1 ·
Replies
1
Views
2K
Feasibility of nuclear detonation for deep planetary science on Mars
  • · Replies 38 ·
2
Replies
38
Views
6K
Undergrad Participant non-compliance: How to include data in analysis
  • · Replies 3 ·
Replies
3
Views
2K