Study Design for assessing efficacy of a New Clinical Sign

[CieloB]

Hi!
I am looking for some guidance as to what statistical tests I would need to do to know the significance of a study.

The situation is this:
Disease in Question is Sinusitis
Gold Standard for Diagnosis is CT, but is seldom done as not all cases require confirmation. Cost-effectiveness guidelines dictates using clinical criteria (2 Major or 1 Major & 2 Minor Criteria as defined) to determine if a patient is a case of Sinusitis or not.
Common clinical sign used is Tenderness to Pressure.
To know about the efficacy of a New Clinical Sign, a pilot study is performed to know if the new Clinical Sign is valid and to know the scope of evaluation needed to know its efficacy. 50 cases and controls are selected. Cases are selected based on the clinical criteria definition as above. Controls are healthy volunteers. Cases and Controls are only gender matched.

Tests for both Tenderness to Pressure and New Sign are performed on both controls and cases and data about number testing positive/negative for each test is obtained.
Now, to know about the efficacy of the New Sign what analyses are to be performed?

The sensitivity and specificity of Tenderness to Pressure are known. CT data is not available for the patients. The cases, as mentioned above, were defined based on the criteria, so are not gold standard proven.

Now how exactly is the pilot study result to be interpreted for significance?
McNemar's test was the one that was suggested, but I am not sure if it can be used as the cases are not determined by using the gold standard.
I would be really grateful if someone can point out which statistical test needs to be done to know the significance of the result.
As i understand it, Even for determining sensitivity and specificity, the case needs to be determined by gold standard. So, what are the parameters that can be defined here? Or can it be derived from the known parameters of the Tenderness to Pressure test? If yes, How?

 

[Dale]

Cost-effectiveness guidelines dictates using clinical criteria (2 Major or 1 Major & 2 Minor Criteria as defined) to determine if a patient is a case of Sinusitis or not.

Generally when this is the accepted practice by the community there is a body of literature which provides evidence for the practical equivalence between the gold standard and the standard clinical criteria. You can simply cite that literature and then use the standard clinical criteria as your gold standard.

My favorite analysis approach is received operating characteristics (ROC) analysis. It is closely related to logistic regression. McNemar's test could also work if your new sign is naturally dichotomous, but most of the tests I have developed have been continuous or ordinal rather than binary.

 

[CieloB]

Thank You Sir for clarifying! I really appreciate it!

My understanding of statistics is very rudimentary, and hence the question may seem a bit naive! But here goes!

Is it necessary that only Mcnemar's test be done if the test is dichotomous, or can ROC be done as an alternative?( new diagnostic test is either positive or negative )
I found ROC analysis available in open Epi but Mcnemar's I was unable to locate in openepi and am not sure about which other site can reliably be used.