Reviewing Image Retrieval Performance Using Chi-Square

In summary, the chi-square test suggests that there is a significant difference between the actual "scores" and the predicted "scores," at the 0.01 level of significance.
  • #1
chisqaw
2
0
Hi ll, I have been asked to review a new image retrieval technique that has improved the retrieval performance compared with the old technique, ie, the hypothesis is that retrieval performance is influenced by retrieval techniques. The following table summarises the retrieval results. (They examined the first 300 retrieved images, 250 are relevant to the query for the new technique and 175 are relevant for the old technique)


Image relevant? Type of techniques Total
New technique Old technique
Yes 250 175 425
No 50 125 175
Total 300 300 600


I was using this distribution at http://davidmlane.com/hyperstat/chi_square_table.html but I am having trouble calculating the Chi Square value assuming the required p value is 0.01 and interpreting the result. Would someone be able to do this and explain how it was doing, or point to a decent tutorial? I only need this done, not to understand it, at least not past what is required.

Many thanks,
Lawrence
 
Physics news on Phys.org
  • #3
I don't understand that at all...
 
  • #4
You are evaluating the "null" hypothesis "technique does not matter" against alternative hypo. "technique makes a difference." To evaluate it, you need to calculate a test statistic, then compare it against the chi-square table; where degrees of freedom = 1 and level of significance = 0.01. Unless I made a mistake, the corresponding value on the chi-square table is 6.6349.

If the result of your calculation:

(actual - predicted)^2/predicted, summed over the two categories

turns out to be > 6.635, then the test is telling you that there is a significant difference between the actual "scores" and the predicted "scores," at the 0.01 level of significance. You should reject the hypothesis that "the technique does not make difference" if your calculation is > 6.635.

In your case:

________Category 1___Category 2
actual = ____ 250 ______ 175
predicted = 425/2 ______ 425/2

(I have assumed that only the first row of your table is relevant for this calculation.)

Do you see the similarity between the test-stat. in your case and the example test stat. (comparing men and women) on the Wikipedia page?
 
Last edited:
  • #5
EnumaElish said:
You are evaluating the "null" hypothesis "technique does not matter" against alternative hypo. "technique makes a difference." To evaluate it, you need to calculate a test statistic, then compare it against the chi-square table; where degrees of freedom = 1 and level of significance = 0.01. Unless I made a mistake, the corresponding value on the chi-square table is 6.6349.

If the result of your calculation:

(actual - predicted)^2/predicted, summed over the two categories

turns out to be > 6.635, then the test is telling you that there is a significant difference between the actual "scores" and the predicted "scores," at the 0.01 level of significance. You should reject the hypothesis that "the technique does not make difference" if your calculation is > 6.635.

In your case:

________Category 1___Category 2
actual = ____ 250 ______ 175
predicted = 425/2 ______ 425/2

(I have assumed that only the first row of your table is relevant for this calculation.)

Do you see the similarity between the test-stat. in your case and the example test stat. (comparing men and women) on the Wikipedia page?

But you do not know the exact frequency of the old technique. So I would suggest to tests :
H0 : there is no difference between the two techniques
As you observe a large number of events, you can assume gaussian distribution and use a Pearson chi2 test like :

chi2 = (250 - 175)^2/(250 + 175) = 13.2
 
  • #6
(a - b)^2/(a + b) = [(a - y)^2 + (b - y)^2]/y where y = (a + b)/2, therefore Barmecides's formula will produce an identical result with the one I posted (which is identical to the formula on the Wikipedia page).
 
Last edited:

1. What is "Reviewing Image Retrieval Performance Using Chi-Square"?

"Reviewing Image Retrieval Performance Using Chi-Square" is a scientific study that evaluates the performance of image retrieval systems using the statistical method of chi-square analysis.

2. Why is chi-square analysis used in this study?

Chi-square analysis is used because it allows for the comparison of observed and expected frequencies in a dataset, making it a useful tool for evaluating the performance of image retrieval systems which rely on the frequency of image features for matching.

3. What are the key findings of this study?

The key findings of this study include the effectiveness of chi-square analysis in evaluating image retrieval performance, the importance of feature selection in image retrieval systems, and the impact of different image feature extraction methods on retrieval performance.

4. How does this study contribute to the field of image retrieval?

This study contributes to the field of image retrieval by providing a comprehensive evaluation framework using chi-square analysis, which can be applied to various image retrieval systems and feature extraction methods. It also highlights the need for further research and improvement in feature selection and extraction techniques to enhance retrieval performance.

5. What are some potential limitations of this study?

Some potential limitations of this study may include the use of a specific dataset and image retrieval system, which may not be representative of all systems and datasets. Additionally, the study may not account for all possible factors that could affect image retrieval performance.

Similar threads

  • Biology and Medical
Replies
1
Views
2K
Replies
2
Views
794
  • Special and General Relativity
Replies
8
Views
2K
  • Sci-Fi Writing and World Building
Replies
2
Views
2K
Writing: Read Only Great Lakes Earth
  • Sci-Fi Writing and World Building
Replies
1
Views
3K
  • Math Proof Training and Practice
2
Replies
67
Views
10K
Replies
2
Views
3K
Back
Top