- #1
greatm31
- 1
- 0
Homework Statement
I've been trying to teach myself some Decision Theory using "Pattern Classification" by Duda, Heart, and Stork. It's a great book but the homework problems are giving me a lot of trouble. This one, concerning minimax, is particularly difficult:
"Assume we have one-dimensional Gaussian distributions p(x|\omega_{i})=N(\mu_{i},\sigma_{i}^{2}) for i=1,2, but completely unknown prior probabilities. Use the minimax criterion to find the optimal decision point x* in terms of \mu_{i} and \sigma_{i} under a zero-one risk."
Just to clarify, the zero-one risk means the loss function is 1 for an incorrect decision, and 0 for a correct decision.
2. The attempt at a solution
Now, my understanding of the minimax criterion is that we want to minimize the maximum Bayesian error for all possible priors. So I was thinking that the goal is to maximize the conditional Bayesian risk:
(key:
a_{i} = action associated with selecting i
\omega_{j} = the so-called "state of nature" (tells whether j is true)
x = the data
c = number of states of nature):
R(a_i|x) = \sum_{j}^c cost(a_{i}|\omega_{j}) \cdot P(\omega_{j}|x)
Since the loss function is just 1 if i \neq j and 0 if i=j,
R(a_{i}|x) = \sum_{j \neq i}^c P(\omega_{j}|x)
= 1 - P(\omega_{i}|x)
OK so this much is already developed in the book. I figure all I have to do is maximize the conditional error R(a_{i}|x) and I'll have the "decision point" - the point at which you have the maximal Bayesian Risk, which should (I think) be independent of the prior probabilities P(\omega_{i}). So that's what I did:
(here I'll use the abbreviation N_{i} = N(\mu_{i},\sigma_{i}^2) = normal distribution with mean \mu_{i} and variance \sigma_{i}^{2}):
R(a_{i}|x) = 1 - P(\omega_{i}|x)
= 1 - \frac{p(x|\omega_{i}) \cdot P(\omega_{i})}{p(x)} (Bayes' Rule)
= 1 - \frac{N_{i} \cdot P(\omega_{i})}{N_{i} \cdot P(\omega_{i})+N_{j} \cdot P(\omega_{j})}
= 1 - \frac{1}{1+\frac{P(\omega_{j})}{P(\omega_{i})} \cdot \frac{N_{j}}{N_{i}}}
To minimize this, we maximize N_{j}/N_{i} (thus independent of priors). I maximize this just by taking the first derivative and setting = 0:
0 = \frac{N_{i} \cdot N_{j}' - N_{j} \cdot N_{i}'}{N_{i}^2}
\Rightarrow N_{i} \cdot N_{j}' = N_{j} \cdot N_{i}'
\Rightarrow N_{i} \cdot \frac{-2 \cdot (x-\mu_{j})}{(2\sigma_{j}^{2})^{2}} \cdot N_{j} = N_{j} \cdot \frac{-2 \cdot (x-\mu_{i})}{(2\sigma_{i}^{2})^{2}} \cdot N_{i} (deriv. of normal)
\Rightarrow \frac{x-\mu_{j}}{\sigma_{j}^{4}} = \frac{x-\mu_{i}}{\sigma_{i}^{4}}
\Rightarrow x = \frac{\sigma_{i}^{4} \cdot \mu_{j} - \sigma_{j}^{4} \cdot \mu_{i}}{\sigma_{i}^4 - \sigma_{j}^{4}}
So that's what I think x* is. The trouble is it doesn't make too much sense (what happens when \sigma_{i} = \sigma_{j} ?). And also I'm not totally sure this is the right way of thinking about things.
Thanks for any help you can provide, even if it's just to tell me I'm completely wrong about my approach.
