Finding Class Posterior Probabilities from Linear Discriminant Function

[sensitive]

Hi

I am doing this exercise (2 class problem with 2-dimensional features) and I have solved the linear discriminant function which turns out be y1(x) - y2(x) = 2x1 +2x2

I am having difficulty in finding the class posterior probabilities frm the linear discriminant function obtained.

But I tried this way and got stuck.

We know that yi(x) = ln{p(x|ci)p(Ci)}

from the linear discriminant function obtained

y1(x) = ln{p(x|C1)p(C1)} = 2x1 + 2x2 + y2(x)
ln{p(x|C1)p(C1)} = 2x1 + 2x2 + ln{p(x|C2)p(C2)}
ln[{p(x|C1)p(C1)} - {p(x|C2)p(C2)} = 2x1 + 2x2

Using exponential for both side we get
p(x|C1)p(C1)/p(x|C2)p(C2) = exp(2x1 + 2x2)

p(C1) and p(C2) are constatnt so we can neglect us giving the following

p(x|C1)/p(x|C2)= exp(2x1 + 2x2)

From this point I am not sure how to separate both posterior probabilities.


please help...Thank you

 

[EnumaElish]

As long as p(x|C1)/p(x|C2)= exp(2x1 + 2x2) is the only condition that needs to hold, p(x|C1) = exp(2x1), p(x|C2)= exp(-2x2) would satisfy it.

 

[tacman]

for sharing your progress and question with us. It seems like you have made some good progress so far. To find the class posterior probabilities from the linear discriminant function, you can use Bayes' theorem which states:

p(ci|x) = p(x|ci)p(ci)/p(x)

Where p(ci|x) is the posterior probability of class ci given the feature vector x, p(x|ci) is the likelihood of x given class ci, p(ci) is the prior probability of class ci, and p(x) is the evidence or marginal probability of x.

In your case, you have already calculated the linear discriminant function y1(x) - y2(x) = 2x1 + 2x2. Now, you can use this function to calculate the likelihoods of both classes by substituting the values of x1 and x2. For example, for class C1, the likelihood would be p(x|C1) = exp(2x1 + 2x2).

Next, you need to calculate the prior probabilities for both classes. Since this is a two-class problem, the prior probabilities would be equal, i.e. p(C1) = p(C2) = 0.5.

Finally, to calculate the evidence or marginal probability of x, you can use the law of total probability:

p(x) = p(x|C1)p(C1) + p(x|C2)p(C2)

Now, you have all the components to calculate the class posterior probabilities using Bayes' theorem. For example, the posterior probability of class C1 given the feature vector x would be:

p(C1|x) = p(x|C1)p(C1)/p(x) = [exp(2x1 + 2x2) * 0.5]/[p(x|C1)*0.5 + p(x|C2)*0.5]

Similarly, you can calculate the posterior probability of class C2 given x.

I hope this helps you in finding the class posterior probabilities from the linear discriminant function. Keep up the good work!