Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Can everybody suggest a better upper bound?

  1. Jul 15, 2011 #1
    I have sent this question a couple of days ago, but it seems that its latex form had problem. So, I decide to send it again.

    I will thank If somebody help me solving this problem.

    Consider a random variable [itex] k_1 [/itex] with the given pmf as:

    [itex]Pr[k_1=l]=\sum_{l_1+2l_2=l} \frac{N!}{(N-l_1-l_2)!l_1!l_2!}p_1^{l_1} p_2^{l_2} (1-(p_1+p_2))^{N-l_1-l_2}[/itex]

    where [itex]l_1,l_2 \in [0,1,...,l] [/itex].

    but we don't have [itex]p_1[/itex] and [itex]p_2[/itex] separately and we know just the value of [itex]p_1+p_2[/itex].

    I want to find at least a good and tight upper bound for the above pmf.

    For example; we can use the inequality of [itex] p_1^{l_1} p_2^{l_2} \leq \frac{l_1!l_2!}{(l_1+l_2)!}(p_1+p_2)^{l_1+l_2} [/itex], but it is not that much tight.

    Can everybody suggest a better upper bound?
  2. jcsd
  3. Jul 15, 2011 #2

    Stephen Tashi

    User Avatar
    Science Advisor

    Do you get a sensible answer if you set [itex] \theta = p_1 + p_2 [/itex] and solve the maximization problem:

    Maximize [itex] f(x) = x^{L_1}(1-x)^{L_2} [/itex] with respect to [itex]x [/itex]
    subject to the constraint [itex] 0 < x < \theta [/itex]

    by using calculus?
  4. Jul 15, 2011 #3
    Set [tex]\theta=1, l_1 = 1, l_2=1[/tex]

    and maximize f(x) wrt x.
    Last edited: Jul 15, 2011
  5. Jul 16, 2011 #4
    If I reword the problem,
    find the supreme (or at least one good) function[itex]f(.)[/itex] which for it we have [itex]p_1^{l_1}p_2^{l_2}\leq f(p_1+p_2,l_1,l_2)[/itex]
    Last edited: Jul 16, 2011
  6. Jul 16, 2011 #5
    The function is maximal when [tex] p_1=0.5, p_2=0.5, l_1=1, l_2=1[/tex]

    That is, the left side of your expression cannot exceed 0.25 if the exponents are from the set of natural numbers (as you defined them) and the sum of the two probabilities are constrained to be less than or equal to one.

    EDIT: Stephen Tashi already gave you the function to maximize in post 2, which you quoted above. I added some constraints so that the answer would make sense in terms of probabilities.
    Last edited: Jul 16, 2011
  7. Jul 17, 2011 #6

    Dear SW VandeCarr,
    let me explain the problem in other view and also my answer.

    Let [itex]p_1, p_2[/itex] be any two probability values with the constraint [itex]p_1+p_2=k\leq 1[/itex] where [itex]k[/itex] is constant. Also, suppose [itex]l_1, l_2 \in N[/itex] are two fixed numbers.
    Then, by maximization, we have: [itex]p_1^{l_1}p_2^{l_2}\leq (\frac{kl_1}{l_1+l_2})^{l_1}(\frac{kl_2}{l_1+l_2})^{l_2}.[/itex]
    Now, it seems that finding a better upper bound which is in terms of [itex]p_1+p_2[/itex] is impossible. Is this conclusion right?
  8. Jul 17, 2011 #7
    Clearly, on the left side of the inequality [tex]p_1^{l_1} p_2^{l_2}[/tex] for any positive integer values of the exponents greater than 1, the value of the expression decreases. If one or both of the exponents take a zero value then the left side of the expression becomes 0.5 or 1 which, while true, is trivial IMO in terms of maximizing the function;

    [tex] f(x)=(x)^{l_1}(1-x)^{l_2}[/tex] [tex]0\leq x \leq 1 [/tex]

    Note for the right side of your inequality, for values of k less than or equal to 1, the expression has a maximum value of 0.25 when the [itex]l[/itex] terms are equal to 1.

    EDIT: I'm not sure you what you mean about the upper bound for [itex]p_1 + p_2[/itex]. Assuming they refer to the same probability space, the upper bound is of course 1. That's defined by the laws of probability.
    Last edited: Jul 17, 2011
  9. Jul 17, 2011 #8
    Are you agree with the given upper bound for [itex]l_1, l_2 \in N [/itex] and [itex]k \leq 1[/itex]?
    I have to find the upper bound "in terms of [itex]p_1 + p_2=k"[/itex] not "for" [itex]p_1 + p_2=k[/itex]
    Also, about the probability space, except [itex]p_1 + p_2[/itex], we also have [itex]p_0=1-p_1-p_2=1-k[/itex].
    Last edited: Jul 17, 2011
  10. Jul 17, 2011 #9
    Even when we find the upper bound of the [itex]p_1^{l_1}p_2^{l_2}[/itex] as [itex](\frac{kl_1}{l_1+l_2})^{l_1}(\frac{kl_2}{l_1+l_2}) ^{l_2}[/itex] with the constraint of [itex]p_1+p_2=k \leq 1,[/itex] but the main problem has not solved.
    If I rephrase the main problem again, it is as follows;

    [tex]f=max_{0\leq x \leq k} \sum_{l_1,l_2 \in A} \frac{N!}{(N-l_1-l_2)!l_1!l_2!} x^{l_1}(k-x)^{l_2}(1-k)^{N-l_1-l_2}[/tex]
    where for the sake of simplicity, [itex]A[/itex] is the set of which the integer values of [itex]l_1, l_2[/itex] are picked up.
    If we want to maximize the above expression in greedy way, we can write
    [tex]f<\sum_{l_1,l_2 \in A} \frac{N!}{(N-l_1-l_2)!l_1!l_2!} (\frac{k l_1}{l_1+l_2})^{l_1}(\frac{k l_2}{l_1+l_2})^{l_2}(1-k)^{N-l_1-l_2}.[/tex]
    But, it is clear that this bound is not that much acceptable.
    Can everybody suggest a better upper bound?(the main question of this thread!)
    Last edited: Jul 17, 2011
  11. Jul 18, 2011 #10
    Could you show how the pmf was derived, so maybe it is possible to derive bounds or an exact formula by another method?
  12. Jul 20, 2011 #11
    We can say, it is a kind of tri-nomial distribution with a constraint of [itex]p_1+p_2=k[/itex] where [itex]0<k<1[/itex]. And [itex]Pr[k_1=l][/itex] is the summation of the above pmf over the constraint of [itex]l_1+2l_2=l[/itex]. If you sum up over all the values of [itex]l[/itex], you can easily get unity, since it is the summation of a tri-nomial distribution.
    Actually, the set [itex]A[/itex] is defined as: [itex]A=\{l_1,l_2 \in \{0,1,...,N\}|l_1+2l_2=l \}[/itex] and [itex]l \in \{0,1,...,2N \}[/itex].
    I really need to improve the given upper bound which is derived in greedy way.
  13. Jul 20, 2011 #12
    Ok so [itex]L=X_1+...+X_N[/itex] where [itex]P[X=1]=p_1[/itex] and [itex]P[X=2]=p_2[/itex], in which case you'd also need [itex]l_1+l_2\le N[/itex] right?
  14. Jul 20, 2011 #13
    yes, you are right. I had considered it in my mind, but, I forgot to mention it. So, we can modify the set as [itex]A=\{l_1,l_2 \in \{0,1,...,N\}|l_1+2l_2=l, l_1+l_2\leq N \}[/itex]
    Last edited: Jul 20, 2011
  15. Jul 20, 2011 #14
    Write [itex]L_N=X_1+...+X_N[/itex], so the p.g.f. with [itex]p_1+p_2=q[/itex] is
    [tex]g(x;p_1,N) = (1-q+p_1x+(q-p_1)x^2)^N[/tex]
    [tex]P[L_N=n] = \tfrac{\partial^n}{\partial x^n}g|_{x=0}[/tex]
    so taking derivatives wrt [itex]p_1[/itex] and applying the Leibniz formula shows that the probability is maximized at either
    [tex]P[L_{N-1}=n-1] = P[L_{N-1}=n-2][/tex]
    or [itex]p_1=0[/itex] or [itex]p_1=q[/itex]. Not sure how to solve the equation though.
  16. Jul 21, 2011 #15
    Could you please define what you mean by [itex]L_i [/itex] and [itex] X_j[/itex]? Sorry, but, I am confused by these two notations.
  17. Aug 8, 2011 #16

    Stephen Tashi

    User Avatar
    Science Advisor


    The usual algorithms for "geometric programming" find the extrema of polynomials with positive coefficients, but people have developed extensions of the algorithms that handle polynomials with coefficients of arbitrary signs. I haven't searched for an online reference to these algorithms, but I recall reading a hardcopy of a paper about them years ago. So if you are seeking numerical answers, I think the algorithms exist.
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook