Derivative of integral over distribution

Click For Summary

Discussion Overview

The discussion revolves around the derivative of an integral involving probability distributions, specifically focusing on the calculations and simulations in MATLAB. Participants are examining the mathematical formulation and the discrepancies observed in the simulation results.

Discussion Character

  • Technical explanation
  • Mathematical reasoning
  • Debate/contested

Main Points Raised

  • Bob presents an integral involving a cumulative distribution function (CDF) and its associated probability density function (PDF), expressing concern over discrepancies between the integral and its derivative when simulated in MATLAB.
  • Bob proposes a derivative of the integral but later acknowledges a need to differentiate with respect to a different variable, p, rather than y.
  • Another participant questions the differentiation process, specifically regarding the treatment of G(p-y) and whether it should yield g(p-y).
  • One participant suggests that regularity conditions for differentiating inside the integral may not apply and asks for clarification on the specific functions involved.
  • Bob clarifies that he is using normal distributions for y and z and admits he has not checked the regularity conditions.
  • There is a mention of a bug in the site's LaTeX preview feature, affecting participants' ability to present their equations clearly.
  • Bob expresses confusion about the units of the vertical axis in his plots and explains his approach to plotting the functions based on fixed values of p.
  • Bob's end goal is to confirm the maximum of the first function by finding the root of the first-order condition.

Areas of Agreement / Disagreement

Participants have not reached a consensus on the correctness of the derivative or the simulation results. There are multiple viewpoints regarding the differentiation process and the conditions required for it.

Contextual Notes

Participants have not fully explored the regularity conditions necessary for differentiating under the integral sign, which may affect the validity of their calculations. Additionally, there are unresolved questions about the units used in the plots presented.

bob_johnson
Messages
5
Reaction score
0
Hi,

I am doing work that requires me to take the derivative of an integral over a distribution. I believe I calculated it correctly, but when I simulate the results in matlab, the plot for the integral and its derivative don't match up.

Here is the equation:

[tex] $\int_{-\infty }^{\infty }\left[ p[1-G(p-y)]+\int_{-\infty }^{p-y}zg(z)dz\right] f(y)dy$ [/tex]

where G is the cdf of z with g the associated pdf, and F is the CDF of y with f the associated pdf.

I believe the derivative is the following:

[tex] $\int_{-\infty }^{\infty }\left[ [1-G(p-y)]-yg(p-y)dz\right] $[/tex]

But, when I simulate the two in MATLAB the graphs don't match up. See attached pdf.

Thanks for your help.

Any insight would be greatly appreciated.

All the best,
Bob
 

Attachments

Last edited:
Physics news on Phys.org
Here is the equation:
[tex]$\int_{-\infty }^{\infty }\left[ p[1-G(p-y)]+\int_{-\infty }^{p-y}zg(z)dz\right] f(y)dy$[/tex]
where G is the cdf of z with g the associated pdf, and F is the CDF of y with f the associated pdf.

I believe the derivative is the following:

[tex]$\int_{-\infty }^{\infty }\left[ [1-G(p-y)]-yg(p-y)dz\right] $[/tex]

But, when I simulate the two in MATLAB the graphs don't match up. See attached pdf.

Thanks for your help.
 
bob_johnson said:
I believe the derivative is the following:

[tex]$\int_{-\infty }^{\infty }\left[ [1-G(p-y)]-yg(p-y)dz\right] $[/tex]

I don't understand what you did. Did you differentiate inside the integral sign with respect to [itex]y[/itex]?

Did you differentiate [itex]G(p-y)[/itex] with respect to [itex]y[/itex]? Why didn't it become [tex]g(p-y)[/tex] ?
 
Good point. My apologies. I need the derivative with respect to p. Thanks for responding!
 
Assuming we can differentiate inside the integral sign, wouldn't we have:

[tex]D_p \left ( p(1 - G(p-y) \right) = 1 - G(p-y) - pg(p-y)[/tex]

and
[tex]D_p \left(\int_{-\infty}^{p-y}zg(z) dz \right) = (p-y)g(p-y)[/tex]
 
Yes, that's what I have thanks. Actually, I just noticed there is a type in the derivative that I wrote above. For some reason, the site's preview for latex doesn't work for me.

the derivative that I got, which confirms with yours, is:

[tex]$\int_{-\infty }^{\infty }\left[ [1-G(p-y)]-pg(p-y)+\left( p-y\right) g(p-y)%<br /> \right] f(y)dy$[/tex]

and thus becomes:

[tex]$\int_{-\infty }^{\infty }\left[ [1-G(p-y)]-yg(p-y)\right] f(y)dy$[/tex]

So, I wonder where I'm going wrong with my simulation. As indicated in my plots in sample.pdf above, the graphs of the two functions just don't match up. Given the functions, he graph of the first-order condition with respect to p should cross 0 where the graph of the first equation is at a maximum. Clearly from the graphs that I attached above, they don't. I'm perplexed. I'm pretty sure that I didn't make a coding mistake, but I guess if your calculation of the derivative agrees with mine, then I did somehow. Any suggestions a to how to determine which graph is correct, because I am using those functions in my simulation.

Thanks again for your help!
 
bob_johnson said:
For some reason, the site's preview for latex doesn't work for me.

It doesn't work for anybody. There's a bug in it. You have to do the preview and then have your browser reload the page. Before you reload it, the preview just looks crazy.
 
Maybe regularity conditions for differentiating inside the integral don't apply. What are the specific functions involved?

In your first plot, why isn't the vertical axis in units of probability?
 
I'm just using the normal distribution.

So, y~N(1,1)
and z~N(0,1)

In other words, G = N(0,1) and F=N(1,1).

I haven't checked the regularity conditions. I guess I should.

I'm not sure what you mean by units of probability.
For the first graph, I fix a value for p, and then just take the expectation over y, using numerical quadrature. So, what I plot as "v" is just the value of that first equation for a given p. I do the same for the second graph, where "fonc" is the value of that second equation.

My end goal is to confirm the choice of maximum of the first function by finding the root (choice of p) of the first-order condition.
 

Similar threads

  • · Replies 4 ·
Replies
4
Views
3K
  • · Replies 1 ·
Replies
1
Views
3K
  • · Replies 4 ·
Replies
4
Views
3K
  • · Replies 11 ·
Replies
11
Views
3K
  • · Replies 2 ·
Replies
2
Views
2K
  • · Replies 3 ·
Replies
3
Views
3K
  • · Replies 5 ·
Replies
5
Views
3K
  • · Replies 4 ·
Replies
4
Views
3K
  • · Replies 14 ·
Replies
14
Views
3K
  • · Replies 3 ·
Replies
3
Views
1K