Probability Density Functions: Transformation of Variables

[Master1022]

Homework Statement

If we have $p(x) = c(x+1)^2 (1 - x)$ where $-1 \leq x \leq 1$ and we have a variable transformation $Z = X^2$, then find $p(z)$

Relevant Equations

Probability transformation

Hi,

I have a question about probability transformations when the transformation function is a many-to-one function over the defined domain.

Question: How do we transform the variables when the transformation function is not a one-to-one function over the domain defined? If we have $p(x) = m(x+1)^2 (1 - x)$ where $-1 \leq x \leq 1$, where $m$ is a constant, and we have a variable transformation $Z = X^2$, then find $p(z)$

Context attempt:
I was reading some lecture notes where all it says is: "if the transformation function $h(x)$ is not one-to-one, then we use a more complicated method".

So I know that usually when we go from $x$ to $z$, then we need to consider the Jacobian determinant $| \frac{\partial x}{\partial z} |$. For the example above, then that becomes:
$$p(z) = m(\sqrt{z} + 1)^2 (1 - \sqrt{z}) \cdot \left|\frac{\partial x}{\partial z} \right| = m(\sqrt{z} + 1)^2 (1 - \sqrt{z}) \cdot \frac{1}{2\sqrt{z}}$$
but this exactly what was done for the situation where the transformation function was 1-to-1 over the domain defined.

For a discrete system when we have a transformation, then we might have something along the lines of:
$$p(Z = z) = \sum p(Z = X^2, X = x) = \sum p(Z = X^2|X = x) p(X = x)$$
but I am confused on how I can utilize this methodology for the continuous case.

Thank you in advance for any help.

 

[jambaugh]

I think this works better (conceptually) in the CDF for the system of which the PDF is the derivative (with some generalizations) of the CDF. Also CDF's can be agnostic about "continuous" vs "discrete" vs "mixed" distributions.

Where $z = \zeta(x)$ is a "reasonable" mapping (e.g. piecewise analytic).
$$P(Z\le z) = \int_{\forall x: \zeta(x)\le z} f(x) dx = \int_{\forall x:\zeta(x)\le z} \frac{d}{dx} P(X\le x) dx$$

where $f$ is the density function in $x$ and the derivative inside the integral is generalized to yield delta functions for steps. The problem with the general case is, of course, working with variable transformations which are not monotone increasing or monotone decreasing i.e. not continuously invertible.

When that is the case your domain of integration for certain z-values will be multiple disjoint intervals (including single points as zero-width closed intervals) where both boundaries will depend on z.

Then when you differentiate the CDF to get the PDF (w.r.t. z) you're going to have to differentiate several (potentially infinitely many) boundaries of the resulting integrals. But, in the end you can express the CDF w.r.t. z as a series of differences of CDF values w.r.t. x. E.g. let $\{[x_1(z),x_2(z)], [x_3(z),x_4(z)], \ldots\}$ be the set of intervals where $x$ satisfies $\zeta(x)<z$. Then:
$$P(Z\le z) = \int_{x_1}^{x_2} f(x)dx + \int_{x_3}^{x_4} f(x) dx + \ldots = F(x_2)-F(x_1) + F(x_4)-F(x_3) \ldots$$
where $F(x) = P(X\le x)$ is the CDF in $x$. To get the PDF you then differentiate w.r.t. $z$ recalling each of the $x$'s depend on $z$.

I don't think you can get simpler and more direct than this without making simplifying (or at least specifying) assumptions about the change of variable.

[edit: Here I implicitly assumed continuity and you'll have to be more careful with open vs closed intervals in the discrete or partially discrete cases.]
[edit2: But then you should just work with sums in the discrete case and break mixed cases into sums of a discrete and a continuous distribution.]

 

[Master1022]

Thank you very much @jambaugh ! I have some follow-up questions as I don't think I understand everything in the post.

I think this works better (conceptually) in the CDF for the system of which the PDF is the derivative (with some generalizations) of the CDF. Also CDF's can be agnostic about "continuous" vs "discrete" vs "mixed" distributions.

Where $z = \zeta(x)$ is a "reasonable" mapping (e.g. piecewise analytic).
$$P(Z\le z) = \int_{\forall x: \zeta(x)\le z} f(x) dx = \int_{\forall x:\zeta(x)\le z} \frac{d}{dx} P(X\le x) dx$$

where $f$ is the density function in $x$ and the derivative inside the integral is generalized to yield delta functions for steps. The problem with the general case is, of course, working with variable transformations which are not monotone increasing or monotone decreasing i.e. not continuously invertible.

Okay, understood (I think)

When that is the case your domain of integration for certain z-values will be multiple disjoint intervals (including single points as zero-width closed intervals) where both boundaries will depend on z.

Then when you differentiate the CDF to get the PDF (w.r.t. z) you're going to have to differentiate several (potentially infinitely many) boundaries of the resulting integrals. But, in the end you can express the CDF w.r.t. z as a series of differences of CDF values w.r.t. x. E.g. let $\{[x_1(z),x_2(z)], [x_3(z),x_4(z)], \ldots\}$ be the set of intervals where $x$ satisfies $\zeta(x)<z$.

Sorry, I don't quite get why this is the case. Could you perhaps explain why? So for the transformation function $\zeta(x)$ which is many-to-one, then we need to split the function up into different regions? (e.g -1 to 0 and 0 to 1)?

Then:
$$P(Z\le z) = \int_{x_1}^{x_2} f(x)dx + \int_{x_3}^{x_4} f(x) dx + \ldots = F(x_2)-F(x_1) + F(x_4)-F(x_3) \ldots$$
where $F(x) = P(X\le x)$ is the CDF in $x$. To get the PDF you then differentiate w.r.t. $z$ recalling each of the $x$'s depend on $z$.
I don't think you can get simpler and more direct than this without making simplifying (or at least specifying) assumptions about the change of variable.

okay thanks. I am still slightly confused about what steps I should do to tackle this problem. Should I:
1) Find the CDF of the original distribution
2) Change the variables to get CDF in terms of Z - I still am not sure how to do this in practice here
3) Then differentiate that to get pdf in terms of z

Thanks in advance

 

[pasmith]

Here you can use the fact that $0 \leq a^2 \leq X^2 \leq b^2$ if and only if either $-|b| \leq X \leq -|a|$ or $|a| \leq X \leq |b|$. Then $$P(X^2 \leq z) = \int_{a^2}^z\,f_{X^2}(z)\,dz = \int_{-z^{1/2}}^{-|a|} f_X(x)\,dx + \int_{|a|}^{z^{1/2}} f_X(x)\,dx$$ for $a^2 \leq z \leq b^2.$