78

I am trying to understand an example from my textbook.

Let's say $Z = X + Y$, where $X$ and $Y$ are independent uniform random variables with range $[0,1]$. Then the PDF is $$f(z) = \begin{cases} z & \text{for $0 < z < 1$} \\ 2-z & \text{for $1 \le z < 2$} \\ 0 & \text{otherwise.} \end{cases}$$

How was this PDF obtained?

Thanks

StubbornAtom
  • 15,215
  • 4
  • 29
  • 66
Zhulu
  • 1,039
  • 1
  • 10
  • 12
  • There are a couple of ways. Have you done convolutions? Not the best way in my opinion, but certainly useful elsewhere. And there is a straightforward geometric approach. – André Nicolas Apr 11 '13 at 00:49
  • what would be the bounds if i were to use convolutions? also, I don't quite understand the geometric approach. Can you direct me to an example? – Zhulu Apr 11 '13 at 01:19
  • For convolution, you want $\int_{-\infty}^\infty f_Y(z-x)f_X(x)\,dx$. So since density is $0$ outside $(0,1)$, we need $0\le z-x\le 1$, or equivalently $x\le z\le x+1$. For $z\le 1$, the first bound is the one to use. For $1\lt z\le 2$, it is the second. – André Nicolas Apr 11 '13 at 01:33
  • So fy is 1 and fx is 1. What am I supposed to write in place of positive and negative infinity? 0 to 1 and then 1 to 2? – Zhulu Apr 11 '13 at 02:04
  • 1
    I sort of gave the bounds. For $0\le z\le 1$, integrate from $x=0$ to $x=z$. For $1\lz\le 2$, integrate from $z-1$$ to $1$. Will maybe write up answer. – André Nicolas Apr 11 '13 at 02:19
  • I assume that this question must have been answered in great detail umpteen times on math.SE. – Dilip Sarwate Apr 11 '13 at 03:55
  • @AndréNicolas I upvoted your answer but have a question about it [here](http://math.stackexchange.com/questions/586389/finding-integration-bounds-for-density-of-sum-of-two-independent-random-variable) – NaN Dec 01 '13 at 00:20
  • @AndréNicolas Hi, I have a brief question for your answer below. Why did you use value "1" to break the interval [0, 2] into two parts instead of another value, such as 0.5 etc. – sundaycat Dec 15 '14 at 05:11
  • 7
    The basic method is to my mind not convolution, but finding the cdf and differentiating. So draw the square on which the joint density lives. The probability that $Z\le z$ is the probability $(X,Y)$ lands in the part of the square "below" the line $x+y=z$. Imagine drawing lines $x+y=z$ for various $z$. The geometry of the "part below" changes at $z=1$. For $z\lt 1$ the part below is a triangle. For $1\lt z\lt 2$ it is the part above that is a triangle. – André Nicolas Dec 15 '14 at 05:36
  • possible duplicate of [Sum of two uniform random variables](http://math.stackexchange.com/questions/220201/sum-of-two-uniform-random-variables) – hola May 11 '15 at 18:22
  • Hi can you check my answer below, I think it is much easier to understand. – GoingMyWay Nov 06 '18 at 12:53
  • If I use a discrete analogy, say X and Y are on {0, 1, 2, 3, 4, 5} (dice marked 0 through 5 instead of 1 through 6), I can calculate the PDF by hand and see the intuition for the Real PDF. It is understandably wonky at the tails, but it gets better if I make X and Y larger. I haven't computed in integral in 35 years, so this discrete analogy helps me get the intuition for the PDF of the actual question. – Jeffrey Goldberg Sep 16 '21 at 17:43

5 Answers5

95

If we want to use a convolution, let $f_X$ be the full density function of$X$, and let $f_Y$ be the full density function of $Y$. Let $Z=X+Y$. Then $$f_Z(z)=\int_{-\infty}^\infty f_X(x)f_Y(z-x)\,dx.$$

Now let us apply this general formula to our particular case. We will have $f_Z(z)=0$ for $z\lt 0$, and also for $z\ge 2$. Now we deal with the interval from $0$ to $2$. It is useful to break this down into two cases (i) $0\lt z\le 1$ and (ii) $1\lt z\lt 2$.

(i) The product $f_X(x)f_Y(z-x)$ is $1$ in some places, and $0$ elsewhere. We want to make sure we avoid calling it $1$ when it is $0$. In order to have $f_Y(z-x)=1$, we need $z-x\ge 0$, that is, $x\le z$. So for (i), we will be integrating from $x=0$ to $x=z$. And easily $$\int_0^z 1\,dx=z.$$ Thus $f_Z(z)=z$ for $0\lt z\le 1$.

(ii) Suppose that $1\lt z\lt 2$. In order to have $f_Y(z-x)$ to be $1$, we need $z-x\le 1$, that is, we need $x\ge z-1$. So for (ii) we integrate from $z-1$ to $1$. And easily $$\int_{z-1}^1 1\,dx=2-z.$$ Thus $f_Z(z)=2-z$ for $1\lt z\lt 2$.

Another way: (Sketch) We can go after the cdf $F_Z(z)$ of $Z$, and then differentiate. So we need to find $\Pr(Z\le z)$.

For a few fixed $z$ values, draw the lines with equation $x+y=z$ on an x-y axis plot. Draw the square $S$ with corners $(0,0)$, $(1,0)$, $(1,1)$, and $(0,1)$.

Then $\Pr(Z\le z)$ is the area of the part $S$ that is "below" the line $x+y=z$. That area can be calculated using basic geometry. For example, when z is 2, the whole square area is under the line so Pr=1. There is a switch in basic shape at $z=1$.

ymzkala
  • 103
  • 2
André Nicolas
  • 491,093
  • 43
  • 517
  • 948
  • Thank very much for writing this up! This really helped me fully understand the concept of convolution. – Zhulu Apr 11 '13 at 03:45
  • 2
    Does the second mehod of calculating areas only work in this case since we are using uniform distributions? – F.Webber May 30 '16 at 18:16
  • 1
    @F.Webber: In the form that I used it, yes, we are reduced to finding area because of uniformity. But first going after the cdf is a **general** procedure. In the non-uniform case, we are finding an integral. The geometry is still useful in determining the bounds on the integration. – André Nicolas May 30 '16 at 18:41
  • So, in general, once we have determined such area, the function we have to integrate in that region would be the joint probability density function? – F.Webber May 30 '16 at 18:46
  • 1
    @F.Webber: Yes, for many problems a double integral, but integration in $n$-dimensional space also comes up. – André Nicolas May 30 '16 at 20:52
  • @F.Webber: You are welcome/ – André Nicolas May 31 '16 at 11:54
  • This is an awesome explanation, thanks a lot. But may I ask you: 1) How do you know that we should break (0, 2) into two cases, not, for example, four? 2) How do you know the bounds of the intervals? – alekscooper Mar 22 '17 at 16:52
44

Here's why we need to break the convolution into cases. The integral we seek to evaluate for each $z$ is $$ f_Z(z):= \int_{-\infty}^\infty f(x)f(z-x)\,dx.\tag1 $$ (On the RHS of (1) I'm writing $f$ instead of $f_X$ and $f_Y$ since $X$ and $Y$ have the same density.) Here the density $f$ is the uniform density $f(x)$, which equals $1$ for $0<x<1$, and is zero otherwise. The integrand $f(x)f(z-x)$ will therefore have value either $1$ or $0$. Specifically, the integrand is $1$ when $$ 0<x<1\qquad\text{and}\qquad 0<z-x<1,\tag2 $$ and equals zero otherwise. To evaluate (1), which is an integral over $x$ (with $z$ held constant), we need to find the range of $x$-values where the conditions listed in (2) are satisfied. How does this range depend on $z$? Plotting the region defined by (2) in the $(x,z)$ plane, we find:

and it's clear how the limits of integration on $x$ depend on the value of $z$:

  1. When $0<z<1$, the limits run from $x=0$ to $x=z$, so $f_Z(z)=\int_0^z 1dx=z.$

  2. When $1<z<2$, the limits run from $x=z-1$ to $x=1$, so $f_Z(z)=\int_{z-1}^11dx=2-z.$

  3. When $z<0$ or $z>2$, the integrand is zero, so $f_Z(z)=0$.

grand_chat
  • 32,010
  • 1
  • 28
  • 54
3

By the hint of jay-sun, consider this idea, if and only if $f_X (z-y) = 1$ when $0 \le z-y \le 1$. So we get

$$ z-1 \le y \le z $$

however, $z \in [0, 2]$, the range of $y$ may not be in the range of $[0, 1]$ in order to get $f_X (z-y) = 1$, and the value $1$ is a good splitting point. Because $z-1 \in [-1, 1]$.

Consider (i) if $z-1 \le 0$ then $ -1 \le z-1 \le 0$ that is $ z \in [0, 1]$, we get the range of $y \in [0, z]$ since $z \in [0, 1]$. And we get $\int_{-\infty}^{\infty}f_X(z-y)dy = \int_0^{z} 1 dy=z$ if $z \in [0, 1]$.

Consider (ii) if $z-1 \ge 0$ that is $ z \in [1, 2]$, so we get the range of $y \in [z-1, 1]$, and $\int_{-\infty}^{\infty}f_X(z-y)dy = \int_{z-1}^{1} 1 dy = 2-z$ if $z \in [1, 2]$.

To sum up, consider to clip the range in order to get $f_X (z-y) = 1$.

GoingMyWay
  • 257
  • 1
  • 12
1

Simple approach for those who don't know convolution.

First we need to find the range of possibilities for the sum.

  • Minimum will occur when both numbers are minimum, so min = 0.
  • Maximum will occur when both numbers are maximum, so max = 2.
  • Most likely outcome (or mode) is when both numbers are same as their mean, so mode = 1.

These three are enough to specify a triangular distribution. We need to make sure that the area under the pdf is 1, which means the height of pdf at mode(h) is

$$ \frac{1}{2}*2*h = 1 $$

This gives $h=1$. All you need know is to find the equations of 2 lines that go from-

  1. (0,0) to (1,1)
  2. (1,1) to (2,0)

Give a shout if anything is not clear.

Innocent
  • 599
  • 2
  • 5
  • 14
  • Your answer doesn't explain: 1. Why the most likely outcome is when both random variables equal their mean. 2. Why the three points are enough to specify a triangular distribution. – Mark Ebden Feb 26 '21 at 01:10
0

The purpose of this answer is to show how a direct application of convolution may lead to the desired result. I take the following results from Cohn, Measure Theory.

Definition of convolution Let $\nu_1$ and $\nu_2$ be finite measures on $(\mathbb{R}^d,\mathscr{B}(\mathbb{R}^d))$, then their convolution $\nu_1\ast\nu_2$ is defined by:

$$ \nu_1 \ast\nu_2(A) = \nu_1 \times\nu_2(\{(x_1,x_2) : x_1+x_2 \in A\})$$

Proposition 10.1.12 Let $\nu_1$ and $\nu_2$ be probability measures on $(\mathbb{R}^d,\mathscr{B}(\mathbb{R}^d))$.

$\vdots$

(c) If $\nu_1$ and $\nu_2$ are absolutely continuous, with densities $f$ and $g$, then $\nu_1\ast\nu_2$ is absolutely continuous with density: $$x \mapsto \int f(x-y)g(y)\lambda(dy)$$

Let $I$ denote the unit interval $[0,1]$, and $U(I)$ the uniform distrbution on $I$. Then the density function corresponding to $U(I)$ is $\chi_I$, the indicator function for $I$. If $X$ and $Y$ are independent random variables whose distributions are given by $U(I)$, then the density of their sum is given by the convolution of their distributions. I.e., if $f_X$ denotes the density for random variable $X$, then

$$ f_{X+Y}(x) = \int f_X(x-y)f_Y(y)\lambda(dy) = \int \chi_I(x-y)\chi_I(y) dy$$

The indicator function of $y$ alone restricts the integration range, so that

$$ \int \chi_I(x-y)\chi_I(y)dy = \int_0^1 \chi_I(x-y) dy$$

The expression $\chi_I(x-y)$ is $0$ if $x-y < 0$ or $x-y > 1$:

$$\chi_I(x-y) = \cases{1 & $x-1 \leq y \leq x$ \\ 0 & otw} $$

This further restricts the range of the integral, which can be rewritten:

$$\int_{max(0,x-1)}^{min(1,x)} 1 dy = min(1,x) - max(0,x-1)$$

The density is $0$ if $x < 0$ or $x > 2$. This fact is hidden in our final expression because we've expressed our indicator functions through the bounds of the integral, but can be recovered by including another indicator function. The PDF as described in the original question follows by considering the relevant cases.

Nathan Chappell
  • 211
  • 2
  • 4