## 4.4 Application: Joint Probability Densities

In this section, we will discuss how multiple integrals are useful in the field of probability theory.

First, we need to define **improper intgerals** over unbounded regions.

:::info

**Definition**
$$\int \cdots \int_{\mathbb R^n} f\,\,dx_1\,\cdots\,dx_n\, = \lim_{R \to \infty} \int_{-R}^R \cdots \int_{-R}^R f\,\,dx_1\,\cdots\,dx_n\,$$if the limit exists.
:::

In general, if $D \subset \mathbb R^n$ is a nice *unbounded* region, then we can define

$$
\int \cdots \int_{D} f\,\,dx_1\,\cdots\,dx_n\, := \lim_{R \to \infty} \int_{-R}^R \cdots \int_{-R}^R f \cdot 1_{D}\,\,dx_1\,\cdots\,dx_n\,
$$where $$1_{D} = \begin{cases}1 & \text{on}\,\,\, D\\ 0 & \text{on}\,\,\,\mathbb R^n \setminus D\,\end{cases}$$
is the indicator function of the set $D$.
<br>

Recall the following definitions from basic probability theory.

Let $X$ be a real valued random variable. Then the **distribution function $F$** of $X$ is given by
$$F(x)=\mathbb P (X \leq x)\,.$$If there is a function $f \geq 0$ such that
$$F(x) = \int_{-\infty}^x f(y)\,dy$$and $$\int_{\mathbb R} f(x)\,dx = 1\,,$$then $f$ is called the **probability density function** of $X$.

The probability density function allows us to calculate probabilities. For example, if the random avriable $X$ has density $f$ then for any nice set $C$, $$\mathbb P(X \in C)=\int_{C} f(x) \,dx\,.$$Also, the expected value of $X$ is
$$\mathbb E(X)=\int_{\mathbb R} x f(x)\, dx\,,$$ and more generally for any function $g:\mathbb R \to \mathbb R$,
$$\mathbb E(g(X))=\int_{\mathbb R} g(x) f(x)\, dx\,$$provided the integral exists.
<br>

In general, if we have two or more random variables, $X_1,\dots,X_n$, we can define their **joint probability distribution** and **joint probability density function** as follows.

:::info

**Definition**

Let $X_1,\dots,X_n$ be a real valued random variable. Then their **joint distribution function $F$** is given by
$$F(x_1,\dots,x_n)=\mathbb P (X_1 \leq x_1,\dots,X_n \leq x_n)\,.$$If there is a function $f \geq 0$ such that
$$F(x_1,\dots,x_n) = \int_{-\infty}^{x_n}\cdots\int_{-\infty}^{x_1}f(y_1,\dots,y_n)\,dy_1\cdots\,dy_1$$ and $$\int\cdots\int_{\mathbb R^n}f\,\,dx_1\cdots\,dx_n = 1\,,$$then $f$ is called the **joint probability density function** of $X_1,\dots,X_n$.
:::
<br>

Just like in dimension one, if the joint probability density function of $X_1,\dots,X_n$ is $f$ and $C \subseteq \mathbb R^n$ is a *nice set*, then
:::success
$$\mathbb P((X_1,\dots,X_n)\in C) = \int\cdots\int_{C}f\,\,dx_1\cdots\,dx_n$$
:::
and for any *reasonable* function $g:\mathbb R^n \to \mathbb R$
:::success
$$\mathbb E(g(X_1,\dots\,X_n))=\int\cdots\int_{\mathbb R^n} g \cdot f \,\,dx_1\cdots\,dx_n$$
:::
provided the integral exists.

In particular, if the joint density of $X_1,\dots,X_n$ is $f$, then the **expected value** of $X_k$ for $1\leq k \leq n$, (the average value of $X_k$) can be computed as follows.

Write $g(x_1,\dots,x_n)=x_k$ (the projection onto the $k$th coordinate.) Then,
\begin{align}
\mathbb E(X_k) = \mathbb E(g(X_1,\dots,X_k)) &= \int\cdots\int_{\mathbb R^n} g \cdot f\,\, dx_1\cdots dx_n \\&=\int\cdots\int_{\mathbb R^n} x_k\,f(x_1,\cdots,x_n)\,\,dx_1\cdots dx_n
\end{align}

Another important notion about radom variables is their independence.

:::info

We say that the random variables $X_1, \dots, X_n$ are **independent** if their joint density function is the product of their individual density functions.
:::
<br>

**Example 1**
Suppose the joint density function for $X$ and $Y$ is given by
$$f(x, y)= \begin{cases}C(x+y) & \text{if}\,\,\,0 \leq x \leq 10,\,\, 0 \leq y \leq 10\,,\\ \\0 & \text{otherwise}.\end{cases}$$

a. Find the value of the constant $C$.
b. Find $\mathbb P(X \leq 7, Y \geq 2).$
c. Compute $\mathbb E(X)$ and $\mathbb E(Y)$.
c. What is the density function of $X$?
d. What is the density function of $Y$?
e. Are $X$ and $Y$ independent?

:::spoiler Answer
- We must have $\iint_{\mathbb R^2} f(x,y)=1$. Since
\begin{align}
\int_{0}^{10}\int_{0}^{10} C(x+y)\,dx\,dy &= C\int_{0}^{10} \frac{x^2}{2}+yx\Big|_{x=0}^{x=10}\,dy \\ &=C\int_{0}^{10} (50+10y) \, dy = C \Big(50y+5y^2\Big|_{0}^{10}\Big) = 1000C\,,
\end{align}we have $C=1/1000$.
- Next,
\begin{align}
\mathbb P(X \leq 7, Y \geq 2) &= \mathbb P(0 \leq X \leq 7, 2\leq Y \leq 10)\\ &=\frac{1}{1000}\int_2^{10}\int_{0}^7 (x+y)\,dx\,dy=\frac{532}{1000}=0.532\,.
\end{align}
- By symmetry, $\mathbb E(X) = \mathbb E(Y)$. So, we compute, \begin{align}
\mathbb E(X) = \frac{1}{1000}\int_{0}^{10}\int_{0}^{10} x(x+y)\,dx\,dy =\frac{35}{6}\,.
\end{align}
- Note that if $C \subset \mathbb [-10,10]$ is an interval,
$$\mathbb P (X \in C) = \mathbb P (X \in C, -\infty \leq Y \leq \infty) =\frac{1}{1000}\int_C \int_{0}^{10} (x+y) \,dy\,dx = \int_C\frac{x+5}{100}\,dx\,.$$Since this is true for any interval $C$, the density of $X$ is
$$f_X(x)= \begin{cases} \frac{x+5}{100} & \text{if}\,\,\,0 \leq x \leq 10,\\ \\0 & \text{otherwise}.\end{cases}$$
- By symmetry, the density of $Y$ is
$$f_Y(y)= \begin{cases} \frac{y+5}{100} & \text{if}\,\,\,0 \leq y \leq 10,\\ \\0 & \text{otherwise}.\end{cases}$$
- Clearly, $f(x,y) \neq f_X(x)\cdot f_Y(y)$. So, $X$ and $Y$ are dependent.
:::
<br>

**Example 2**
Waiting times are typically modelled by the exponential random variables whose density is
$$E(t)= \begin{cases}0 & \text{if}\,\,\,t < 0 \\ r^{-1}e^{-t/r} & \text{if}\,\,\,t \ge 0\end{cases}$$where $r$ is the mean waiting time.

The manager of a movie theatre determines that the average time a person waits in line to buy a ticket for this week’s film is $10$ minutes and the average time a person waits to buy popcorn is $5$ minutes.

Assuming that these waiting times are independent, find the probability that a person waits a total of less than $20$ minutes before taking their seat.

:::spoiler Answer
If $X$ is the waiting time for tickets and $Y$ is the waiting time for popcorn, their density functions are $$E_X(x)= \begin{cases}0 & \text{if}\,\,\,x < 0 \\ \frac{1}{10}e^{-t/10} & \text{if}\,\,\,x \ge 0\end{cases}$$and $$E_Y(y)= \begin{cases}0 & \text{if}\,\,\,y < 0 \\ \frac{1}{5}e^{-y/5} & \text{if}\,\,\,y \ge 0\end{cases}$$respectively. Since $X$ and $Y$ are independent, their joint density function is $$f(x,y)=E_X(x)E_Y(y) =\begin{cases}\frac{1}{50}e^{-x/10}e^{-y/5} &\text{if}\,\,\,x \geq 0, y \geq 0 \\ 0 & \text{otherwise}\end{cases}.$$

We have to find $$\mathbb P (X+Y < 20)= \mathbb P((X,Y) \in D) = \iint_D f(x,y)\,\,dx\,dy$$where $D$ is the region below.
Hence,\begin{align}\mathbb P (X+Y < 20) &= \frac{1}{50}\int_{0}^{20}\int_{0}^{20-y} e^{-x/10}e^{-y/5}\,dx\,dy \\ &= \frac{1}{50}\int_{0}^{20} -10e^{-x/10}e^{-y/5}\Big|_{x=0}^{x=20-y}\,dy \\ &= \frac{1}{5}\int_{0}^{20} (1-e^{-2+y/10})e^{-y/5}\,dy \\ &= \frac{1}{5}\int_{0}^{20} e^{-y/5}\,dy - \frac{1}{5e^2}\int_{0}^{20}e^{-y/10}\,dy \\
&= (1-e^{-4}) - 2(e^{-2}-e^{-4}) = 1-2e^{-4}+e^{-2} = (1-e^{-2})^2 \approx 0.75.
\end{align}
:::
<br>

**Example 3**
Let $a,b,$ and $c$ be real numbers selected independently and uniformly randomly from the interval $(0,1)$. What is the probability that the equation $ax^2+bx+c=0$ has at least one real solution?

:::spoiler Answer
Since $a,b,$ and $c$ are chosen indepdently and uniformly randomly, the triples $(a,b,c)$ are distributed uniformly randomly on $I_3=(0,1)\times (0,1)\times (0,1)$ and we have to compute $$\mathbb P(b^2-4ac \geq 0)\,.$$

Note that because $0<a,b,c<1$, $b^2 - 4ac > 0 \iff1>b>2\sqrt{a}\sqrt{c}>0$. Therefore $b$ ranges from $2\sqrt{a}\sqrt{c}$ to $1$, and $1 > 2\sqrt{a}\sqrt{c} > 0$ on the $ac-$plane looks like this.


So, the required probability is
\begin{align}\mathbb P(b^2-4ac \geq 0) &=\iint_{R_1\cup R_2}\int_{2\sqrt{a}\sqrt{c}}^1 1 \,db\,dc\,da \\ &= \int_0^{1/4}\int_0^1\int_{2\sqrt{a}\sqrt{c}}^1 1 \,db\,dc\,da+\int_{1/4}^1\int_{0}^{1/4a}\int_{2\sqrt{a}\sqrt{c}}^1 1
\,db\,da\,dc\\ &= \frac{5}{36}+ \frac{1}{12} \ln 4 \approx 0.25\end{align}
:::
<br>

**Example 4**

Suppose the two randomvariables $X$ and $Y$ have a jointly density function $f$. Define two new random variables $U = X+Y$ and $V = X-Y$. What is the joint density of $U$ and $V$?

:::spoiler Answer

We need to find $g$ such that $$\mathbb P((U,V)\in C) = \iint_C g(u,v) \,du\,dv.$$Observe that
\begin{align}\mathbb P((U,V)\in C) &= \mathbb P((X+Y,X-Y)\in C) \\&= \mathbb P((X,Y)\in T^{-1}(C)) = \iint_{T^{-1}C}f(x,y)\,dx\,dy\end{align}where $T(x,y)=(x+y,x-y)$ with $T^{-1}(u,v)=\left(\frac{u+v}{2},\frac{u-v}{2}\right)\,.$

So, we can apply the change of coordinates formula:
$$\iint_{T^{-1}C}f(x,y)\,dx\,dy = \iint_C f\left(\frac{u+v}{2},\frac{u-v}{2}\right) \,\frac{1}{2}du\,dv\,,$$noting that the Jacobian determinant of the $T^{-1}$ (that transforms $(u,v)$ to $(x,y)$) is $$\det J_{T^{-1}}=\begin{vmatrix}1/2 & 1/2 \\ 1/2 & -1/2\end{vmatrix}=-\frac{1}{2}\,.$$

Therefore, the joint density of $U$ and $V$ is
$$\frac{1}{2}f\left(\frac{u+v}{2},\frac{u-v}{2}\right)\,.$$
:::

<br>

---
**Reference**
1. *Chapter 15.4 & 15.6* : Stewart, J. (2012). Calculus (8th ed.). Boston: Cengage Learning.
2. *Chapter 3.7* : Corral, M. (2021). Vector Calculus. [https://www.mecmath.net/](https://www.mecmath.net/)

---