(a) Let $X$ be a random variable that takes only non-negative values. Prove that for any $a > 0$,
\[P(X \geq a) \leq \frac{E[X]}{a}.\] This inequality is called Markov’s inequality.

(b) Let $X$ be a random variable with finite mean $\mu$ and variance $\sigma^2$. Prove that for any $a >0$,
\[P\left(|X – \mu| \geq a \right) \leq \frac{\sigma^2}{a^2}.\] This inequality is called Chebyshev’s inequality.

Solution.

We give two proofs of Markov’s inequality.

First Proof of Markov’s Inequality

For the first proof, let us assume that $X$ is a discrete random variable. The case when $X$ is a continuous random variable is identical except summations are replaced by integrals. The mean $E[X]$ is by definition
\begin{align*}
E[X] &= \sum_{x, p(x)>0} x p(x).
\end{align*}
Here, each term $xp(x)$ is a non-negative number as $X$ is non-negative and $p(x)$ is a probability. Thus, omitting some terms reduces the sum.

Hence we have
\begin{align*}
E[X] &= \sum_{x} x p(x) \geq \sum_{x \geq a} x p(x).
\end{align*}
(We omitted those $x$ such that $x \lt a$.)

Now, since $x \geq a$, we have
\[\sum_{x \geq a} x p(x) \geq \sum_{x \geq a} a p(x) = a \sum_{x \geq a} p(x).\] Note that
\[\sum_{x \geq a} p(x) = P(X \geq a).\] It follows that we obtain
\[E[X] \geq a P(X \geq a).\] Dividing this by $a>0$, we obtain the Markov’s inequality
\[P(X \geq a) \leq \frac{E[X]}{a}.\]

Second Proof of Markov’s Inequality

Let us give an alternative proof. We define a new random variable $I$ by
\begin{align*}
I =
\begin{cases}
1 & \text{ if } X \geq a\\
0 & \text{ otherwise}.
\end{cases}
\end{align*}
(This is called an indicator variable for the event $X \geq a$.)

When $X \geq a$, we have $I = 1$. Thus,
\[\frac{X}{a} \geq 1 = I.\] If, on the other hand, $X \lt a$, then as both $X$ and $a$ are non-negative, we have
\[\frac{X}{a} \geq 0 = I.\] Therefore, in either case, we have the inequality
\[\frac{X}{a} \geq I. \]

This implies the inequality of their expected values
\[E\left[\frac{X}{a}\right] \geq E[I].\tag{*}\] By linearity of expected value, we see that
\[E\left[\frac{X}{a}\right] = \frac{E[X]}{a}.\] Also, we have
\begin{align*}
E[I] &= 0\cdot p(0) + 1\cdot p(1)\\
&= p(1)\\
&= P(X \geq a)
\end{align*}

It follows from (*) that
\[\frac{E[X]}{a} \geq P(X \geq a).\] This completes the proof of Markov’s inequality.

Proof of Chebyshev’s Inequality

The proof of Chebyshev’s inequality relies on Markov’s inequality.
Note that $|X – \mu| \geq a$ is equivalent to $(X-\mu)^2 \geq a^2$. Let us put
\[Y = (X-\mu)^2.\] Then $Y$ is a non-negative random variable.

Applying Markov’s inequality with $Y$ and constant $a^2$ gives
\begin{align*}
P(Y \geq a^2) \leq \frac{E[Y]}{a^2}.
\end{align*}

Now, the definition of the variance of $X$ yields that
\[E[Y]=E[(X-\mu)^2] = V[X] = \sigma^2.\]

Combining these computations gives
\begin{align*}
P(|X-\mu| \geq a) &= P((X-\mu)^2 \geq a^2)\\[6pt] &= P(Y \geq a^2)\\[6pt] &\leq \frac{E[Y]}{a^2}\\[6pt] &= \frac{\sigma^2}{a^2},
\end{align*}
which concludes the proof of Chebyshev’s inequality.

Let $X\sim \mathcal{N}(\mu, \sigma)$ be a normal random variable with parameter $\mu=6$ and $\sigma^2=4$. Find the following probabilities using the Z-table below.

(a) Find $P(X \lt 7)$.

(b) Find $P(X \lt 3)$.

(c) Find $P(4.5 \lt X \lt 8.5)$.

Solution.

To make use of the Z-table, we need to relate current problems to a standard normal distribution $\mathcal{N}(0, 1)$ with mean $0$ and deviation $1$. To do this, as $X$ is a normal random variable with mean $\mu = 6$ and standard deviation $\sigma = 2$, the new random variable $Z$ defined by
\[Z = \frac{X – 6}{2}\] is a standard normal random variable.

Solution of (a)

The inequality $X < 7$ is equivalent to the inequality \[Z = \frac{X - 6}{2} < \frac{7-6}{2} = 0.5.\] Thus, the required probability is \begin{align*} P(X < 7) = P(Z < 0.5)= \Phi(0.5). \end{align*} Here, $\Phi(x)$ is the cumulative distribution function of a standard normal random variable $Z$. The value of $\Phi(0.5)$ can be found from the Z-table. Looking at row 0.5 and column 0.00, we see that $\Phi(0.5) \approx 0.6915$. Hence, the answer is
\[P(X < 7 ) \approx 0.6915.\]

Solution of (b)

As we did in Part (a), we transform the inequality and get
\begin{align*}
P(X \lt 3) &= P\left(\frac{X-6}{2} \lt \frac{3-6}{2}\right)\\[6pt] &= P(Z \lt -1.5)\\
&= \Phi(-1.5).
\end{align*}

Now, note that the Z-table gives the values of $\Phi(x)$ for only non-negative $x$.
Thus, to compute $\Phi(-1.5)$, we use the symmetry of the graph of a standard normal distribution. As $\Phi(-1.5)$ is the area under the bell curve from $-\infty$ to $-1.5$, this is equal to the area under the bell curve from $1.5$ to $\infty$ by symmetry, which is the same as $1-\Phi(1.5)$.

See the figure below. In the figure, the left orange region is $\Phi(-1.5)$, which is equal to the right orange region. Since the total area under the curve $\Phi(x)$ is $1$, the area of the right orange region is $1 – \Phi(1.5)$.

Thus
\begin{align*}
\Phi(-1.5) &= 1 – \Phi(1.5)\\
&\approx 1 – 0.9332 && \text{ from the Z-table}\\
&= 0.0668.
\end{align*}
Thus, we obtain the probability
\[P(X < 3) \approx 0.0668.\]

Solution of (c)

Again, by normalizing, we obtain
\begin{align*}
P(4.5 < X < 8.5) &= P\left( \frac{4.5 - 6}{2} < X < \frac{X - 6}{2} < \frac{8.5 - 6}{2} \right)\\ &=P(-0.75 < Z < 1.25)\\ &= \Phi(1.25) - \Phi(-0.75). \end{align*} Now, to find the value of $\Phi(-0.75)$ from the Z-table, we use the symmetry of the bell curve as in part (b) and we see $\Phi(-0.75) = 1 – \Phi(0.75)$.
Thus,
\begin{align*}
P(4.5 < X < 8.5) &= \Phi(1.25) - \Phi(-0.75)\\ &= \Phi(1.25) - (1-\Phi(0.75))\\ &= \Phi(1.25) + \Phi(0.75) - 1\\ &\approx 0.8944 + 0.7734 - 1\\ &=0.6678. \end{align*} Note that we found values $\Phi(1.25)\approx 0.8944$ and $\phi(0.75)\approx 0.7734$ from the Z-table.
In conclusion, we have
\[P(4.5 < X < 8.5) \approx 0.6678.\]

Z-table

The Z-table below gives numerical values for the cummulative distribution function $\Phi(x)$ of the standard normal random variable $Z$.

(You may need a large display to see the whole table.)

\begin{array}{rrrrrrrrrrr}
\hline
& 0.00 & 0.01 & 0.02 & 0.03 & 0.04 & 0.05 & 0.06 & 0.07 & 0.08 & 0.09 \\
\hline
0.0 & 0.5000 & 0.5040 & 0.5080 & 0.5120 & 0.5160 & 0.5199 & 0.5239 & 0.5279 & 0.5319 & 0.5359 \\
0.1 & 0.5398 & 0.5438 & 0.5478 & 0.5517 & 0.5557 & 0.5596 & 0.5636 & 0.5675 & 0.5714 & 0.5753 \\
0.2 & 0.5793 & 0.5832 & 0.5871 & 0.5910 & 0.5948 & 0.5987 & 0.6026 & 0.6064 & 0.6103 & 0.6141 \\
0.3 & 0.6179 & 0.6217 & 0.6255 & 0.6293 & 0.6331 & 0.6368 & 0.6406 & 0.6443 & 0.6480 & 0.6517 \\
0.4 & 0.6554 & 0.6591 & 0.6628 & 0.6664 & 0.6700 & 0.6736 & 0.6772 & 0.6808 & 0.6844 & 0.6879 \\
0.5 & 0.6915 & 0.6950 & 0.6985 & 0.7019 & 0.7054 & 0.7088 & 0.7123 & 0.7157 & 0.7190 & 0.7224 \\
0.6 & 0.7257 & 0.7291 & 0.7324 & 0.7357 & 0.7389 & 0.7422 & 0.7454 & 0.7486 & 0.7517 & 0.7549 \\
0.7 & 0.7580 & 0.7611 & 0.7642 & 0.7673 & 0.7704 & 0.7734 & 0.7764 & 0.7794 & 0.7823 & 0.7852 \\
0.8 & 0.7881 & 0.7910 & 0.7939 & 0.7967 & 0.7995 & 0.8023 & 0.8051 & 0.8078 & 0.8106 & 0.8133 \\
0.9 & 0.8159 & 0.8186 & 0.8212 & 0.8238 & 0.8264 & 0.8289 & 0.8315 & 0.8340 & 0.8365 & 0.8389 \\
1.0 & 0.8413 & 0.8438 & 0.8461 & 0.8485 & 0.8508 & 0.8531 & 0.8554 & 0.8577 & 0.8599 & 0.8621 \\
1.1 & 0.8643 & 0.8665 & 0.8686 & 0.8708 & 0.8729 & 0.8749 & 0.8770 & 0.8790 & 0.8810 & 0.8830 \\
1.2 & 0.8849 & 0.8869 & 0.8888 & 0.8907 & 0.8925 & 0.8944 & 0.8962 & 0.8980 & 0.8997 & 0.9015 \\
1.3 & 0.9032 & 0.9049 & 0.9066 & 0.9082 & 0.9099 & 0.9115 & 0.9131 & 0.9147 & 0.9162 & 0.9177 \\
1.4 & 0.9192 & 0.9207 & 0.9222 & 0.9236 & 0.9251 & 0.9265 & 0.9279 & 0.9292 & 0.9306 & 0.9319 \\
1.5 & 0.9332 & 0.9345 & 0.9357 & 0.9370 & 0.9382 & 0.9394 & 0.9406 & 0.9418 & 0.9429 & 0.9441 \\
1.6 & 0.9452 & 0.9463 & 0.9474 & 0.9484 & 0.9495 & 0.9505 & 0.9515 & 0.9525 & 0.9535 & 0.9545 \\
1.7 & 0.9554 & 0.9564 & 0.9573 & 0.9582 & 0.9591 & 0.9599 & 0.9608 & 0.9616 & 0.9625 & 0.9633 \\
1.8 & 0.9641 & 0.9649 & 0.9656 & 0.9664 & 0.9671 & 0.9678 & 0.9686 & 0.9693 & 0.9699 & 0.9706 \\
1.9 & 0.9713 & 0.9719 & 0.9726 & 0.9732 & 0.9738 & 0.9744 & 0.9750 & 0.9756 & 0.9761 & 0.9767 \\
2.0 & 0.9772 & 0.9778 & 0.9783 & 0.9788 & 0.9793 & 0.9798 & 0.9803 & 0.9808 & 0.9812 & 0.9817 \\
2.1 & 0.9821 & 0.9826 & 0.9830 & 0.9834 & 0.9838 & 0.9842 & 0.9846 & 0.9850 & 0.9854 & 0.9857 \\
2.2 & 0.9861 & 0.9864 & 0.9868 & 0.9871 & 0.9875 & 0.9878 & 0.9881 & 0.9884 & 0.9887 & 0.9890 \\
2.3 & 0.9893 & 0.9896 & 0.9898 & 0.9901 & 0.9904 & 0.9906 & 0.9909 & 0.9911 & 0.9913 & 0.9916 \\
2.4 & 0.9918 & 0.9920 & 0.9922 & 0.9925 & 0.9927 & 0.9929 & 0.9931 & 0.9932 & 0.9934 & 0.9936 \\
2.5 & 0.9938 & 0.9940 & 0.9941 & 0.9943 & 0.9945 & 0.9946 & 0.9948 & 0.9949 & 0.9951 & 0.9952 \\
2.6 & 0.9953 & 0.9955 & 0.9956 & 0.9957 & 0.9959 & 0.9960 & 0.9961 & 0.9962 & 0.9963 & 0.9964 \\
2.7 & 0.9965 & 0.9966 & 0.9967 & 0.9968 & 0.9969 & 0.9970 & 0.9971 & 0.9972 & 0.9973 & 0.9974 \\
2.8 & 0.9974 & 0.9975 & 0.9976 & 0.9977 & 0.9977 & 0.9978 & 0.9979 & 0.9979 & 0.9980 & 0.9981 \\
2.9 & 0.9981 & 0.9982 & 0.9982 & 0.9983 & 0.9984 & 0.9984 & 0.9985 & 0.9985 & 0.9986 & 0.9986 \\
3.0 & 0.9987 & 0.9987 & 0.9987 & 0.9988 & 0.9988 & 0.9989 & 0.9989 & 0.9989 & 0.9990 & 0.9990 \\
\hline
\end{array}

Let $X$ be an exponential random variable with parameter $\lambda$.

(a) For any positive integer $n$, prove that
\[E[X^n] = \frac{n}{\lambda} E[X^{n-1}].\]

(b) Find the expected value of $X$.

(c) Find the variance of $X$.

(d) Find the standard deviation of $X$.

Solution.

Solution of (a)

Recall that the probability density function $f(x)$ of an exponential random variable with parameter $\lambda$ is given by
\begin{align*}
f(x) =
\begin{cases}
\lambda e^{-\lambda x} & \text{ if } x \geq 0\\
0 & \text{ if } x < 0 \end{cases} \end{align*} and the parameter $\lambda$ must be positive. It follows from this and the definition of expectation, we get \begin{align*} E[X^n] &= \int_0^{\infty} x^n \cdot \lambda e^{-\lambda x} dx. \end{align*} Applying integral by parts with \[u = x^n, dv=\lambda e^{-\lambda x} dx\] and hence \[du = nx^{n-1}dx, v = -e^{-\lambda x},\] we obtain (from $\int u dv = uv - \int v du$) \begin{align*} E[X^n] &= \int_0^{\infty} x^n \cdot \lambda e^{-\lambda x} dx\\[6pt] &= \left[x^n \cdot (-e^{-\lambda x})\right]_0^{\infty} - \int_0^{\infty} (-e^{\lambda x}) \cdot nx^{n-1} dx\\[6pt] &= 0 + n \int_0^{\infty} e^{\lambda x} x^{n-1} dx\\[6pt] &= \frac{n}{\lambda} \int_0^{\infty} x^{n-1} \cdot \lambda e^{\lambda x} dx\\[6pt] &= \frac{n}{\lambda} E[X^{n-1}]. \end{align*} This proves the required equality \[E[X^n] = \frac{n}{\lambda} E[X^{n-1}].\]

Solution of (b)

The expected value $E[X]$ can be obtained from the formula we just proved in part (a) by substituting $n=1$. Thus, we have
\begin{align*}
E[X] = \frac{1}{\lambda} E[1] = \frac{1}{\lambda}.
\end{align*}

Solution of (c)

We calculate the variance using the formula
\[V(X) = E[X^2] – (E[X])^2.\] We know $E[X] = 1/\lambda$ from part (b). To compute $E[X^2]$, let $n=2$ in the formula in part (a). Then
\begin{align*}
E[X^2] &= \frac{2}{\lambda}E[X]\\[6pt] &= \frac{2}{\lambda} \cdot \frac{1}{\lambda}\\[6pt] &= \frac{2}{\lambda^2}.
\end{align*}
Combining these, we obtain
\begin{align*}
V(X) &= E[X^2] – (E[X])^2\\[6pt] &= \frac{2}{\lambda^2} – \frac{1}{\lambda^2}\\[6pt] & = \frac{1}{\lambda^2}.
\end{align*}
Therefore, the variance of $X$ is
\[V(X) = \frac{1}{\lambda^2}.\]

Solution of (d)

Taking the square root of the variance $V(X)$, we obtain the standard deviation
\[\sigma = \frac{1}{\lambda}.\] Click here if solved 5The post Expected Value and Variance of Exponential Random Variable first appeared on Problems in Mathematics.]]> https://yutsumura.com/expected-value-and-variance-of-exponential-random-variable/feed/ 0 7252 https://yutsumura.com/condition-that-a-function-be-a-probability-density-function/ https://yutsumura.com/condition-that-a-function-be-a-probability-density-function/#respond Fri, 07 Feb 2020 06:29:19 +0000 https://yutsumura.com/?p=7247 Let $c$ be a positive real number. Suppose that $X$ is a continuous random variable whose probability density function is given by \begin{align*} f(x) = \begin{cases} \frac{1}{x^3} & \text{ if } x \geq c\\...

Let $X$ and $Y$ be geometric random variables with parameter $p$, with $0 \leq p \leq 1$. Assume that $X$ and $Y$ are independent.

Let $n$ be an integer greater than $1$. Let $k$ be a natural number with $k\leq n$. Then prove the formula
\[P(X=k \mid X + Y = n) = \frac{1}{n-1}.\]

Hint.

If this problem is a bit abstract for you, then you might want to try the following problem, which is more concrete.

Probability that Alice Tossed a Coin Three Times If Alice and Bob Tossed Totally 7 Times

Solution.

The definition of a conditional probability yields
\[P(X=k \mid X + Y = n) = \frac{P\left((X=k) \cap (X+Y=n)\right)}{P(X+Y=n)} \tag{*}\] We first compute the numerator of the last expression. We have
\begin{align*}
&P\left((X=k) \cap (X+Y=n)\right)\\
&= P\left((X=k) \cap (Y=n-X)\right)\\
&= P\left((X=k) \cap (Y=n-k)\right)\\
&= P(X=k) \cdot P(Y=n-k).
\end{align*}
Here, the last step follows because $X$ and $Y$ are independent.

Now, as $X$ and $Y$ are geometric random variables with parameter $p$, we see that by putting $q=1-p$
\begin{align*}
P(X=k) &= q^{k-1}p \\
P(Y= n – k) &= q^{n-k-1}p.
\end{align*}
Hence, the numerator becomes
\begin{align*}
&P\left((X=k) \cap (X+Y=n)\right)\\
&= P(X=k) \cdot P(Y=n-k)\\
&= q^{k-1}p \cdot q^{n-k-1}p\\
&= q^{n-2}p^2.
\end{align*}

Next, we compute the denominator of (*). Applying the law of total probability, we get
\begin{align*}
P(X+Y=n) &= \sum_{k=1}^{n-1} P(X=k) P(X+Y=n \mid X=k). \tag{**}
\end{align*}
The probability $P(X+Y=n \mid X=k)$ can be computed as follows.
\begin{align*}
&P(X+Y=n \mid X=k)\\
&= P(Y = n-X \mid X=k)\\
&= P(Y=n-k \mid X=k)\\
&= P(Y=n-k).
\end{align*}
The last step follows because $X$ and $Y$ are independent. Then the equation (**) becomes
\begin{align*}
P(X+Y=n) &= \sum_{k=1}^{n-1} P(X=k) \cdot P(Y=n-k) \\ &= \sum_{k=1}^{n-1} q^{k-1}p \cdot q^{n-k-1}p\\
&= \sum_{k=1}^{n-1} q^{n-2} p^2\\
&= (n-1)q^{n-2}p^2.
\end{align*}

Combining the computations of the numerator and the denominator, we obtain the desired probability from (*).
\begin{align*}
P(X=k \mid X + Y = n) &= \frac{P\left((X=k) \cap (X+Y=n)\right)}{P(X+Y=n)}\\
&= \frac{q^{n-2}p^2}{(n-1)q^{n-2}p^2}\\
&= \frac{1}{n-1}.
\end{align*}
This completes the proof of the formula
\[P(X=k \mid X + Y = n) = \frac{1}{n-1}.\]

Application

As an application of this problem, solve the next problem.

Its solution is available in the post Probability that Alice Tossed a Coin Three Times If Alice and Bob Tossed Totally 7 Times.

We have a stick of a unit length. Two points on the stick will be selected randomly (uniformly along the length of the stick) and independently. Then we break the stick at these two points so that we get three pieces of the stick. What is the probability that these three pieces form a triangle?

Solution.

Let us call the left end of the stick the origin. Let $X$ be the length from the origin to the first selected point. Let $Y$ be the length from the origin to the second selected point.

Note that when $X=Y$, we have only two pieces and they cannot form a triangle, so we assume $X \neq Y$.
We first assume that $X < Y$. Then after breaking the stick, we have three pieces of length $X$, $Y-X$, and $1-Y$, respectively.

Now, let us consider a condition so that three segments of length $a$, $b$, and $c$ form a triangle in general. It is necessary and sufficient that the sum of the lengths of any two segments is bigger than the length of the rest to form a triangle.
Namely, we must satisfy all of the following inequalities.
\begin{align*}
a + b &> c\\
b + c &> a\\
c + a &> b.
\end{align*}

Applying this condition to the current situation, we must have
\begin{align*}
X + (Y-X) &> 1-Y\\
(Y-X) + (1-Y) & > X\\
(1- Y) + X &> Y – X.
\end{align*}
Simplifying these, we obtain
\begin{align*}
Y &> \frac{1}{2}\\
X &< \frac{1}{2}\\ Y &< X + \frac{1}{2}. \end{align*} The region in the unit square satisfying these inequality is depicted in the figure below (orange region).

Note that the joint density function of $X$ and $Y$ are uniformly distributed on the unit square. Thus, the probability that three pieces form a triangle under the assumption $X < Y$ is given by the area of the orange triangular region, which is $1/8$. Note that by symmetry, the case $X > Y$ gives the same probability. Hence, the desired probability is
\begin{align*}
&P(\text{form a triangle}) \\
&= P(\text{form a triangle} \mid X < Y) + P(\text{form a triangle} \mid X > Y)\\
&\frac{1}{8}+\frac{1}{8} = \frac{1}{4}.
\end{align*}

Alternatively, we can find the probability for the case $X > Y$ as follows. By symmetry argument, we simply need to exchange $X$ and $Y$ and obtain the conditions
\begin{align*}
X &> \frac{1}{2}\\
Y &< \frac{1}{2}\\ X &< Y + \frac{1}{2}. \end{align*} The blue region below shows the area surrounded by these inequalities. The area of this blue region is $1/8$. Thus, the total area is $1/8 + 1/8 = 1/4$ as before.

Let $c$ be a fixed positive number. Let $X$ be a random variable that takes values only between $0$ and $c$. This implies the probability $P(0 \leq X \leq c) = 1$. Then prove the next inequality about the variance $V(X)$.
\[V(X) \leq \frac{c^2}{4}.\]

Proof.

Recall that the variance $V(X)$ of a random variable $X$ can be computed using expected values as
\[V(X) = E[X^2] – \left(E[X]\right)^2.\] We try to find the upper bound $c^2/4$ of the right-hand side.

As we know $0\leq X \leq c$, we get
\begin{align*}
E[X^2] &= E[XX]\\
&\leq E[cX]\\
&= cE[X],
\end{align*}
where the last step follows since $c$ is a constant and by the linearity of the expected values.

It follows that
\begin{align*}
V(X) &= E[X^2] – \left(E[X]\right)^2\\
& \leq cE[X] – \left(E[X]\right)^2.
\end{align*}

For the sake of simplicity, let us put $z = E[X]$. Then the last expression is a quadratic function
\[-z^2 + cz\] with variable $z$. Note that the graph of the equation
\[-z^2 + cz = -z(z-c)\] is a parabola that opens downward. This attains the maximal value at its vertex, whose $z$-coordinate is the middle point of the $z$-intercept $z=0, c$.

Thus, the maximal value is obtained when $z = \frac{0 + c}{2} = \frac{c}{2}$ and it is
\begin{align*}
-\left(\frac{c}{2}\right)^2 + c \left(\frac{c}{2}\right) = \frac{c^2}{4}.
\end{align*}

Therefore, we have obtained the desired inequality
\[V(X) \leq \frac{c^2}{4}.\] Click here if solved 6The post Upper Bound of the Variance When a Random Variable is Bounded first appeared on Problems in Mathematics.]]> https://yutsumura.com/upper-bound-of-the-variance-when-a-random-variable-is-bounded/feed/ 0 7233 https://yutsumura.com/probability-that-alice-tossed-a-coin-three-times-if-alice-and-bob-tossed-totally-7-times/ https://yutsumura.com/probability-that-alice-tossed-a-coin-three-times-if-alice-and-bob-tossed-totally-7-times/#comments Sat, 01 Feb 2020 00:38:51 +0000 https://yutsumura.com/?p=7228 Alice tossed a fair coin until a head occurred. Then Bob tossed the coin until a head occurred. Suppose that the total number of tosses for Alice and Bob was $7$. Assuming that each...

There are two boxes containing red and blue balls. Let us call the boxes Box A and Box B. Each box contains the same number of red and blue balls. More specifically, Box A has 5 red balls and 5 blue balls. Box B has 20 red balls and 20 blue balls. You choose one box. Then draw two balls randomly from the chosen box without replacement, that is, you will not return the first ball into the box before picking up the second ball.

If you draw two balls with the same color, then you win. Otherwise, you lose. To maximize the chance of winning, which box should you pick?

Solution.

Let us consider the probability that you draw two red balls from the box you choose.
Since both boxes contain the same number of red and blue balls, the probability that your first ball is red is $1/2$ no matter which box you choose. More rigorously, if you choose Box A, then the probability that the first ball is red is $5/10 = 1/2$. Also, if you choose Box B, then the probability is $20/40=1/2$.

Thus, the difference (if any) comes from the probability of the second ball. Suppose that you choose Box A. Assume that your first ball was red. Then Box A currently contains 4 red balls and 5 blue ball. Hence the probability that the second ball is red is $4/9$.
Similarly, if you choose Box B and the first ball is red, then the probability that the second ball is red is $19/39$ since there are 19 red balls and 20 blue balls in Box B.

So, which probability is bigger? As we have
\[4/9 = 0.444\dots \text{ and } 19/39 = 0.487\cdots\] The probability for Box B is bigger. Thus, the probability of getting two red ball is higher when you choose Box B. The same argument applies for two blue balls. Therefore, to maximize the chance of winning, you should select Box B.

Remark.

Let us give an intuitive solution. This approach is not mathematically rigorous but it might help you reach the conclusion quicker.
First, you may think the actual numbers $5$ and $20$ are not so importance. What is importance is which number is bigger. So, instead, we consider extremely $2$ and $2,000,000,000$ (any gigantic number is fine). So Box A contains only 2 red balls and 2 blue balls. On the other hand Box B contains $2,000,000,000$ balls for each color.

As before the probability of getting the first red ball is the same regardless of your choice of a box. For the second ball, however, there is only one red ball remained in Box A, which is a half of the number of blue balls in Box A. On the other hand, Box B still contains 1,999,999,999 red balls and 2,000,000,000 blue balls. The ratio of two colors in Box B is almost 1. Thus, you have a better chance of getting the second red ball if you choose Box B.

A box of some snacks includes one of five toys. The chances of getting any of the toys are equally likely and independent of the previous results.

(a) Suppose that you buy the box until you complete all the five toys. Find the expected number of boxes that you need to buy.

(b) Find the variance and the standard deviation of the event in part (a).

Solution.

Solution of (a)

Let $X$ be the number of boxes that you need to buy until you complete all the five toys. Our goal is to compute the expected value $E[X]$.
To achieve this, we consider the next random variables. Let $X_i$ be the number of boxes you need to buy to get $i$th toy after getting $i-1$ toys. Then it is clear from definition that
\[X = X_1 + X_2 + X_3 + X_4 + X_5.\] For example, $X_1$ is the number of boxes you need to buy to get the first toy. Since whenever you open the first box, it is guaranteed that you get a new toy, we have $X_1 = 1$.
Also, to get the second toy after the first one, there are $4/5$ chance of getting new toy and $1/5$ chance of getting the same toy as the first one. Thus, $X_2$ is a geometric random variable with parameter $4/5$. We denote this as $X \sim G_{4/5}$. Similarly, we get
\[X_3 \sim G_{3/5}, \quad X_4 \sim G_{2/5}, \quad \text{ and } X_5 \sim G_{1/5}.\]

By the linearity of expectation, we have
\begin{align*}
E[X] &= E[X_1 + X_2 + X_3 + X_4 + X_5]\\
&= E[X_1] + E[X_2] + E[X_3] + E[X_4] + E[X_5]\\
&= E[1] + E[G_{4/5}] + E[G_{3/5}] + E[G_{2/5}] + E[G_{1/5}] \end{align*}
Now, the expected value of a geometric random variable $G_p$ is given by
\[E[G_p] = \frac{1}{p}.\] It follows that
\begin{align*}
E[X] &= 1 + \frac{5}{4} +\frac{5}{3} + \frac{5}{2} + \frac{5}{1}\\
&= 5\left(1+ \frac{1}{2} + \frac{1}{3} + \frac{1}{4} + \frac{1}{5}\right)\\
&\approx 11.41667
\end{align*}
Thus, the expected number of boxes that you need to buy to complete all the five toys is 11.41667.

Solution of (b)

Now we compute the variance of $X$. Recall that the variance of a geometric random variable $G_p$ is given by
\[V(G_p) = \frac{1-p}{p^2}.\] As we have seen that
\begin{align*}
X &= X_1 + X_2 + X_3 + X_4 + X_5\\
&\sim 1 + G_{4/5} + G_{3/5} + G_{2/5} + G_{1/5}
\end{align*}
and as each random variable $X_i$ is independent of each other, we obtain
\begin{align*}
V(X) &= V(1 + G_{4/5} + G_{3/5} + G_{2/5} + G_{1/5})\\[6pt] &= V(1) + V(G_{4/5}) + V(G_{3/5}) + V(G_{2/5}) + V(G_{1/5})\\[6pt] &= 0 + \frac{1-\frac{4}{5}}{\left(\frac{4}{5}\right)^2} + \frac{1-\frac{3}{5}}{\left(\frac{3}{5}\right)^2} + \frac{1-\frac{2}{5}}{\left(\frac{2}{5}\right)^2} + \frac{1-\frac{1}{5}}{\left(\frac{1}{5}\right)^2}\\[6pt] &\approx 25.17361
\end{align*}
Thus, the variance is $V(X) = 25.17361$.

The standard deviation is the square root of the variance. Hence, we obtain
\[\sigma(X) \approx \sqrt{25.17361} \approx 5.01733.\]

Remark.

This type of problems is called a Coupon Collecting Problem.