Let $X$ and $Y$ be geometric random variables with parameter $p$, with $0 \leq p \leq 1$. Assume that $X$ and $Y$ are independent.

Let $n$ be an integer greater than $1$. Let $k$ be a natural number with $k\leq n$. Then prove the formula
\[P(X=k \mid X + Y = n) = \frac{1}{n-1}.\]

Hint.

If this problem is a bit abstract for you, then you might want to try the following problem, which is more concrete.

Probability that Alice Tossed a Coin Three Times If Alice and Bob Tossed Totally 7 Times

Solution.

The definition of a conditional probability yields
\[P(X=k \mid X + Y = n) = \frac{P\left((X=k) \cap (X+Y=n)\right)}{P(X+Y=n)} \tag{*}\] We first compute the numerator of the last expression. We have
\begin{align*}
&P\left((X=k) \cap (X+Y=n)\right)\\
&= P\left((X=k) \cap (Y=n-X)\right)\\
&= P\left((X=k) \cap (Y=n-k)\right)\\
&= P(X=k) \cdot P(Y=n-k).
\end{align*}
Here, the last step follows because $X$ and $Y$ are independent.

Now, as $X$ and $Y$ are geometric random variables with parameter $p$, we see that by putting $q=1-p$
\begin{align*}
P(X=k) &= q^{k-1}p \\
P(Y= n – k) &= q^{n-k-1}p.
\end{align*}
Hence, the numerator becomes
\begin{align*}
&P\left((X=k) \cap (X+Y=n)\right)\\
&= P(X=k) \cdot P(Y=n-k)\\
&= q^{k-1}p \cdot q^{n-k-1}p\\
&= q^{n-2}p^2.
\end{align*}

Next, we compute the denominator of (*). Applying the law of total probability, we get
\begin{align*}
P(X+Y=n) &= \sum_{k=1}^{n-1} P(X=k) P(X+Y=n \mid X=k). \tag{**}
\end{align*}
The probability $P(X+Y=n \mid X=k)$ can be computed as follows.
\begin{align*}
&P(X+Y=n \mid X=k)\\
&= P(Y = n-X \mid X=k)\\
&= P(Y=n-k \mid X=k)\\
&= P(Y=n-k).
\end{align*}
The last step follows because $X$ and $Y$ are independent. Then the equation (**) becomes
\begin{align*}
P(X+Y=n) &= \sum_{k=1}^{n-1} P(X=k) \cdot P(Y=n-k) \\ &= \sum_{k=1}^{n-1} q^{k-1}p \cdot q^{n-k-1}p\\
&= \sum_{k=1}^{n-1} q^{n-2} p^2\\
&= (n-1)q^{n-2}p^2.
\end{align*}

Combining the computations of the numerator and the denominator, we obtain the desired probability from (*).
\begin{align*}
P(X=k \mid X + Y = n) &= \frac{P\left((X=k) \cap (X+Y=n)\right)}{P(X+Y=n)}\\
&= \frac{q^{n-2}p^2}{(n-1)q^{n-2}p^2}\\
&= \frac{1}{n-1}.
\end{align*}
This completes the proof of the formula
\[P(X=k \mid X + Y = n) = \frac{1}{n-1}.\]

Application

As an application of this problem, solve the next problem.

Its solution is available in the post Probability that Alice Tossed a Coin Three Times If Alice and Bob Tossed Totally 7 Times.

The post Conditional Probability When the Sum of Two Geometric Random Variables Are Known first appeared on Problems in Mathematics.

Alice tossed a fair coin until a head occurred. Then Bob tossed the coin until a head occurred. Suppose that the total number of tosses for Alice and Bob was $7$.

Assuming that each toss is independent of each other, what is the probability that Alice tossed the coin exactly three times?

Hint.

Let $X$ be the number of Alice’s tosses until she got a head.

Then $X$ is a geometric random variable with parameter $p=1/2$.
Recall that a geometric random variable can be interpreted as the number of trials until getting the first success and each trial is successful with probability $p$ and fails with probability $q=1-p$.

Note that
\[P(X=k)= q^{k-1} p = (1-p)^{k-1} p.\]

Solution.

Let $X$ be the number of Alice’s tosses until she got a head. Also, let $Y$ be the number of Bob’s tosses until he got a head. As each toss is an independent event and the coin is fair, the random variables $X$ and $Y$ are geometric with parameter $p=1/2$ (the probability of success(head) is $p=1/2$ and the probability of failure(tail) is $1-p=1/2$).

This question asks to find the conditional probability
\[P(X=3 \mid X+Y=7) = \frac{P\left((X=3) \cap (X+Y=7) \right)}{P(X+Y=7)}.\]

Let us first compute the numerator as follows.
\begin{align*}
P\left((X=3) \cap (X+Y=7) \right) &= P\left((X=3) \cap (Y=7-X) \right) \\
&= P\left((X=3) \cap (Y=4) \right) \\
&= P(X=3) \cdot P(Y=4)
\end{align*}
since $X$ and $Y$ are independent.

Now, as $X$ and $Y$ follow the geometric distribution with parameter $p$, we have
\begin{align*}
P(X=3) &= \left(\frac{1}{2}\right)^2 \frac{1}{2} = \frac{1}{2^3}\\
P(Y=4) &= \left(\frac{1}{2}\right)^3 \frac{1}{2} = \frac{1}{2^4}.
\end{align*}
The product of these gives the numerator $\frac{1}{2^7}$.

Next, let us calculate the denominator $P(X+Y=7)$.
The law of total probability yields
\begin{align*}
P(X+Y=7) = \sum_{k=1}^6 P(X=k) P(X+Y = 7 \mid X=k).
\end{align*}

As before, since $X$ is a geometric random variable with parameter $p=1/2$, we have
\[P(X=k) = \left(\frac{1}{2}\right)^{k-1} \frac{1}{2} = \frac{1}{2^k}.\]

The second factor $P(X+Y \mid X= k)$ can be computed as follows.
\begin{align*}
&P(X+Y = 7 \mid X= k)\\
&= P(Y = 7 – X \mid X = k)\\
&= P(Y = 7 – k \mid X = k)\\
&= P(Y=7 – k)\\
&= \left(\frac{1}{2}\right)^{(7-k)-1} \frac{1}{2} = \frac{1}{2^{7-k}}
\end{align*}
Here, the third equality follows because $X$ and $Y$ are independent.

It follows that the denominator becomes
\begin{align*}
P(X+Y=7) &= \sum_{k=1}^6 \frac{1}{2^k} \cdot \frac{1}{2^{7-k}}\\
&= \sum_{k=1}^6 \frac{1}{2^7}\\
&= \frac{6}{2^7}.
\end{align*}

Combining these computations, it follows that the desired conditional probability is
\begin{align*}
P(X=3 \mid X+Y=7) = \frac{1/2^7}{6/2^7}=\frac{1}{6}.
\end{align*}

Abstraction

If you solved this problem, then try the following problem as well. This is more abstract than the current problem but the idea is the same.

Its solution can be found in the post: Conditional Probability When the Sum of Two Geometric Random Variables Are Known

The post Probability that Alice Tossed a Coin Three Times If Alice and Bob Tossed Totally 7 Times first appeared on Problems in Mathematics.

A box of some snacks includes one of five toys. The chances of getting any of the toys are equally likely and independent of the previous results.

(a) Suppose that you buy the box until you complete all the five toys. Find the expected number of boxes that you need to buy.

(b) Find the variance and the standard deviation of the event in part (a).

Solution.

Solution of (a)

Let $X$ be the number of boxes that you need to buy until you complete all the five toys. Our goal is to compute the expected value $E[X]$.
To achieve this, we consider the next random variables. Let $X_i$ be the number of boxes you need to buy to get $i$th toy after getting $i-1$ toys. Then it is clear from definition that
\[X = X_1 + X_2 + X_3 + X_4 + X_5.\] For example, $X_1$ is the number of boxes you need to buy to get the first toy. Since whenever you open the first box, it is guaranteed that you get a new toy, we have $X_1 = 1$.
Also, to get the second toy after the first one, there are $4/5$ chance of getting new toy and $1/5$ chance of getting the same toy as the first one. Thus, $X_2$ is a geometric random variable with parameter $4/5$. We denote this as $X \sim G_{4/5}$. Similarly, we get
\[X_3 \sim G_{3/5}, \quad X_4 \sim G_{2/5}, \quad \text{ and } X_5 \sim G_{1/5}.\]

By the linearity of expectation, we have
\begin{align*}
E[X] &= E[X_1 + X_2 + X_3 + X_4 + X_5]\\
&= E[X_1] + E[X_2] + E[X_3] + E[X_4] + E[X_5]\\
&= E[1] + E[G_{4/5}] + E[G_{3/5}] + E[G_{2/5}] + E[G_{1/5}] \end{align*}
Now, the expected value of a geometric random variable $G_p$ is given by
\[E[G_p] = \frac{1}{p}.\] It follows that
\begin{align*}
E[X] &= 1 + \frac{5}{4} +\frac{5}{3} + \frac{5}{2} + \frac{5}{1}\\
&= 5\left(1+ \frac{1}{2} + \frac{1}{3} + \frac{1}{4} + \frac{1}{5}\right)\\
&\approx 11.41667
\end{align*}
Thus, the expected number of boxes that you need to buy to complete all the five toys is 11.41667.

Solution of (b)

Now we compute the variance of $X$. Recall that the variance of a geometric random variable $G_p$ is given by
\[V(G_p) = \frac{1-p}{p^2}.\] As we have seen that
\begin{align*}
X &= X_1 + X_2 + X_3 + X_4 + X_5\\
&\sim 1 + G_{4/5} + G_{3/5} + G_{2/5} + G_{1/5}
\end{align*}
and as each random variable $X_i$ is independent of each other, we obtain
\begin{align*}
V(X) &= V(1 + G_{4/5} + G_{3/5} + G_{2/5} + G_{1/5})\\[6pt] &= V(1) + V(G_{4/5}) + V(G_{3/5}) + V(G_{2/5}) + V(G_{1/5})\\[6pt] &= 0 + \frac{1-\frac{4}{5}}{\left(\frac{4}{5}\right)^2} + \frac{1-\frac{3}{5}}{\left(\frac{3}{5}\right)^2} + \frac{1-\frac{2}{5}}{\left(\frac{2}{5}\right)^2} + \frac{1-\frac{1}{5}}{\left(\frac{1}{5}\right)^2}\\[6pt] &\approx 25.17361
\end{align*}
Thus, the variance is $V(X) = 25.17361$.

The standard deviation is the square root of the variance. Hence, we obtain
\[\sigma(X) \approx \sqrt{25.17361} \approx 5.01733.\]

Remark.

This type of problems is called a Coupon Collecting Problem.

The post Coupon Collecting Problem: Find the Expectation of Boxes to Collect All Toys first appeared on Problems in Mathematics.