Total probability theorem

From LogicWiki

The Total Probability Theorem is a theorem of the Probability Calculus that is useful in solving many problems of probability. It is used when you need to determine the probability of a simple sentence but you are given only the probability of that sentence conditional on a collection of other sentences that form a partition. These conditional probabilities can be thought of as partial probabilities of the sentence you're interested in; what the total probability theorem does is combine them to determine the total probability of that sentence.

There are two versions of the total probability theorem: a simple version and a more general version. The simple version is a special case of the general version, so there really is only one version of the theorem. Sometimes it is easier to apply just the simple version, though, so we give both.

The Simple Total Probability Theorem

This version of the theorem states that for any sentences ${\displaystyle A,B,}$

The proof of this theorem relies on the fact that ${\displaystyle A}$ and ${\displaystyle (A\wedge B)\vee (A\wedge \neg B)}$ are logically equivalent (this can be seen by constructing a truth table).

By Rule 7, ${\displaystyle Pr(A)=Pr((A\wedge B)\vee (A\wedge \neg B))}$.

Now notice that ${\displaystyle A\wedge B}$ and ${\displaystyle A\wedge \neg B}$ are mutually exclusive (because ${\displaystyle B}$ and ${\displaystyle \neg B}$ are mutually exclusive), we can apply Rule 3 to get:

${\displaystyle Pr(A)=Pr(A\wedge B)+Pr(A\wedge \neg B)}$.

To deal with these conjunctions, we apply Rule 4 twice. This gives

${\displaystyle Pr(A\wedge B)=Pr(B)\cdot Pr(A|B)}$ and ${\displaystyle Pr(A\wedge \neg B)=Pr(\neg B)\cdot Pr(A|\neg B)}$.

Substituting these into the previous step gives the final result:

${\displaystyle Pr(A)=Pr(B)\cdot Pr(A|B)+Pr(\neg B)\cdot Pr(A|\neg B)}$.

Example of the Simple Theorem

Consider the following question: If it’s cloudy in the afternoon, there’s a 45% chance it will rain tonight. If it’s not cloudy in the afternoon, there’s a 5% chance it will rain tonight. There’s an 80% chance that it will be cloudy this afternoon. What is the probability of it raining tonight?

This question asks for the probability of a simple sentence - the probability of it raining tonight. You are given two conditional probabilities: If it’s cloudy in the afternoon, there’s a 45% chance it will rain tonight and If it’s not cloudy in the afternoon, there’s a 5% chance it will rain tonight. Letting ${\displaystyle C}$ abbreviate "it is cloudy this afternoon" and ${\displaystyle R}$ abbreviate "it will rain tonight," this gives us ${\displaystyle Pr(R|C)=0.45}$ and ${\displaystyle Pr(R|\neg C)=0.05}$ (these are the partial probabilities; what we want is the total probability ${\displaystyle Pr(R)}$).

You are also told that there’s an 80% chance that it will be cloudy this afternoon, which means ${\displaystyle Pr(C)=0.8}$. By Rule 5, this means that ${\displaystyle Pr(\neg C)=0.2}$. You now have everything you need to solve this problem using the total probability theorem.

${\displaystyle Pr(R)=Pr(C)\cdot Pr(R|C)+Pr(\neg C)\cdot Pr(R|\neg C)}$

${\displaystyle Pr(R)=0.8\cdot 0.45+0.2\cdot 0.05}$

${\displaystyle Pr(R)=0.37}$.

Thus, the probability of it raining tonight is 0.37.

Remember! The total probability theorem is used when you're asked to find the probability of a sentence and are given certain conditional probabilities of that sentence.

The General Total Probability Theorem

The simple version of the total probability theorem is used to determine the probability of a sentence that's distributed over another sentence and its negation. The general version of this theorem is used to determine the probability of a sentence that's distributed over a partition of sentences. Because a sentence and its negation are an example of a partition, the simple version of the total probability theorem is a special case of the general version.

For any sentence ${\displaystyle A}$ and any partition ${\displaystyle B_1,B_2,\dots ,B_n}$, this theorem states that:

It is proven analogously to the simple version, beginning with the fact that ${\displaystyle A}$ and ${\displaystyle (A\wedge B_1)\vee (A\wedge B_2)\vee \dots \vee (A\wedge B_n)}$ are logically equivalent (in virtue of ${\displaystyle B_1,\dots ,B_n}$ being a partition).

By Rule 7, ${\displaystyle Pr(A)=Pr((A\wedge B_1)\vee (A\wedge B_2)\vee \dots \vee (A\wedge B_n))}$.

Since each of these disjuncts is mutually exclusive with the others, we can apply Rule 3 ${\displaystyle n}$ times to get:

${\displaystyle Pr(A)=Pr(A\wedge B_1)+Pr(A\wedge B_2)+\dots +Pr(A\wedge B_n)}$

Then applying Rule 4 to these ${\displaystyle n}$ terms gives:

${\displaystyle Pr(A)=Pr(B_1)\cdot Pr(A|B_1)+Pr(B_2)\cdot Pr(A|B_2)+\dots +Pr(B_n)\cdot Pr(A|B_n)}$

Example of the General Theorem

You are planning on purchasing a new widget from the widget store. There are four different factories that produce widgets, located in California, Kansas, Minnesota, and Arizona. The California plant produces 400 widgets a month, Kansas produces 350, Minnesota produces 150 and Arizona produces 100. Widgets made in California have a 5% failure rate, those made in Kansas have a 20% failure rate, those made in Minnesota have a 12% failure rate, while those made in Arizona have a 30% failure rate. When you buy a widget, you don't know which factory it was produced in. What is the probability that the widget you buy will fail?

Since you're only buying one widget, and that widget must come from one of the four factories and cannot come from more than one, the following sentences form a partition:

"The widget was made in California," abbreviated as ${\displaystyle C}$

"The widget was made in Kansas," abbreviated as ${\displaystyle K}$

"The widget was made in Minnesota," abbreviated as ${\displaystyle M}$

"The widget was made in Arizona," abbreviated as ${\displaystyle A}$

From the production numbers, you can determine the probability of these sentences. Notice that there are 1000 widgets produced in a month, which tells you that:

${\displaystyle Pr(C)=0.4}$

${\displaystyle Pr(K)=0.35}$

${\displaystyle Pr(M)=0.15}$

${\displaystyle Pr(A)=0.1}$

You need to determine ${\displaystyle Pr(D)}$, where ${\displaystyle D}$ abbreviates "The widget is defective." You are given the following information from the failure rates:

${\displaystyle Pr(D|C)=0.05}$

${\displaystyle Pr(D|K)=0.2}$

${\displaystyle Pr(D|M)=0.12}$

${\displaystyle Pr(D|A)=0.3}$

Now it's just a matter of applying the general version of the theorem, which yields:

${\displaystyle Pr(D)=Pr(C)\cdot Pr(D|C)+Pr(K)\cdot Pr(D|K)+Pr(M)\cdot Pr(D|M)+Pr(A)\cdot Pr(D|A)}$

${\displaystyle Pr(D)=0.4\cdot 0.05+0.35\cdot 0.2+0.15\cdot 0.12+0.1\cdot 0.3}$

${\displaystyle Pr(D)=0.138}$

There is a 13.8% chance that the widget you buy will be defective.

See Also

Probability Spaces (for a picture of the total probability theorem), Bayes' Theorem (in which the total probability theorem is used)