Trouble viewing the formulas? You need a MathML compatible browser.

# Total probability theorem

Revision as of 17:47, 17 April 2007; Current revision
←Older revision | Newer revision→

The Total Probability Theorem is a theorem of the Probability Calculus that is useful in solving many problems of probability. It is used when you need to determine the probability of a simple sentence but you are given only the probability of that sentence conditional on a collection of other sentences that form a partition. These conditional probabilities can be thought of as partial probabilities of the sentence you're interested in; what the total probability theorem does is combine them to determine the total probability of that sentence.

There are two versions of the total probability theorem: a simple version and a more general version. The simple version is a special case of the general version, so there really is only one version of the theorem. Sometimes it is easier to apply just the simple version, though, so we give both.

## The Simple Total Probability Theorem

This version of the theorem states that for any sentences $A,B,$

 $P\phantom{\rule{0}{0ex}}r\left(A\right)=P\phantom{\rule{0}{0ex}}r\left(B\right)\cdot P\phantom{\rule{0}{0ex}}r\left(A|B\right)+P\phantom{\rule{0}{0ex}}r\left(¬B\right)\cdot P\phantom{\rule{0}{0ex}}r\left(A|¬B\right)$

The proof of this theorem relies on the fact that $A$ and $\left(A\wedge B\right)\vee \left(A\wedge ¬B\right)$ are logically equivalent (this can be seen by constructing a truth table).

By Rule 7, $P\phantom{\rule{0}{0ex}}r\left(A\right)=P\phantom{\rule{0}{0ex}}r\left(\left(A\wedge B\right)\vee \left(A\wedge ¬B\right)\right)$.

Now notice that $A\wedge B$ and $A\wedge ¬B$ are mutually exclusive (because $B$ and $¬B$ are mutually exclusive), we can apply Rule 3 to get:

$P\phantom{\rule{0}{0ex}}r\left(A\right)=P\phantom{\rule{0}{0ex}}r\left(A\wedge B\right)+P\phantom{\rule{0}{0ex}}r\left(A\wedge ¬B\right)$.

To deal with these conjunctions, we apply Rule 4 twice. This gives

$P\phantom{\rule{0}{0ex}}r\left(A\wedge B\right)=P\phantom{\rule{0}{0ex}}r\left(B\right)\cdot P\phantom{\rule{0}{0ex}}r\left(A|B\right)$ and $P\phantom{\rule{0}{0ex}}r\left(A\wedge ¬B\right)=P\phantom{\rule{0}{0ex}}r\left(¬B\right)\cdot P\phantom{\rule{0}{0ex}}r\left(A|¬B\right)$.

Substituting these into the previous step gives the final result:

$P\phantom{\rule{0}{0ex}}r\left(A\right)=P\phantom{\rule{0}{0ex}}r\left(B\right)\cdot P\phantom{\rule{0}{0ex}}r\left(A|B\right)+P\phantom{\rule{0}{0ex}}r\left(¬B\right)\cdot P\phantom{\rule{0}{0ex}}r\left(A|¬B\right)$.

## Example of the Simple Theorem

Consider the following question: If it’s cloudy in the afternoon, there’s a 45% chance it will rain tonight. If it’s not cloudy in the afternoon, there’s a 5% chance it will rain tonight. There’s an 80% chance that it will be cloudy this afternoon. What is the probability of it raining tonight?

This question asks for the probability of a simple sentence - the probability of it raining tonight. You are given two conditional probabilities: If it’s cloudy in the afternoon, there’s a 45% chance it will rain tonight and If it’s not cloudy in the afternoon, there’s a 5% chance it will rain tonight. Letting $C$ abbreviate "it is cloudy this afternoon" and $R$ abbreviate "it will rain tonight," this gives us $P\phantom{\rule{0}{0ex}}r\left(R|C\right)=0.45$ and $P\phantom{\rule{0}{0ex}}r\left(R|¬C\right)=0.05$ (these are the partial probabilities; what we want is the total probability $P\phantom{\rule{0}{0ex}}r\left(R\right)$).

You are also told that there’s an 80% chance that it will be cloudy this afternoon, which means $P\phantom{\rule{0}{0ex}}r\left(C\right)=0.8$. By Rule 5, this means that $P\phantom{\rule{0}{0ex}}r\left(¬C\right)=0.2$. You now have everything you need to solve this problem using the total probability theorem.

$P\phantom{\rule{0}{0ex}}r\left(R\right)=P\phantom{\rule{0}{0ex}}r\left(C\right)\cdot P\phantom{\rule{0}{0ex}}r\left(R|C\right)+P\phantom{\rule{0}{0ex}}r\left(¬C\right)\cdot P\phantom{\rule{0}{0ex}}r\left(R|¬C\right)$

$P\phantom{\rule{0}{0ex}}r\left(R\right)=0.8\cdot 0.45+0.2\cdot 0.05$

$P\phantom{\rule{0}{0ex}}r\left(R\right)=0.37$.

Thus, the probability of it raining tonight is 0.37.

Remember! The total probability theorem is used when you're asked to find the probability of a sentence and are given certain conditional probabilities of that sentence.

## The General Total Probability Theorem

The simple version of the total probability theorem is used to determine the probability of a sentence that's distributed over another sentence and its negation. The general version of this theorem is used to determine the probability of a sentence that's distributed over a partition of sentences. Because a sentence and its negation are an example of a partition, the simple version of the total probability theorem is a special case of the general version.

For any sentence $A$ and any partition ${B}_{1},{B}_{2},\dots ,{B}_{n}$, this theorem states that:

 $P\phantom{\rule{0}{0ex}}r\left(A\right)=P\phantom{\rule{0}{0ex}}r\left({B}_{1}\right)\cdot P\phantom{\rule{0}{0ex}}r\left(A|{B}_{1}\right)+P\phantom{\rule{0}{0ex}}r\left({B}_{2}\right)\cdot P\phantom{\rule{0}{0ex}}r\left(A|{B}_{2}\right)+\dots +P\phantom{\rule{0}{0ex}}r\left({B}_{n}\right)\cdot P\phantom{\rule{0}{0ex}}r\left(A|{B}_{n}\right)$

It is proven analogously to the simple version, beginning with the fact that $A$ and $\left(A\wedge {B}_{1}\right)\vee \left(A\wedge {B}_{2}\right)\vee \dots \vee \left(A\wedge {B}_{n}\right)$ are logically equivalent (in virtue of ${B}_{1},\dots ,{B}_{n}$ being a partition).

By Rule 7, $P\phantom{\rule{0}{0ex}}r\left(A\right)=P\phantom{\rule{0}{0ex}}r\left(\left(A\wedge {B}_{1}\right)\vee \left(A\wedge {B}_{2}\right)\vee \dots \vee \left(A\wedge {B}_{n}\right)\right)$.

Since each of these disjuncts is mutually exclusive with the others, we can apply Rule 3 $n$ times to get:

$P\phantom{\rule{0}{0ex}}r\left(A\right)=P\phantom{\rule{0}{0ex}}r\left(A\wedge {B}_{1}\right)+P\phantom{\rule{0}{0ex}}r\left(A\wedge {B}_{2}\right)+\dots +P\phantom{\rule{0}{0ex}}r\left(A\wedge {B}_{n}\right)$

Then applying Rule 4 to these $n$ terms gives:

$P\phantom{\rule{0}{0ex}}r\left(A\right)=P\phantom{\rule{0}{0ex}}r\left({B}_{1}\right)\cdot P\phantom{\rule{0}{0ex}}r\left(A|{B}_{1}\right)+P\phantom{\rule{0}{0ex}}r\left({B}_{2}\right)\cdot P\phantom{\rule{0}{0ex}}r\left(A|{B}_{2}\right)+\dots +P\phantom{\rule{0}{0ex}}r\left({B}_{n}\right)\cdot P\phantom{\rule{0}{0ex}}r\left(A|{B}_{n}\right)$

## Example of the General Theorem

You are planning on purchasing a new widget from the widget store. There are four different factories that produce widgets, located in California, Kansas, Minnesota, and Arizona. The California plant produces 400 widgets a month, Kansas produces 350, Minnesota produces 150 and Arizona produces 100. Widgets made in California have a 5% failure rate, those made in Kansas have a 20% failure rate, those made in Minnesota have a 12% failure rate, while those made in Arizona have a 30% failure rate. When you buy a widget, you don't know which factory it was produced in. What is the probability that the widget you buy will fail?

Since you're only buying one widget, and that widget must come from one of the four factories and cannot come from more than one, the following sentences form a partition:

"The widget was made in California," abbreviated as $C$

"The widget was made in Kansas," abbreviated as $K$

"The widget was made in Minnesota," abbreviated as $M$

"The widget was made in Arizona," abbreviated as $A$

From the production numbers, you can determine the probability of these sentences. Notice that there are 1000 widgets produced in a month, which tells you that:

$P\phantom{\rule{0}{0ex}}r\left(C\right)=0.4$

$P\phantom{\rule{0}{0ex}}r\left(K\right)=0.35$

$P\phantom{\rule{0}{0ex}}r\left(M\right)=0.15$

$P\phantom{\rule{0}{0ex}}r\left(A\right)=0.1$

You need to determine $P\phantom{\rule{0}{0ex}}r\left(D\right)$, where $D$ abbreviates "The widget is defective." You are given the following information from the failure rates:

$P\phantom{\rule{0}{0ex}}r\left(D|C\right)=0.05$

$P\phantom{\rule{0}{0ex}}r\left(D|K\right)=0.2$

$P\phantom{\rule{0}{0ex}}r\left(D|M\right)=0.12$

$P\phantom{\rule{0}{0ex}}r\left(D|A\right)=0.3$

Now it's just a matter of applying the general version of the theorem, which yields:

$P\phantom{\rule{0}{0ex}}r\left(D\right)=P\phantom{\rule{0}{0ex}}r\left(C\right)\cdot P\phantom{\rule{0}{0ex}}r\left(D|C\right)+P\phantom{\rule{0}{0ex}}r\left(K\right)\cdot P\phantom{\rule{0}{0ex}}r\left(D|K\right)+P\phantom{\rule{0}{0ex}}r\left(M\right)\cdot P\phantom{\rule{0}{0ex}}r\left(D|M\right)+P\phantom{\rule{0}{0ex}}r\left(A\right)\cdot P\phantom{\rule{0}{0ex}}r\left(D|A\right)$

$P\phantom{\rule{0}{0ex}}r\left(D\right)=0.4\cdot 0.05+0.35\cdot 0.2+0.15\cdot 0.12+0.1\cdot 0.3$

$P\phantom{\rule{0}{0ex}}r\left(D\right)=0.138$

There is a 13.8% chance that the widget you buy will be defective.