Trouble viewing the formulas? You need a MathML compatible browser.

Bayes' theorem

From LogicWiki

Bayes' Theorem is a result of the probability calculus that allows you to calculate conditional probabilities on the basis of the "reverse" conditional probability. That is, Bayes' Theorem lets you calculate $P r (A | B)$ if you know $P r (B | A)$ and some other information.

Bayes' Theorem is incredibly useful in problems where you learn some new information (gain some new evidence) and have to update probabilities to account for this new information.

The theorem is named after Reverend Thomas Bayes (1702 - 1761), a Presbyterian minister and Fellow of the Royal Society of London.

1 Statement of the Theorem
2 Example
3 Bigger Versions
4 Example
5 Prior and Posterior Probabilities
6 See Also

[edit]

Statement of the Theorem

The simplest version of Bayes' Theorem is the following (for any sentences $A, B$ ):

P r (A | B) = \frac{P r (B | A) \cdot P r (A)}{P r (B)}

This is a simple consequence of Rule 4 of the probability calculus. Rule 4 says that:

$P r (A \land B) = P r (A) \cdot P r (B | A)$

and

$P r (B \land A) = P r (B) \cdot P r (A | B)$

Since $P r (A \land B) = P r (B \land A)$ (by Rule 7), we have:

$P r (A) \cdot P r (B | A) = P r (B) \cdot P r (A | B)$

Dividing each side by Pr(B) gives:

$P r (A | B) = \frac{P r (A) \cdot P r (B | A)}{P r (B)}$

Which is precisely Bayes' Theorem.

[edit]

Example

Here is a typical case for Bayes' Theorem. I suspect that my mechanic is a crook, in fact I think it's 25% likely that he is. If he is a crook, there's a 90% chance that he will install a broken muffler in my car. The unconditioned probability that my muffler will be broken is 0.3. Suppose that it is in fact broken; how likely is it that my mechanic actually is a crook?

In this problem, we gain some new evidence (we observe a broken muffler). This should change the probability assigned to "the mechanic is a crook." But how much should it change? That is where Bayes' Theorem comes in.

We know:

$P r (m e c h a n i c i s a c r o o k) = 0.25$

$P r (m u f f l e r i s b r o k e n | m e c h a n i c i s a c r o o k) = 0.9$

$P r (m u f f l e r i s b r o k e n) = 0.3$

We want:

$P r (m e c h a n i c i s a c r o o k | m u f f l e r i s b r o k e n)$ .

By Bayes' Theorem, this is:

$\frac{P r (m e c h a n i c i s a c r o o k) \cdot P r (m u f f l e r i s b r o k e n | m e c h a n i c i s a c r o o k)}{P r (m u f f l e r i s b r o k e n)}$

$= \frac{0.25 \cdot 0.9}{0.3} = 0.75$

Hence,

$P r (m e c h a n i c i s a c r o o k | m u f f l e r i s b r o k e n) = 0.75$ .

[edit]

Bigger Versions

Often, in Bayes' Theorem problems, the value of the denominator - $P r (B)$ in our statement above - is not given directly. In such cases, the total probability theorem is used. This two-step process can be combined into one, yielding bigger versions of Bayes' Theorem: one corresponding to the simple version of the total probability theorem, and the other to the general version.

P r (A | B) = \frac{P r (B | A) \cdot P r (A)}{P r (A) \cdot P r (B | A) + P r (\neg A) \cdot P r (B | \neg A)}

And, if $A_{1}, A_{2}, \dots A_{n}$ is a partition, then

P r (A | B) = \frac{P r (B | A) \cdot P r (A)}{P r (A_{1}) \cdot P r (B | A_{1}) + P r (A_{2}) \cdot P r (B | A_{2}) + \dots + P r (A_{n}) \cdot P r (B | A_{n})}

These versions result from substituting the appropriate version of the total probability theorem into the first version of Bayes' Theorem.

[edit]

Example

Here's a typical Bayes' Theorem problem. I present you with two urns, labelled 1 and 2. Urn 1 contains 1 red ball and 9 black balls, while urn 2 contains 9 red balls and 1 black ball. I'm going to first choose an urn by rolling a six-sided die: if it comes up 3, I will draw a ball at random from urn 1, anything else and I will draw from urn 2. Without showing you which urn I drew from, I present you with a black ball. How likely is it that I drew from urn 1?

Initially it seems more likely that I drew from urn 1 because it is mainly black balls, but the die-rolling changes things. How much? Let's use Bayes' Theorem.

We are given:

$P r (d r e w f r o m 1) = 1 / 6$

$P r (d r e w f r o m 2) = 5 / 6$

$P r (d r e w b l a c k | d r e w f r o m 1) = 0.9$

$P r (d r e w b l a c k | d r e w f r o m 2) = 0.1$

We want:

$P r (d r e w f r o m 1 | d r e w b l a c k)$

(probabilities like this are sometimes called "inverse probabilities" as they calculate "backwards." They tell you how likely a past event was given that a present event happened, whereas something like "what's the probability that a black ball is drawn from urn 2" calculates "forwards" from picking the urn to drawing the ball)

According to Bayes' Theorem:

$P r (d r e w f r o m 1 | d r e w b l a c k)$

$\frac{P r (d r e w b l a c k | d r e w f r o m 1) \cdot P r (d r e w f r o m 1)}{P r (d r e w f r o m 1) \cdot P r (d r e w b l a c k | d r e w f r o m 1) + P r (d r e w f r o m 2) \cdot P r (d r e w b l a c k | d r e w f r o m 2)}$

$= \frac{0.9 \cdot 1 / 6}{0.9 \cdot 1 / 6 + 0.1 \cdot 5 / 6}$

$= 0.265$

So the probability that I drew from urn 1 is only 0.265. If you think about it, this tells you that, since I showed you a black ball, the probability that I rolled a 3 on the six-sided die is 0.265, which is kind of interesting.

Another classic example along these lines is the Monty Hall problem.

[edit]

Prior and Posterior Probabilities

Sometimes the result of Bayes' Theorem - $P r (A | B)$ - is called the posterior probability of $A$ , while $P r (A)$ is called the prior probability of $A$ . This is usually done when the problem involves updating a probability on the basis of some new evidence: $P r (A)$ is the probability of $A$ before learning evidence $B$ (hence "prior") while $P r (A | B)$ is the probability of $A$ after learning evidence $B$ (hence "posterior").

This is particularly relevant when thinking of probabilities as degrees of belief. Bayes' Theorem tells you the right way to update your degrees of belief when you learn new evidence. A prime example of why this is imporant is the medical test.

[edit]

Bayes' theorem

From LogicWiki

Contents

Statement of the Theorem

Example

Bigger Versions

Example

Prior and Posterior Probabilities

See Also

Views

Personal tools

Navigation

Search

Toolbox