Bernoulli's Theorem

From LogicWiki

Bernoulli's Theorem is signficant for frequentist interpretations of probability. According to this interpretation , the probability of a sentence represents the relative frequency of that sentence being true in the long run.

We can write the relative frequency of a sentence as k/n, where n is the number of trials we run and k is the number of outcomes that make the sentence true. So if we toss a fair coin 4 times and get HHTH, the relative frequency of tails is 1/4.

Then Bernoulli's Theorem says that for any sentence A, any small error ${\displaystyle ϵ}$, and any small difference x, there is a number of trials N, such that for any n > N, the following holds:

${\displaystyle Pr((Pr(A)-ϵ)\le k/n\le (Pr(A)+ϵ))>(1-x)}$

In other words, this theorem tells us that there is a certain number of trials for which the probability that the relative frequency will be within some margin of error of the actual probability Pr(A) of the sentence A is greater than 1 - x for however small we want to make x. More generally, what the theorem tells us is that relative frequency eventually approaches the actual probability. For this reason, Bernoulli's theorem serves as justification of the frequentist idea that probability can be thought of as frequency in the long run.

Let's fill this out with an example. In the example above we saw that the relative frequency of tails is 1/4. However, the probability of tails is 1/2 (because the coin is fair). Bernoulli's Theorem assures us that there is some number of trials that we can run so that the relative frequencyof tails will eventually approach the actual probability of tails (i.e. 1/2).

Bernoulli's Theorem does not tell us how big N is; that is, it does not tell us how long we have to wait before we can be quite certain that relative frequency will approach probability. But if you think about it, the theorem should not tell us that. For the size of N will vary on a case by case basis.