Trouble viewing the formulas? You need a MathML compatible browser.

# Evidence

Revision as of 20:45, 11 September 2006; Current revision

Evidence (or the evidence for an hypothesis) refers to those facts discovered that make a given hypothesis more likely to be true. Evidence is important to determine what we should believe and how strongly, and it plays an essential role in the methods of science where evidence determines which theories are confirmed and how strongly they are confirmed.

## Evidence and Confirmation

(main article: Confirmation theory)

To determine the probability of an hypothesis H given some new evidence E, we use Bayes' theorem to determine $P\phantom{\rule{0}{0ex}}r\left(H|E\right)$. This tells us how likely H is given that E is true, but in confirmation theory we are interested in how much the probability of H was changed by E. That is, we are interested in the difference between $P\phantom{\rule{0}{0ex}}r\left(H|E\right)$ and $P\phantom{\rule{0}{0ex}}r\left(H\right)$; this motivated our definition of the degree of confirmation:

$\frac{P\phantom{\rule{0}{0ex}}r\left(H|E\right)}{P\phantom{\rule{0}{0ex}}r\left(H\right)}$

By Bayes' Theorem, we know that this fraction is equal to:

$\frac{P\phantom{\rule{0}{0ex}}r\left(E|H\right)}{P\phantom{\rule{0}{0ex}}r\left(E\right)}$

Looking at this term lets us see what makes good evidence. Good evidence is evidence that makes these fractions as high as possible, it confirms the hypothesis to the highest degree possible. Looking at this second fraction shows two factors that constitute good evidence.

First, we want $P\phantom{\rule{0}{0ex}}r\left(E|H\right)$ to be as high as possible. This conditional probability measures how probable the evidence is given that (or assuming that) the hypothesis is true. For this term to be high, the evidence must be something that the hypothesis says is almost certain to happen. In other words, the evidence observed should not be surprising if the hypothesis actually is true.

Second, we want $P\phantom{\rule{0}{0ex}}r\left(E\right)$ to be as low as possible. This probability is how probable the evidence is on its own, without knowing whether or not the hypothesis is true. For this term to be low, the evidence must be something we don't expect to happen, it must be surprising.

Putting these together yields the criterion for good evidence: if the evidence is surprising ($P\phantom{\rule{0}{0ex}}r\left(E\right)$ is low) but would not be surprising if we knew that the hypothesis was true (if $P\phantom{\rule{0}{0ex}}r\left(E|H\right)$ is high), then the evidence is good evidence for the hypothesis.

## Quantity and Variety of Evidence

From the above criterion of good evidence, we can justify certain evidentiary practices. Suppose you work for a pharmaceutical company and are in charge of testing a new, experimental drug. How do you design your tests? What sort of subjects do you test the drug on?

Would you test the drug on one person, or on one thousand people? The latter is better, but why? Simply because Pr(successful trial on 1000 people) is significantly smaller than Pr(successful trial on 1 person): it's a lot more surprising if the drug works on one thousand people than if it works on just one person. Since the thousand person trial yields more surprising evidence, it is the better evidence for testing the drug.

Likewise, you would want a diverse sample of people consisting of people of all ages, races, genders, and health conditions. This would be a better sample than a group of male undergraduate students, ages 18-24. Why? Because Pr(drug is successful on the diverse sample) is much smaller than Pr(drug is successful on homogeneous sample), and surprising evidence is better evidence.

In general, when choosing what evidence to test for (and hence what experiments to design), you want to find evidence that is the most surprising, but would not be surprising if your chosen hypothesis was true.

## Requirement of Total Evidence

The work above shows that the probability of an hypothesis is increased by supporting evidence, and decreased (perhaps even dropped to zero) by disconfirming evidence. For these reasons, it is important to base probability assignments on the totality of the evidence. This requirement is put in place to make probability assignments as accurate as possible.

Failure to follow this requirement (whether intentional or not) is a fallacy in reasoning. Consider an infomercial that promises to make you hundreds of thousands of dollars a year, because Brad here made over \$50,000 last week alone. What the informercials fail to tell you is that Brad's results are atypical, and that 98% of people in the program fail to make any money at all. The original argument offered by the informercial is thus fallacious because it neglects a crucial piece of evidence, evidence that in fact disconfims the stated conclusion of the infomercial.

It is therefore important to base your probability assignments on your total evidence. Doing so gives you the most accurate assignments possible, which is very important when making decisions (for example, do you spend your life savings on the fantastic money making opportunity offered by the informercial or not?)