# AP Statistics Curriculum 2007 Bayesian Prelim

(Difference between revisions)
 Revision as of 20:44, 23 July 2009 (view source)DaveZes (Talk | contribs)← Older edit Revision as of 22:24, 22 October 2009 (view source)IvoDinov (Talk | contribs) (minor formatting and added examples)Newer edit → Line 1: Line 1: - '''Bayes Theorem''' + ==[[EBook | Probability and Statistics Ebook]] - Bayes Theorem== + ===Introduction=== Bayes theorem, or "Bayes Rule" can be stated succinctly by the equality Bayes theorem, or "Bayes Rule" can be stated succinctly by the equality - $P(A|B) = \frac{P(B|A) \cdot P(A)} {P(B)}$ + : $P(A|B) = \frac{P(B|A) \cdot P(A)} {P(B)}$ In words, "the probability of event A occurring given that event B occurred is equal to the probability of event B occurring given that event A occurred times the probability of event A occurring divided by the probability that event B occurs." In words, "the probability of event A occurring given that event B occurred is equal to the probability of event B occurring given that event A occurred times the probability of event A occurring divided by the probability that event B occurs." Line 11: Line 12: $f(Y|X) = \frac{f(X|Y) \cdot f(Y)} { f(X) }$ $f(Y|X) = \frac{f(X|Y) \cdot f(Y)} { f(X) }$ + ===Example=== + Suppose a Laboratory blood test is used as evidence for a disease. Assume P(positive Test| Disease) = 0.95, P(positive Test| no Disease)=0.01 and P(Disease) = 0.005. Find P(Disease|positive Test)=? + + Denote D = {the test person has the disease}, $D^c$ = {the test person does not have the disease} and  T = {the test result is positive}. Then +
$P(D | T) = {P(T | D) P(D) \over P(T)} = {P(T | D) P(D) \over P(T|D)P(D) + P(T|D^c)P(D^c)}=$ + $={0.95\times 0.005 \over {0.95\times 0.005 +0.01\times 0.995}}=0.193.$
+ + ===Bayesian statstics=== What is commonly called '''Bayesian Statistics''' is a very special application of Bayes Theorem. What is commonly called '''Bayesian Statistics''' is a very special application of Bayes Theorem. Line 17: Line 26: Using Bayes Theorem we may write Using Bayes Theorem we may write - $f(\mu|\mathbf{x}) = \frac{f(\mathbf{x}|\mu) \cdot f(\mu)} { f(\mathbf{x}) }$ + : $f(\mu|\mathbf{x}) = \frac{f(\mathbf{x}|\mu) \cdot f(\mu)} { f(\mathbf{x}) }$ In this formulation, we solve for $f(\mu|\mathbf{x})$, the "posterior" density of the population parameter, $\mu$. In this formulation, we solve for $f(\mu|\mathbf{x})$, the "posterior" density of the population parameter, $\mu$. Line 26: Line 35: $f(\mathbf{x}) = \int_{\mu} f( \mathbf{x} \cap \mu) d\mu = \int_{\mu} f( \mathbf{x} | \mu ) f(\mu) d\mu$ $f(\mathbf{x}) = \int_{\mu} f( \mathbf{x} \cap \mu) d\mu = \int_{\mu} f( \mathbf{x} | \mu ) f(\mu) d\mu$ + + ==See also== + * [[EBook#Chapter_III:_Probability |Probability Chapter]] + + ==References== + +

## Probability and Statistics Ebook - Bayes Theorem

### Introduction

Bayes theorem, or "Bayes Rule" can be stated succinctly by the equality $P(A|B) = \frac{P(B|A) \cdot P(A)} {P(B)}$

In words, "the probability of event A occurring given that event B occurred is equal to the probability of event B occurring given that event A occurred times the probability of event A occurring divided by the probability that event B occurs."

Bayes Theorem can also be written in terms of densities or likelihood functions over continuous random variables. Let's call $f(\star)$ the density (or in some cases, the likelihood) defined by the random process $\star$. If X and Y are random variables, we can say $f(Y|X) = \frac{f(X|Y) \cdot f(Y)} { f(X) }$

### Example

Suppose a Laboratory blood test is used as evidence for a disease. Assume P(positive Test| Disease) = 0.95, P(positive Test| no Disease)=0.01 and P(Disease) = 0.005. Find P(Disease|positive Test)=?

Denote D = {the test person has the disease}, Dc = {the test person does not have the disease} and T = {the test result is positive}. Then $P(D | T) = {P(T | D) P(D) \over P(T)} = {P(T | D) P(D) \over P(T|D)P(D) + P(T|D^c)P(D^c)}=$ $={0.95\times 0.005 \over {0.95\times 0.005 +0.01\times 0.995}}=0.193.$

### Bayesian statstics

What is commonly called Bayesian Statistics is a very special application of Bayes Theorem.

We will examine a number of examples in this Chapter, but to illustrate generally, imagine that x is a fixed collection of data that has been realized from under some known density, f(X), that takes a parameter, μ, whose value is not certainly known.

Using Bayes Theorem we may write $f(\mu|\mathbf{x}) = \frac{f(\mathbf{x}|\mu) \cdot f(\mu)} { f(\mathbf{x}) }$

In this formulation, we solve for $f(\mu|\mathbf{x})$, the "posterior" density of the population parameter, μ.

For this we utilize the likelihood function of our data given our parameter, $f(\mathbf{x}|\mu)$, and, importantly, a density f(μ), that describes our "prior" belief in μ.

Since $\mathbf{x}$ is fixed, $f(\mathbf{x})$ is a fixed number -- a "normalizing constant" so to ensure that the posterior density integrates to one. $f(\mathbf{x}) = \int_{\mu} f( \mathbf{x} \cap \mu) d\mu = \int_{\mu} f( \mathbf{x} | \mu ) f(\mu) d\mu$