1 Three proofs from week 2
All three results below were stated in week 2. The first two were set as exercises there; the odds form was stated without proof. Here are the proofs.
1.1 The law of total probability#
Principle 1 (The law of total probability)
Where B_1, \dots, B_n partition \Omega, and \Cr(B_i) > 0 for each i,
\Cr(A) = \Cr(B_1)\Cr(A \mid B_1) + \dots + \Cr(B_n)\Cr(A \mid B_n).
Proof
The B_i partition \Omega, so they cut A into the disjoint pieces A \cap B_1, \dots, A \cap B_n, as in the picture above. By finite additivity, the probability of A is the sum of the probabilities of those pieces:
\Cr(A) = \Cr(A \cap B_1) + \dots + \Cr(A \cap B_n).
Now take the definition of conditional probability, \Cr(A \mid B_i) = \Cr(A \cap B_i) / \Cr(B_i), and multiply through by \Cr(B_i). That gives an equation for each piece:
\Cr(A \cap B_i) = \Cr(B_i)\Cr(A \mid B_i).
Substituting that into the sum, once for each i, replaces every \Cr(A \cap B_i) by \Cr(B_i)\Cr(A \mid B_i):
\Cr(A) = \Cr(B_1)\Cr(A \mid B_1) + \dots + \Cr(B_n)\Cr(A \mid B_n),
which is what we wanted to show.1.2 Bayes’ theorem#
Principle 2 (Bayes’ theorem)
\Cr(A \mid B) = \frac{\Cr(B \mid A)\,\Cr(A)}{\Cr(B)}
\qquad \text{provided } \Cr(A), \Cr(B) > 0.
Proof
What we want to show:
\Cr(A \mid B) = \frac{\Cr(B \mid A)\,\Cr(A)}{\Cr(B)}.
Substitute the definition of \Cr(A \mid B) = \Cr(A \cap B) / \Cr(B) on the left:
\boldsymbol{\frac{\Cr(A \cap B)}{\Cr(B)}} = \frac{\Cr(B \mid A)\,\Cr(A)}{\Cr(B)}.
Substitute the definition of \Cr(B \mid A) = \Cr(A \cap B) / \Cr(A) on the right:
\frac{\Cr(A \cap B)}{\Cr(B)} = \frac{\boldsymbol{\big(\Cr(A \cap B) / \Cr(A)\big)}\,\Cr(A)}{\Cr(B)}.
\Cr(A) was dividing the numerator, so it belongs in the denominator:
\frac{\Cr(A \cap B)}{\Cr(B)} = \frac{\Cr(A \cap B)\,\Cr(A)}{\boldsymbol{\Cr(A)}\,\Cr(B)}.
\Cr(A) appears top and bottom, so it cancels:
\frac{\Cr(A \cap B)}{\Cr(B)} = \frac{\Cr(A \cap B)}{\Cr(B)}.
The two sides are now identical, so this last equation certainly holds. Every step was either a substitution of a definition or a rearrangement, so each line holds if and only if the line before it does. The chain therefore runs back up to the first line, which is what we wanted to show.
1.3 The odds form#
Week 2 defined the Bayes factor as \Cr(E \mid H_1) / \Cr(E \mid H_2): how much better the evidence E is explained by H_1 than by H_2. The odds form of Bayes’ theorem was stated there without proof:
\underbrace{\frac{\Cr(H_1 \mid E)}{\Cr(H_2 \mid E)}}_{\text{posterior odds}}
=
\underbrace{\frac{\Cr(E \mid H_1)}{\Cr(E \mid H_2)}}_{\text{Bayes factor}}
\times
\underbrace{\frac{\Cr(H_1)}{\Cr(H_2)}}_{\text{prior odds}}.
Proof
Start from the posterior odds:
\frac{\Cr(H_1 \mid E)}{\Cr(H_2 \mid E)}.
Substitute using Bayes’ theorem for \Cr(H_1 \mid E) = \Cr(E \mid H_1)\,\Cr(H_1) / \Cr(E):
\frac{\boldsymbol{\big(\Cr(E \mid H_1)\,\Cr(H_1) / \Cr(E)\big)}}{\Cr(H_2 \mid E)}.
\Cr(E) was dividing the numerator, so it belongs in the denominator:
\frac{\Cr(E \mid H_1)\,\Cr(H_1)}{\boldsymbol{\Cr(E)}\,\Cr(H_2 \mid E)}.
Substitute using Bayes’ theorem for \Cr(H_2 \mid E) = \Cr(E \mid H_2)\,\Cr(H_2) / \Cr(E):
\frac{\Cr(E \mid H_1)\,\Cr(H_1)}{\Cr(E)\,\boldsymbol{\big(\Cr(E \mid H_2)\,\Cr(H_2) / \Cr(E)\big)}}.
That \Cr(E) was dividing the denominator, so it belongs in the numerator:
\frac{\Cr(E \mid H_1)\,\Cr(H_1)\,\boldsymbol{\Cr(E)}}{\Cr(E)\,\Cr(E \mid H_2)\,\Cr(H_2)}.
\Cr(E) now appears top and bottom, so it cancels:
\frac{\Cr(E \mid H_1)\,\Cr(H_1)}{\Cr(E \mid H_2)\,\Cr(H_2)}.
Separate the two conditional credences from the two priors:
\boldsymbol{\frac{\Cr(E \mid H_1)}{\Cr(E \mid H_2)} \times \frac{\Cr(H_1)}{\Cr(H_2)}}.
The first of those factors is the Bayes factor and the second is the prior odds. So the posterior odds we started from are the Bayes factor times the prior odds, which is what we wanted to show.