Rules of Probability Calculation

Here I summarise the rules of probability calculus. They have been stated above; here they are grouped together, so they can be found easily.

1. Probability is a number between 0 and 1: $0\leq P(x) \leq 1$
2. If the outcome is certain, the probability of that outcome is 1: If $P(x)$ is the probability for the variable $X$, $P(x)=1$ if and only if $X$ takes the value $x$.
3. Definition of conditional probability: The conditional probability is defined by this formula
   
   $$P(x\vert y) = \frac{P(x,y)}{P(y)}$$
   
   

4. The probability that a variable takes one of a set of mutually exclusive outcomes is the sum of the probability of the outcomes: $P(x_1 \mbox{ or } x_2 \mbox{ or } \dots \mbox{ or } x_n) = P(x_1) + P(x_2) + \dots P(x_n)$ if $X$ can take no more than one of the values $x_1, \dots, x_n$.
5. The probability of independent events is computed by multiplying the probability of each event: $P(x, y) = P(x)P(y)$ if $X$ taking the value $x$ does not depend whether or not $Y$ takes the value $y$.
6. Bayes Rule:
   
   $$P(y\vert x) = P(x\vert y)\frac{P(y)}{P(x)}$$
   
   

7. Bayes Rule (alternative form):
   
   $$P(y\vert x) = P(x\vert y)\frac{P(y)}{\sum_iP(x\vert y_i)P(y_i)}$$
   
   

8. Distribution of a single variable from the joint distribution: One can find the distribution for $X$ from the joint distribution of $X$ and $Y$ by summing over all values of $Y$,
   
   $$P(x) = \sum_{\mbox{all y}} P(x,y).$$
   
   
   The single distribution determined from the joint is called the marginal distribution; this process is called marginalisation. Note: It is not possible to infer the joint distribution from the marginals.