Estimating Discrete Probabilities

Estimating of probability of discrete states is easiest. Given the interpretation of probability discussed in section , it is reasonable to use the frequency of the outcome in the sample. In other words, the estimate of the probability of the outcome $x_i$ is

$$\hat{p}=n_i/n ,$$

where $n_i$ is the number of times the outcome $x_i$ occur in the sample, $n$ is the total number of points in the sample, and the   over the $p$ is to denote that it is an estimate of the true (unknown) probability $p$.

Example:

Consider flipping a thumbtack (a flat-headed drawing pin, like you would put on someone's chair during your school days). It can land with the point pointing up or with the point touching the ground. What is the probability that it lands pointy side up? To answer this, we can do some experiments. Suppose we flip a thumbtack 20 times, and it comes up pointy side up 13 times and pointy side down times. Our estimate of the probability of a thumbtack landing with pointy side up would be $13/20=0.65$.

It is important to emphasise that this is only an estimate of the true probability. Since there will always be fluctuations and variations in a finite sample, it is possible that the true probability is different from this estimate. For example, if the true probability was $1/2$ it would be possible to get this data, although this data would be less likely than data in which the frequency is closer to $1/2$. Our estimate is the best estimate from the data in the sense that the data would be less likely if the probability was different. Thus, in a sense, this estimate is the mostly likely value given the data. For this reason, it is called a maximum likelihood estimate. This can be made more precise, see section .