What is the Meaning of Probability?

What is the meaning of probabilistic statements; how do we interpret the statement that the
probability of a flipped coin coming up heads is ? There are two
interpretations. The most
commonly used interpretation is that the probability of an event is
the expected frequency that the event occurs. In other words, if you were to do
an experiment many times, the average number of times which you get a
particular result is the
probability of that result (more strictly, this ratio tends to the
probability of the result in the limit of a large number of
experiments). Probability used in this way is called * objective*
probability, as it is based on the view that probability can be measured
objectively via experiments. This is also called the frequency
interpretation. The objective interpretation is appropriate if there
is a definite probabilistic process which generates the outcomes. If you do a number of experiments, the number of times an
outcome occurs divided by the number of experiments is an estimate of
this true probability of the process. The estimate approaches the true probability as
the number of experiments goes to infinity.

For example of coin flipping, you may believe that the process is fundamentally random with fixed probability. To estimate the probability of a coin coming up heads, flip it 100 times, say, and take ratio of the number of heads to the total number of flips. For any particular trial, this would not come out exactly . Sometimes it would come out , sometimes , and so forth. However, as the number of flips got bigger and bigger, this ratio come closer and closer to , if that is the true probability.

Objective probabilities can be computed as frequencies. A common
situation is one in which you would like to compute the probability of
a particular outcome . Suppose there
are possible outcomes, all equally likely, and possible ways
of getting the outcome . Then

**Example:**- What is the probability of rolling a pair of dice so
that the dots on the upward faces total to 7?
There are 36 different rolls of the dice. There are 5 ways to total to 7. Thus, assuming that each outcome is equally likely, the probability is 5/36.

There is a second interpretation of probability, called * subjective
probability*. It will not be used much in this
course, but you may see it in later courses on artificial
intelligence. This interprets probability as degree of belief. In this
interpretation, when I say that the
probability that a flipped coin will come up heads it , I am not
describing the results of any experiments (necessarily). I am saying
that I believe that the coin is equally likely to come up heads as
tails. This interpretation makes sense of situations in which there
may be no way of getting an estimate of a ``true'' probability by
repeated trials. As examples, a doctor may say that the probability of
a treatment curing you is 80% or a meteorologist may say that the
probability of rain tomorrow is 60%. It is not possible to generate
many sick copies of you and measure the fraction cured, nor are we
able to generate
many copies of today and measure the frequency of rainy
tomorrows. These experts are telling you something about the strength
of their belief in a given outcome.

You may wonder why belief should behave like frequencies. Why should the same rules of calculation be used for belief as for objective probability? There is an argument which says that it is sensible for beliefs to behave this way. Take the case of the doctor. The doctor may require a very accurate measure of the likelihood of the outcome, because decisions will be made on the basis of it. For example, the amount of time and money will the doctor be willing to spend on treating you may depend on this likelihood. Thus, the doctor's beliefs may be measurable from this behaviour. If the doctor's beliefs satisfy the laws of probability, the doctor may spend money and time sensibly; if the beliefs violate basic probability rules, the doctor might not use time and money sensibly. (This notion is usually stated in terms of betting behaviour. The doctor's belief's can be determined by setting betting odds with the doctor; if these beliefs violate rules of probability, you can design a set of bets which will cause the doctor to lose money no matter what the outcomes. This is a so-called Dutch Book. See the text ``Artificial Intelligence'', by Ian Pratt, for a good exposition of this view.) Thus, probability theory gives a formalism for quantifying beliefs in a sensible way. More than that, it gives a way of combining beliefs with objective evidence.

Viewed this way, Bayes rule can be taken as a way of updating beliefs
due to new evidence. If is a belief in some classification or
prediction, etc and is the evidence, Bayes rule is expressed as

The prior, is your prior belief, that is, what you think the likelihood of each possibility is prior to receiving the evidence. This formula tells how you should change your belief after receiving the evidence . What might seem to be nice about this is that you can give a computer or an automated system ``beliefs'', and those beliefs can change based on new evidence. Whether humans actually update belief in this way is another matter. Whether humans