The Binomial Distribution

"Bi" means "two" (like a bicycle has two wheels) ...
... so this is about things with two results.

bicycle
   

head tails dollar

Tossing a Coin:

We say the probability of the coin landing H is ½
And the probability of the coin landing T is ½

die

Throwing a Die:

We say the probability of a four is 1/6 (one of the six faces is a four)
And the probability of not four is 5/6 (five of the six faces are not a four)

Note that a die has 6 sides but here we look at only two cases: "four: yes" or "four: no"

Let's Toss a Coin!

Toss a fair coin three times ... what is the chance of getting two Heads?

Tossing a coin three times (H is for heads, T for Tails) can get any of these 8 outcomes:

HHH   coin headcoin headcoin head
HHT   coin headcoin headcoin tail
HTH   coin headcoin tailcoin head
HTT   coin headcoin tailcoin tail
THH   coin tailcoin headcoin head
THT   coin tailcoin headcoin tail
TTH   coin tailcoin tailcoin head
TTT   coin tailcoin tailcoin tail

Which outcomes do we want?

"Two Heads" could be in any order: "HHT", "THH" and "HTH" all have two Heads (and one Tail).

So 3 of the outcomes produce "Two Heads".

What is the probability of each outcome?

Each outcome is equally likely, and there are 8 of them, so each outcome has a probability of 1/8

So the probability of event "Two Heads" is:

Number of
outcomes we want
    Probability of
each outcome
 
3   ×   1/8   =  3/8

So the chance of getting Two Heads is 3/8

We used special words:

What About 3 Heads, etc?

The calculations are (P means "Probability of"):

We can write this in terms of a Random Variable, X, = "The number of Heads from 3 tosses of a coin":

And this is what it looks like as a Bar Graph:

binomial 1
It is symmetrical!

Making a Formula

Now imagine we want the chances of 5 heads in 9 tosses: to list all 512 outcomes will take a long time!

So let's make a formula.

 

In our previous example, how can we get the values 1, 3, 3 and 1 ?

pascals triangle

 

Well, they are actually in the third row of Pascal’s Triangle ... !

 

Can we make them using a formula?

Sure we can, and here it is:

binomial n choose k = n! / k!(n-k)!

It is often called "n choose k"

You can read more about it at Combinations and Permutations.

Let's try it:

Example: with 3 tosses, what are the chances of 2 Heads?

We have n=3 and k=2:

n!k!(n-k)! =3!2!(3-2)!
=3×2×12×1 × 1
= 3

So there are 3 outcomes that have "2 Heads"

(We knew that already, but now we have a formula for it.)

Let's use it for a harder question:

Example: with 9 tosses, what are the chances of 5 Heads?

We have n=9 and k=5:

n!k!(n-k)! =9!5!(9-5)!
=9×8×7×6×5×4×3×2×15×4×3×2×1 × 4×3×2×1
=126

So 126 of the outcomes will have 5 heads

 

And for 9 tosses there are a total of 29 = 512 outcomes, so we get the probability:

 

Number of
outcomes we want
  Probability of
each outcome
   
126   ×   1512   =   126512

So:

P(X=5)  =  126512  = 0.24609375 

About a 25% chance.

(Easier than listing them all.)

Bias!

So far the chances of success or failure have been equally likely.

But what if the coins are biased (land more on one side than another) or choices are not 50/50.

Example: You sell sandwiches. 70% of people choose chicken, the rest choose something else.

What is the probability of selling 2 chicken sandwiches to the next 3 customers?

This is just like the heads and tails example, but with 70/30 instead of 50/50.

Let's draw a tree diagram:

tree chicken other

The "Two Chicken" cases are highlighted.

The probabilities for "two chickens" all work out to be 0.147, because we are multiplying two 0.7s and one 0.3 in each case. In other words

0.147 = 0.7 × 0.7 × 0.3

Or, using exponents:

= 0.72 × 0.31

The 0.7 is the probability of each choice we want, call it p

The 2 is the number of choices we want, call it k

And we have (so far):

= pk × 0.31

The 0.3 is the probability of the opposite choice, so it is: 1−p

The 1 is the number of opposite choices, so it is: n−k

Which gives us:

= pk(1-p)(n-k)

Where

Example: (continued)

So we get:

pk(1-p)(n-k) =0.72(1-0.7)(3-2)
=0.72(0.3)(1)
=0.7 × 0.7 × 0.3
=0.147

which is what we got before, but now using a formula

Now we know the probability of each outcome is 0.147

But we need to include that there are three such ways it can happen: (chicken, chicken, other) or (chicken, other, chicken) or (other, chicken, chicken)

Example: (continued)

The total number of "two chicken" outcomes is:

n!k!(n-k)! =3!2!(3-2)!
=3×2×12×1 × 1
=3

And we get:

Number of
outcomes we want
    Probability of
each outcome
   
3   ×   0.147   =   0.441

 

So the probability of event "2 people out of 3 choose chicken" = 0.441

OK. That was a lot of work for something we knew already, but now we have a formula we can use for harder questions.

Example: Sam says "70% choose chicken, so 7 of the next 10 customers should choose chicken" ... what are the chances Sam is right?

So we have:

And we get:

pk(1-p)(n-k) =0.77(1-0.7)(10-7)
=0.77(0.3)(3)
=0.0022235661

That is the probability of each outcome.

 

And the total number of those outcomes is:

n!k!(n-k)!  =10!7!(10-7)!
=10×9×8×7×6×5×4×3×2×17×6×5×4×3×2×1 × 3×2×1
=10×9×83×2×1
=120

And we get:

Number of
outcomes we want
    Probability of
each outcome
   
120   ×   0.0022235661   =   0.266827932

 

So the probability of 7 out of 10 choosing chicken is only about 27%

 

Moral of the story: even though the long-run average is 70%, don't expect 7 out of the next 10.

Putting it Together

Now we know how to calculate how many:

n!k!(n-k)!

And the probability of each:

pk(1-p)(n-k)

When multiplied together we get:

Probability of k out of n ways:

P(k out of n) =  n!k!(n-k)! pk(1-p)(n-k)

The General Binomial Probability Formula

Important Notes:

Quincunx

quincunx

 

Have a play with the Quincunx (then read Quincunx Explained) to see the Binomial Distribution in action.

Throw the Die

die

A fair die is thrown four times. Calculate the probabilities of getting:

In this case n=4, p = P(Two) = 1/6

X is the Random Variable ‘Number of Twos from four throws’.

Substitute x = 0 to 4 into the formula:

P(k out of n) =  n!k!(n-k)! pk(1-p)(n-k)

Like this (to 4 decimal places):

Summary: "for the 4 throws, there is a 48% chance of no twos, 39% chance of 1 two, 12% chance of 2 twos, 1.5% chance of 3 twos, and a tiny 0.08% chance of all throws being a two (but it still could happen!)"

This time the Bar Graph is not symmetrical:

binomial 0 to 4 skewed
It is not symmetrical!

It is skewed because p is not 0.5

 

bike

Sports Bikes

Your company makes sports bikes. 90% pass final inspection (and 10% fail and need to be fixed).

What is the expected Mean and Variance of the 4 next inspections?

First, let's calculate all probabilities.

X is the Random Variable "Number of passes from four inspections".

Substitute x = 0 to 4 into the formula:

P(k out of n) =  n!k!(n-k)! pk(1-p)(n-k)

Like this:

Summary: "for the 4 next bikes, there is a tiny 0.01% chance of no passes, 0.36% chance of 1 pass, 5% chance of 2 passes, 29% chance of 3 passes, and a whopping 66% chance they all pass the inspection."

Mean, Variance and Standard Deviation

Let's calculate the Mean, Variance and Standard Deviation for the Sports Bike inspections.

There are (relatively) simple formulas for them. They are a little hard to prove, but they do work!

The mean, or "expected value", is:

μ = np

For the sports bikes:

μ = 4 × 0.9 = 3.6

So we can expect 3.6 bikes (out of 4) to pass the inspection.
Makes sense really ... 0.9 chance for each bike times 4 bikes equals 3.6

The formula for Variance is:

Variance: σ2 = np(1-p)

And Standard Deviation is the square root of variance:

σ = √(np(1-p))

For the sports bikes:

Variance: σ2 = 4 × 0.9 × 0.1 = 0.36

Standard Deviation is:

σ = √(0.36) = 0.6

 

Note: we could also calculate them manually, by making a table like this:

X P(X) X × P(X) X2 × P(X)
0 0.0001 0 0
1 0.0036 0.0036 0.0036
2 0.0486 0.0972 0.1944
3 0.2916 0.8748 2.6244
4 0.6561 2.6244 10.4976
  SUM: 3.6 13.32

The mean is the Sum of (X × P(X)):

μ = 3.6

The variance is the Sum of (X2 × P(X)) minus Mean2:

Variance: σ2 = 13.32 − 3.62 = 0.36

Standard Deviation is:

σ = √(0.36) = 0.6

And we got the same results as before (yay!)

 

Summary