Skip to main content

Section 3.5 Sampling Distributions and the Central Limit Theorem

What Are Sampling Distributions?

Starting with our next section, we are going to be looking at how to make inferences (i.e. estimations or decisions) about a population based on what we observe in a sample. It is therefore important that we understand how sampling works, and in particular, how the sample mean and sample proportions work. Recall that the mean of s sample containing values \(x_1, x_2, \ldots, x_n\) is computed as follows.

\begin{equation*} \overline{x} = \frac{x_1+x_2+\cdots+ x_n}{n} \end{equation*}

This formula should look familiar from Subsection 1.3.4. We restate it here, however, to consider a question. If we select a sample of \(n\) values and look at their average over and over again, will we see more or less variation between these averages than we would if we look at the individual values in the population? We shall see that there is in fact less variation between the means of different samples than between individual values. This is because it is a lot harder to get a group of \(n\) individuals who all vary in the same way from the population average than it is to get a single individual who varies from the population average.

Besides taking the average of some variable in a sample, we may also wish to examine the proportion of individuals in a sample who have some characteristic. This is similar to the binomial processes we studied in Section 3.2. The only real difference is that instead of looking at \(x\text{,}\) the number of successes, we look at \(\frac{x}{n}\text{,}\) the proportion of successes.

Definition 3.5.1.

If a sample of size \(n\) is drawn from a population and \(x\) of those \(n\) individuals have some characteristic, then the sample proportion with that characteristic is:

\begin{equation*} \hat p = \frac{x}{n}\text{.} \end{equation*}

In this section, we will try to develop our understanding of how these sample statistics, \(\overline x\) and \(\hat p\) work. In particular, we will introduce an extremely important theorem called the central limit theorem.

Subsection 3.5.1 Sample Means

To better understand how means of samples behave, let's take a look at a sampling example.

You wish to draw samples of size three from a finite population of values \(\lbrace 0, 3, 6, 9, 12 \rbrace\text{.}\)

  1. What are the possible samples of size three and what are their sample means?

  2. If \(\overline{X}\) is a random variable whose value is the mean of the sample, give the probability distribution for \(\overline{X}\text{.}\)

Solution

  1. Our first task is to list all possible samples of size three and compute their means. We know that there are \(C(5,3) = 10\) of them, and they are:

    Sample Sample Mean (\(\overline x\))
    \(\lbrace 0, 3, 6 \rbrace\) \(3\)
    \(\lbrace 0, 3, 9 \rbrace\) \(4\)
    \(\lbrace 0, 3, 12 \rbrace\) \(5\)
    \(\lbrace 0, 6, 9 \rbrace\) \(5\)
    \(\lbrace 0, 6, 12 \rbrace\) \(6\)
    \(\lbrace 0, 9, 12 \rbrace\) \(7\)
    \(\lbrace 3, 6, 9 \rbrace\) \(6\)
    \(\lbrace 3, 6, 12 \rbrace\) \(7\)
    \(\lbrace 3, 9, 12 \rbrace\) \(8\)
    \(\lbrace 6, 9, 12 \rbrace\) \(9\)
    Table 3.5.3. Samples and their Means

  2. Our final task is to construct a probability distribution for \(\overline X\text{,}\) the sample means. This is done by counting the number of samples that have each value and dividing by the total of \(10\) possible samples. While we are at this, we'll also find the expected value of \(\overline X\text{.}\)

    \(\overline x\) \(P\left(\overline X = \overline x\right)\) \(\overline x \times P\left(\overline X = \overline x\right)\)
    \(3\) \(\frac{1}{10}\) \(\frac{3}{10}\)
    \(4\) \(\frac{1}{10}\) \(\frac{4}{10}\)
    \(5\) \(\frac{2}{10}\) \(\frac{10}{10}\)
    \(6\) \(\frac{2}{10}\) \(\frac{12}{10}\)
    \(7\) \(\frac{2}{10}\) \(\frac{14}{10}\)
    \(8\) \(\frac{1}{10}\) \(\frac{8}{10}\)
    \(9\) \(\frac{1}{10}\) \(\frac{9}{10}\)
    \(E(\overline{X}) =\) \(6\)
    Table 3.5.4. Distribution of Sample Means

Note that several interesting things have happened. First, when we take the mean of all of our possible samples, we see repetition. That is, more than one sample produces the values \(5\text{,}\) \(6\text{,}\) and \(7\) as a mean. Consider the graphs below of the individual values from the population, which is uniform, versus the sample means, which is starting to look more mound shaped.

(a) Individual Values
(b) Sample Means
Figure 3.5.5. Distributions Based on \(\lbrace 0, 3, 6, 9, 12\rbrace\)

Also notice that the distribution of sample means has less variation than the distribution of individual values from the population. Another interesting thing that has happened is that the expected value of \(\overline X\) is \(6\text{,}\) which is exactly the population mean. That is, when we take the mean of the sample means, the average of the averages, we get the same thing as the mean of the entire population.

These two factors combined will play an important role on the next page, where we introduce one of the most important theorems in statistics.

Figure 3.5.6. Distribution of Sample Means I
Figure 3.5.7. Distribution of Sample Means II

Samples of size three are drawn from the finite population \(\lbrace 1, 3, 5, 7, 9 \rbrace\) without replacement.

Question: what is the mean of the sampling distribution for this sample?

Answer

5

A sample of size three is drawn from the population \(\lbrace 5, 10, 15, 20, 25, 30 \rbrace\) without replacement.

Question: which of the following is NOT a valid sample?

  1. \(\lbrace 5, 25, 15 \rbrace\)

  2. \(\lbrace 10, 30, 25 \rbrace\)

  3. \(\lbrace 5, 15, 15 \rbrace\)

  4. \(\lbrace 25, 15, 20 \rbrace\)

Answer

(c)

If a population is uniformly distributed, then the distribution of sample means for a sample of any size \(n\) will also be uniformly distributed.

Question: is the above statement true or false?

Answer

False

Subsection 3.5.2 The Central Limit Theorem

Making inferences about populations based on samples relies heavily on an understanding of how these samples behave. We must know what we can expect to see in samples that are drawn from a population with a uniform distribution, a normal distribution, or some other distribution. In the previous subsection, we saw an example in which a sample drawn from a uniform distribution had sample means that had a more mound-shaped distribution. This is a particular instance of the next theorem.

According to the central limit theorem, if we take samples and look at their means, they will have approximately a normal distribution, and the bigger the sample we take, the better the sample means will match a normal distribution. How big does \(n\) need to be? That depends on the way the individual population values are distributed.

Let's apply this to several examples.

A population of church mice has tail lengths that are uniformly distributed between 1.5 and 3 inches long. In which of the following situations could you use a normal distribution to compute the likelihood of getting a sample of \(n\) church mice that has a mean tail length of at least 2 inches?

  1. You sample \(n = 5\) church mice

  2. You sample \(n = 100\) church mice

Solution

The answer to this question relies on the central limit theorem. Note that the distribution of individual church mouse tails is not normal, but uniform.

  1. We can not use a normal distribution because the population distribution is not normal, and \(n\) is less than 30.

  2. We can use a normal distribution. The sample size of \(n=100\) is big enough (at leat 30) that even though the population distribution is uniform, the distribution of sample means will be very close to a normal distribution.

At a certain golf club, the members have handicaps that have a normal distribution with a mean of \(\mu = 9.6\) and standard deviation of \(\sigma = 1.85\text{.}\) A sample of these golfers is selected for a tournament. In which of the following instances can we use a normal distribution to compute the probability that the average handicap for the selected golfers will be less than 10?

  1. The club selects \(n = 10\) members for the tournament

  2. The club selects \(n = 60\) members for the tournament

Solution

Again, we must use the central limit theorem. In this case, however, the underlying distribution for the population is already normal.

  1. Since the population distribution is normal, the distribution for \(\overline x\) is also normal, no matter what \(n\) is.

  2. See the answer above—we can again use a normal distribution for \(\overline x\text{.}\)

Figure 3.5.15. Central Limit Theorem I
Figure 3.5.16. Central Limit Theorem II

According to the central limit theorem, the distribution of the sample means for samples of size \(n\) is always normal.

Question: is the following statement true or false?

Answer

False

Samples of size \(45\) are drawn from a population of values and the mean of the samples are exampled.

Question: can we use a normal distribution to approximate the distribution of these sample means?

Answer

Yes

The population of penny dates is skewed to the right. Samples of 20 pennies are drawn and the average of these sets of 20 dates are noted.

Question: can we approximate the distribution of sample means with a normal distribution?

Answer

No

Subsection 3.5.3 Sampling Distribution of the Mean

In those cases where the central limit theorem applies, it gives us a powerful tool for dealing with sample means. It not only tells us that these sample means will have a normal distribution, it also provides the mean and standard deviation of that normal distribution.

In the next several examples, we will be asked to find probabilities for sample means. It is important to remember that we must differentiate between the probability of an individual being in a certain range, and the probability of the mean of a sample being in that range. These probabilities will be different, and we must use the appropriate normal distribution for each one.

The postal service has determined that the weight of a certain size shipping container will be normally distributed with a mean of \(\mu = 19.4\) ounces and a standard deviation of \(\sigma = 4.5\) ounces.

  1. What is the probability that a randomly selected container will weigh more than 22 ounces?

  2. What is the probability that a sample of 10 such containers will have an average weight of more than 22 ounces?

Solution

Since the population has a normal distribution, we can use the central limit theorem. However, we do not need it for the first question as we are selecting an individual from a normally distributed population. Let's let \​(W\) be the weight of an individual, and \(\overline W\) the weight of the sample of 10. From the central limit theorem, both \(W\) and \(\overline W\) will have the same mean, but the standard deviation for \(\overline W\) will be

\begin{equation*} \sigma_{\overline W} = \frac{\sigma_W}{\sqrt{n}} = \frac{4.5}{\sqrt{10}} \approx 1.423\text{.} \end{equation*}

We sketch both the normal distribution for \(W\) (blue) and the one for \(\overline W\) (red) in the same picture to see how they compare.

Figure 3.5.22. Distributions for \(W\) and \(\overline W\)

  1. We want to find \(P(W \gt 22)\text{.}\) Note that this is not a binomial approximation, so we do not apply a continuity correction! Instead, we compute the z-score for 22 and compute as shown.

    \begin{align*} P(W \gt 22) \amp = P\left(\frac{W-\mu}{\sigma} \gt \frac{22-19.4}{4.5}\right)\\ \amp = P(Z \gt 0.58)\\ \amp = 1 - P(Z \lt 0.58)\\ \amp = 1 - 0.7190\\ \amp = 0.2810 \end{align*}

    This is a fairly likely occurrence.

  2. Now we need to find \(P(\overline{W} \gt 22)\text{.}\) The variable \(\overline{W}\text{.}\) We do a very similar computation to the one shown above, but we use the standard deviation for the sampling distribution when computing the z-score.

    \begin{align*} P(\overline W \gt 22) \amp = P\left(\frac{\overline{W} - \mu}{\sigma/\sqrt{n}} \gt \frac{22-19.4}{4.5/\sqrt{10}}\right)\\ \amp = P(Z \gt 1.83)\\ \amp = 1 - P(Z \lt 1.83)\\ \amp = 1 - 0.9664\\ \amp = 0.0336 \end{align*}

    This is a much less likely occurrence.

In the example above, note how much less likely it is to get a sample, even a small one of size 10, that varies too much from the mean. To see how sample size affects the sampling distribution, consider the following example.

The time it takes an adult to complete a certain standardized test follows am unknown distribution with mean \(\mu = 82.5\) minutes and standard deviation \(\sigma = 13.1\) minutes.

  1. Suppose that a sample of 50 adults are given the test. Find the probability that the average time they take to complete the test is at least 85 minutes.

  2. If a second sample of 100 adults is selected, what is the probability that their average completion time is at least 85 minutes?

Solution

Since the standard deviation for our sampling distribution depends on the sample size, the two parts of this example will have different distributions.

  1. The average time it takes a sample of 50 adults to complete this test will have a normal distribution (because \(n \gt 30\)) with mean 82.5 and standard deviation

    \begin{equation*} \frac{13.1}{\sqrt{50}} \approx 1.853\text{.} \end{equation*}

    The probability their mean score is at least 85 can be computed as shown.

    \begin{align*} P(\overline{X} \geq 85) \amp = P\left(\frac{\overline{X}-\mu}{\sigma/\sqrt{n}} \geq \frac{85-82.5}{1.853}\right)\\ \amp = P(Z \geq 1.35)\\ \amp = 1 - P(Z \lt 1.35)\\ \amp = 1 - 0.9115\\ \amp = 0.0885 \end{align*}

    This is not likely, but not unusual either.

  2. For a sample of 100 adults, the mean is still 82.5, but the standard deviation is now

    \begin{equation*} \frac{13.1}{\sqrt{100}} = 1.31\text{.} \end{equation*}

    The probability computation therefore looks a little different.

    \begin{align*} P(\overline{X} \geq 85) \amp = P\left(\frac{\overline{X}-\mu}{\sigma/\sqrt{n}} \geq \frac{85-82.5}{1.31}\right)\\ \amp = P(Z \geq 1.91)\\ \amp = 1 - P(Z \lt 1.91)\\ \amp = 1 - 0.9719\\ \amp = 0.0281 \end{align*}

    So the probability that the group of 100 averages more than 85 minutes is about a fourth as likely as for the group of 50.

The graph below shows the two sampling distributions that we used in this example. Notice how much smaller the shaded area under the red curve (where \(n=100\)) is as compared to the blue (where \(n=50\))

Figure 3.5.24. Sampling Distribution for \(\overline X\) when \(n=50\) and \(n=100\)
Figure 3.5.25. Sampling Distribution for the Mean I
Figure 3.5.26. Sampling Distribution for the Mean II

The loudness of a cat&s meow has a normal distribution with a mean of 50 decibels and a standard deviation of 9 decibels. A sample of 10 cats are collected and the average loudness of their meows is measured.

Question: what is the probability that the average loudness of these 10 cats' meows is at least 56 decibels?

Answer

0.0174

The distribution of penny dates is skewed to the left with a mean of 2001 and standard deviation 2.4 years. A sample of 49 pennies is drawn and the average date of the pennies is noted.

Question: what is the probability that the mean date for this sample is less than 2000?

Answer

0.0018

The distribution of gas mileage in a certain fleet of automobiles is normal with a mean of 23.6 mpg and a standard deviation of 4.1 mpg. A sample of 5 cars is selected at random from this fleet and their gas mileage is tested.

Question: what is the probability that the average gas mileage in this sample is more than 25 mpg?

Answer

0.2236

Subsection 3.5.4 Sample Distribution of a Proportion

At the beginning of this section, we introduced the idea of a sample proportion. To help solidify this concept, the next example shows how we might compute such a proportion.

In a sample of 200 butterflies, 75 were yellow monarchs. What is the proportion of yellow Monarch butterflies in this sample?

Solution

The sample proportion is the number of individuals who have the desired characteristic, which we call \(x\text{,}\) divided by the sample size, which is \(n\text{.}\) In this case,

\begin{equation*} \hat p = \frac{x}{n} = \frac{75}{200} = 0.375\text{.} \end{equation*}

Note that we use \(\hat p\) for a proportion in a sample instead of \(p\text{.}\) The variable \(p\) was used in binomial probabilities where it stood for the probability of a success. Thought of another way, \(p\) is the proportion of individuals in the population with the desired characteristic. So \(p\) and \(\hat p\) really measure the likelihood of the same characteristic. It is just that one does it for the population and the other for a sample.

To understand how sample proportions will be distributed—and thus how we can compute their probabilities—we need to think back to Section 3.4. There we saw how to approximate binomial probabilities using a normal distribution. As long as the criteria for approximation was met, we could use a normal distribution to figure out \(P(X=x)\text{,}\) the probability of \(x\) successes in \(n\) trials.

How does this relate to proportions? A proportion in a sample is simply a ratio of the \(x\) successes to the \(n\) trials in a binomial process. Thus, the distribution of sample proportions will have a mean which is the mean of a binomial distribution divided by \(n\text{.}\) It will have a standard deviation which is the standard deviation of the binomial distribution divided by \(n\text{.}\) These formulas are summarized below.

Note that we are not restricted to a sample size of 30 or more, as we are when using the central limit theorem for sample means. Instead, we must check to make sure that the criteria for approximating a binomial distribution with a normal distribution are met. Namely, we need to verify that \(n\times p\) and \(n\times q\) are both greater than 5. Consider the following example.

Only 2% of the world's population has naturally blond hair. Suppose that you were drawing a sample from the world population, and you wished to use a normal distribution to compute probabilities for the sample proportion, \(\hat p\text{,}\) of people with blond hair. How large a sample should you take?

Solution

In order to use a normal distribution to compute probabilities for \(\hat p\) we need to ensure that \(n\times p\) and \(n\times q\) are both greater than 5. Note that the \(p\) in this formula is the proportion of people in the population with blond hair, not the proportion in our sample. We therefore need to have

\begin{equation*} n\times p \gt 5 \Rightarrow n(0.02) \gt 5 \Rightarrow n \gt \frac{5}{0.02} \Rightarrow n \gt 250\text{.} \end{equation*}

Since \(q = .98\) is much larger than \(p\text{,}\) we do not need to check \(n\times q\text{.}\) We therefore must sample at least 251 individuals to apply the sampling distribution formulas above.

Figure 3.5.33. Sampling Distribution for a Proportion I
Figure 3.5.34. Sampling Distribution for a Proportion I

Thirteen percent of the people in a certain population like pickled beets. A sample of 80 individuals is drawn from the population and the proportion of the sample that like pickled beets is measured.

Question: can the sampling distribution for \(\hat p\) be used to approximate probabilities?

Answer

Yes

Suppose that 8 percent of fish in a certain lake are trout. A fisherman catches 40 fish over the course of a long weekend, and notes the proportion of those fish which are trout.

Question: can a normal distribution be used to approximate probabilities for the sample proportion \(\hat p\text{?}\)

Answer

No

Suppose that 32% of individuals in a certain large city have been mugged. A sample of \(n\) individuals is to be collected and the sample proportion \(\hat p\) will be analyzed using a normal distribution.

Question: what is the smallest sample size n for which the sampling distribution of \(\hat p\) can be approximated by a normal distribution?

Answer

16

With this information in mind, we are ready to look at examples involving sample proportions. Several of these can be found below.

As seen in Example 3.5.32, only 2% of the world's population are naturally blond. Suppose that you take a sample of 500 individuals from the world population. What is the probability that you will find less than 1.5% of your sample has blond hair?

Solution

First note that \(n\times p = 500(0.02) = 10\) and \(n\times q = 500(0.98) = 490\) are both bigger than 5. We can therefore use the sampling distribution for a sample proportion. The mean and standard deviation are shown below.

\begin{align*} \mu_{\hat p} \amp = p = 0.02\\ \sigma_{\hat p} \amp = \sqrt{\frac{pq}{n}} = \sqrt{\frac{(0.02)(0.98)}{500}} \approx 0.00626 \end{align*}

Therefore, we can compute the probability that less than 1.5% of our sample has blond hair as follows.

\begin{align*} P(\hat P \lt 0.015) \amp = P\left(\frac{\hat P - p}{\sqrt{\frac{pq}{n}}} \lt \frac{0.015 - 0.02}{0.00626}\right)\\ \amp = P(Z \lt -0.80)\\ \amp = 0.2119 \end{align*}

A certain ballot measure needs 60% approval to pass. Suppose that there is actually 58% support for the ballot measure amongst registered voters. A polling organization wishes to determine if the ballot measure is likely to pass. To do this, they randomly select a sample of 900 registered voters. If they predict the measure will pass when \(\hat p \geq 0.60\text{,}\) what is the probability that they will mistakenly predict passage of the ballot measure?

Solution

Checking that we can apply the normal distribution, we compute \(n\times p = 900(0.58) = 522\) and \(n\times q = 900(0.42) = 378\text{,}\) both of which are greater than 5. Now the mean and standard deviation for our normal distribution of \(\hat p\) are:

\begin{align*} \mu_{\hat p} \amp = 0.58\\ \sigma_{\hat p} \amp = \sqrt{\frac{(0.58)(0.42)}{900}} \approx 0.01645 \end{align*}

Therefore, the probability that they mistaken predict passage of the ballot measure is:

\begin{align*} P(\hat P \geq 0.60) \amp = P\left(\frac{\hat P - p}{\sqrt{\frac{pq}{n}}} \geq \frac{0.60 - 0.58}{0.01645}\right)\\ \amp = P(Z \geq 1.22)\\ \amp = 1 - P(Z \lt 1.22)\\ \amp = 1 - 0.8888\\ \amp = 0.1112 \end{align*}

So, there is approximately an 11% chance the polling organizaiton will get it wrong.

The failure rate of 11% is not very good if one wants to develop a reputation for being reliable. How could the polling organization lower the likelihood that they make a wrong prediction? As we shall see in the next chapter, one method is to recognize that there is the possibility of an “error” in sampling. Therefore, the pollsters should not predict a win based on a sample proportion of 0.60. Another method is to lower the variation between samples by raising the sample size. If \(n\) gets bigger, \(\sigma_{\hat p}\) gets smaller, and the probability of being wrong will go down.

Figure 3.5.40. Sampling Distribution for a Proportion III
Figure 3.5.41. Sampling Distribution for a Proportion IV

Twelve percent of frogs in a certain swamp region are, for one reason or another, unable to croak. A sample of 100 frogs is collected and the proportion, \(\hat p\text{,}\) of frogs that can not croak is measured.

Question: what is the probability that at least 10% of the frogs in the sample can not croak?

Answer

0.7324

In a recent survey of American households, 83% were found to have multiple television sets. Suppose that a sample of 2000 American households are called and the proportion that have multiple television sets is noted.

Question: what is \(P(\hat P \lt 0.84)\text{?}\)

Answer

0.8830

Thirty percent of a certain population have characteristic \(X\text{.}\) A sample of 1500 individuals is selected from this population, and the proportion of those with characteristic \(X\) is noted.

Question: what is \(P(\hat P \gt 0.33)\text{?}\)

Answer

0.0055