Six Sigma Kaizen logo

Stat Dojo / Foundations

Sampling and the Central Limit Theorem

Why averages behave better than individual values, what a standard error means, and how to sample so the answer is not biased.

Question it answers
How well does a sample average estimate the process average?
Data needed
A random, representative sample
Key output
Standard error and an interval for the mean
Rule
Standard error = s / √n
Assumptions
Independent, representative observations; enough n for the shape
Excel
STDEV.S/SQRT(COUNT), CONFIDENCE.T, RAND, GAMMA.INV
Minitab
Display Descriptive Statistics (SE Mean); Calc > Random Data
Why it matters
It explains why averages, tests, and control charts work

The Idea in Plain Language

You almost never measure everything. You take a sample and use it to learn about the whole population. Two ideas make that work. First, the sample must represent the population. Second, the average of a sample behaves in a predictable way, which is the central limit theorem (CLT).

The CLT says: take many samples of size n from almost any population, and the averages of those samples will pile up in a bell shape centered on the population mean, with a spread of σ/√n. That spread is the standard error. It is why t-tests, control charts for averages, and confidence intervals work even when individual values are not normal.

Why it matters. The standard error tells you how much a sample average can be off. To halve it you need four times as much data, which is the most important cost-versus-precision trade in data collection.

Sampling Well

MethodHowBest forWatch out
Simple randomEvery item has an equal chanceUniform populations, a list to draw fromNeeds a complete list
SystematicEvery k-th itemProduction lines, easy to runHidden cycles that match k
StratifiedRandom samples within groups (shifts, machines)Known groups that differNeed to know the groups
ClusterSample whole groups (cartons, batches)Cheap access to groupsLess precise than random
Rational subgroupSmall samples taken close in timeControl charts: spread within, change betweenSubgroups that mix sources
ConvenienceWhatever is easyNothing reliableBiased; avoid
Bias beats sample size. A large biased sample, such as only the first shift or only the parts that reached the gauge, tells you precisely the wrong thing. Fix how you sample before you worry about how many.

For more on choosing a method, see the Sampling Methods entry. For how many to take, see Sample Size and Power.

Seeing the Central Limit Theorem

Repair times in a plant are right-skewed. The population mean is 9.8 hours and the standard deviation is 5.9 hours. We drew 4,000 samples at each size and plotted the averages.

Single values (n = 1) SD = 5.83 0 5 10 15 20 25 30 Averages of n = 5 SD = 2.57 0 5 10 15 20 25 30 Averages of n = 30 SD = 1.06 0 5 10 15 20 25 30 Repair time (hours), dashed line = mean of each histogram
Left: single repair times are skewed. Middle and right: the averages become more symmetric and much more tightly grouped as n grows.
Sample size nPredicted standard error σ/√nSimulated SD of the averagesSkewness of the averages
15.865.831.10
52.622.570.47
301.071.060.21
Conclusion. The average of 30 repair times has a standard error of 1.07 hours, against 5.9 for a single repair, and its distribution is close to normal even though the individual times are not. The simulated values match σ/√n, and the averages are centered on the population mean.

How large must n be? For roughly symmetric data, 5 to 10 is often enough. For moderately skewed data, 30 is a common rule. For extremely skewed data or heavy tails, you may need hundreds. The CLT also needs independent observations.

Standard Error in Practice

One real sample of 25 repairs gives a mean of 8.94 hours and a sample standard deviation of 6.14 hours.

  1. Standard error = s / √n = 6.14 / √25 = 1.228 hours.
  2. 95% interval for the mean = 8.94 ± 2.064 × 1.228 = 6.40 to 11.47 hours.
  3. To halve the standard error to 0.614 you need n = (6.14 / 0.614)² = 100 repairs, four times as many.
2.62 n = 5 1.85 n = 10 1.17 n = 25 0.83 n = 50 0.59 n = 100 0.29 n = 400 Standard error (hours) Standard error (hours)
Standard error falls with the square root of the sample size. Going from 100 to 400 observations halves it; going from 5 to 10 improves it by only 29%.
Standard deviation or standard error? The standard deviation describes how much individual values vary and does not shrink with more data. The standard error describes how much the sample mean varies and shrinks with n. Use the first to describe the process and the second to describe your estimate.

Run It in Excel and Minitab

ExcelStep by step

  1. Standard error: =STDEV.S(A2:A26)/SQRT(COUNT(A2:A26)) (1.228).
  2. Interval: =CONFIDENCE.T(0.05, STDEV.S(A2:A26), COUNT(A2:A26)) gives the margin (2.535).
  3. Simulate the CLT: in a grid of 30 columns, enter =GAMMA.INV(RAND(), 2.8, 3.5) in many rows, then average each row with =AVERAGE(A2:AD2) and make a histogram of the averages. Press F9 to redraw.
  4. Random sample from a list: add =RAND() beside each item, sort by it, and take the top n.

MinitabStep by step

  1. Standard error: Stat > Basic Statistics > Display Descriptive Statistics shows SE Mean next to the mean and standard deviation.
  2. Simulate: Calc > Random Data > Gamma to fill 30 columns (shape 2.8, scale 3.5), then Calc > Row Statistics with Mean to average across the row, then Graph > Histogram of the averages.
  3. Random sample from a column: Calc > Random Data > Sample From Columns.
  4. Sample size for a target margin: Stat > Power and Sample Size > 1-Sample t or Stat > Basic Statistics > 1-Sample t with a confidence interval, to see how the margin shrinks.
Minitab session window: Display Descriptive Statistics (typed excerpt, simplified)
Descriptive Statistics: Repair

Variable   N   Mean  SE Mean  StDev  Minimum     Q1  Median     Q3  Maximum
Repair    25  8.94    1.228   6.14   2.30  3.90   8.20  11.55    28.50

Reading and Reporting

  1. Describe how you sampled, and why it should represent the process.
  2. Report the mean with its standard error or interval, not only the mean.
  3. Report the standard deviation to describe the process, and the standard error to describe the estimate.
  4. Say how many observations and whether they were independent.

Common Mistakes

MistakeWhy it misleadsBetter
Confusing standard deviation with standard errorThe first does not shrink with nState which one, and use each for its purpose
Believing n = 30 always makes data normalHeavy skew or outliers need morePlot the averages or use a bootstrap
Sampling only what is convenientBias does not average outUse random or stratified sampling
Using the CLT on non-independent dataAutocorrelation makes the effective n much smallerCheck the time order; subsample or model it
Applying the CLT to individual valuesIt describes averages, not single itemsUse the distribution of the individuals for specification limits
Ignoring the finite populationWhen the sample is a big share of a small population, the SE is overstatedApply the finite population correction

Try It Yourself

Cycle times have a standard deviation of 8 minutes. You plan to average 16 observations.

  • What is the standard error of the average?
  • How many observations would you need for a standard error of 1 minute?
Show the answer

Standard error = 8 / √16 = 2.0 minutes.

For 1 minute: n = (8 / 1)² = 64 observations, four times as many to halve the standard error.

Sampling and the Central Limit Theorem: Frequently Asked Questions

What is the central limit theorem in simple terms?

Averages of random samples tend to follow a bell curve, whatever the shape of the individual values, and the larger the sample the closer the fit and the narrower the curve.

What is the standard error?

It is the standard deviation of a statistic, usually the sample mean. For the mean it equals the standard deviation divided by the square root of the sample size, and it describes how far the sample mean is likely to be from the true mean.

Is n = 30 enough?

It is a rule of thumb and works for moderately skewed data. For symmetric data a smaller n is enough. For very skewed data or data with outliers you may need far more.

Why do control charts use subgroup averages?

Averages of subgroups are close to normal by the central limit theorem, so the 3-sigma limits for the chart behave predictably even if individual values are not normal.

Does a bigger sample fix a biased sample?

No. A bigger sample reduces random error but not bias. The estimate just becomes more precisely wrong.

Sources and Further Reading

  • NIST/SEMATECH, e-Handbook of Statistical Methods, sections on sampling and the central limit theorem (itl.nist.gov/div898/handbook).
  • David S. Moore, George P. McCabe, and Bruce A. Craig, Introduction to the Practice of Statistics, Freeman.
  • William G. Cochran, Sampling Techniques, Wiley.
  • Minitab Support, “Random Data” and “Row Statistics” (support.minitab.com).

This content is educational. Worked examples use made-up data. Menu names for Minitab follow recent versions of Minitab Statistical Software and can differ slightly in older releases; Excel steps use Microsoft 365 and the Analysis ToolPak.