Imagine two pollsters are each trying to estimate the true proportion p of adults in a city who plan to vote for a ballot measure. The truth — which nobody knows — is p = 0.50.
Pollster A uses a well-designed simple random sample (SRS), but only calls 40 people. Run that poll a thousand times and the estimates p̂ bounce all over the place — 0.42, 0.58, 0.50, 0.65 — but they average out right around 0.50.
Pollster B calls 2,000 people, but only dials numbers listed in a phone book that skews older. Run that poll a thousand times and the estimates are tightly bunched — 0.44, 0.45, 0.43, 0.46 — but they cluster around 0.44, not 0.50.
Which pollster do you trust? Pollster A is all over the place but centered on the truth. Pollster B is precise but consistently wrong. Today we name these two ideas — bias and variability — and learn why a bigger sample fixes one of them and not the other.
In Lessons 15 and 16 you built sampling distributions — the distribution of a statistic (like p̂ or x̄) over all possible samples of a fixed size n. A sampling distribution has a center, a spread, and a shape. This lesson is about what the center and the spread tell us about the quality of an estimator.
A statistic is an unbiased estimator of a parameter if the mean of its sampling distribution equals the true value of the parameter. In symbols, the sampling distribution is centered at the parameter.
p̂ is an unbiased estimator of p, because the mean of the sampling distribution of p̂ is exactly p (you saw μ_p̂ = p in Lesson 15).x̄ is an unbiased estimator of μ, because μ_x̄ = μ (Lesson 16).If the center of the sampling distribution is off from the true parameter, the estimator is biased. Bias is a systematic error — the estimates miss in the same direction, on average, no matter how many times you sample. Bias comes from the method: a sampling design that systematically over- or under-represents part of the population (undercoverage, voluntary response, nonresponse), or a measurement instrument that's miscalibrated (a scale that always reads 2 lb heavy).
Key idea: Bias is about accuracy — are you aiming at the right target? An unbiased estimator's sampling distribution is centered on the parameter; on average, it hits the bullseye.
The variability of a statistic is the spread of its sampling distribution — how far individual estimates tend to fall from the center. Less variability means the estimates are more tightly clustered, which we call greater precision.
We measure variability with the standard deviation of the sampling distribution:
σ_p̂ = sqrt(p(1 − p) / n)σ_x̄ = σ / √nKey idea: Variability is about precision — how consistent are your shots? Low variability means repeated samples give nearly the same answer (whether or not that answer is correct).
Picture a dartboard. The bullseye is the true parameter. Each dart is one sample's estimate. Bias and variability are two independent dials, giving four combinations:
[GRAPH: Four dartboard targets arranged in a 2×2 grid, each with a red bullseye at the center marking the true parameter, and several blue dots showing where repeated sample estimates land.
Top-left — "Low bias, low variability (ideal)": dots tightly clustered, centered on the bullseye.
Top-right — "Low bias, high variability": dots widely scattered but evenly surrounding the bullseye, average position at the center.
Bottom-left — "High bias, low variability": dots tightly clustered, but the whole cluster sits off to the upper-right, away from the bullseye.
Bottom-right — "High bias, high variability": dots widely scattered AND the scatter is centered away from the bullseye.
Column headers read "Low variability" and "High variability"; row headers read "Low bias" and "High bias".]
Look again at the formulas. The sample size n is in the denominator, under a square root:
σ_p̂ = sqrt( p(1 − p) / n ) σ_x̄ = σ / √nSo as n increases, both standard deviations decrease — larger samples produce less variability (more precision). But because of the √n, the payoff shrinks:
n (since √4 = 2).n only divides the standard deviation by √2 ≈ 1.414, a reduction of about 29%.This is the famous square-root law: precision improves with the square root of sample size, not in direct proportion. Quadrupling your survey cost only halves your error.
A common surprise: the variability of p̂ and x̄ depends on the sample size n, not on the population size N. A well-designed SRS of n = 1000 is essentially as precise for a country of 300 million as it is for a town of 30,000. (This holds as long as the 10% condition is met — the sample is less than 10% of the population — so that sampling without replacement behaves like independent draws. It's n that drives precision, not what fraction of the population you've sampled.)
Here is the single most important idea in this lesson. Increasing n reduces variability, but it does NOT reduce bias.
Bias lives in the center of the sampling distribution, and the center is determined by the method, not the sample size. If your phone-book sampling frame systematically excludes younger voters, then every sample — whether n = 40 or n = 40,000 — is drawn from that same skewed frame. A bigger biased sample just gives you a more precise estimate of the wrong number. You shrink the spread around a center that's still in the wrong place.
The fix for bias is a better method — a proper random sampling design, better coverage of the population, a calibrated instrument. The fix for variability is a larger sample. They are two different problems with two different cures.
Problem. A bathroom scale always reads exactly 3 pounds too high. A person weighs themselves 50 times in a row (their true weight is constant). Describe the bias and variability of the 50 readings as an estimate of true weight.
Strategy. Ask the two questions separately: Is the center right (bias)? Are the readings consistent (variability)?
Solution. The readings are extremely consistent — the scale gives nearly the same number every time — so variability is low (high precision). But every reading is centered about 3 lb above the truth, so the estimator is biased (low accuracy). This is the high-bias, low-variability target.
Interpretation. The scale is precise but inaccurate. Weighing yourself more times (a "larger sample" of readings) won't help — every reading inherits the same 3-lb calibration error. The cure is to recalibrate the scale (fix the method), not to weigh more often.
Problem. A polling organization estimates the proportion p of voters favoring a candidate. With a sample of n = 500, the standard deviation of p̂ is about 0.022 (assuming p ≈ 0.5). What sample size is needed to cut the standard deviation in half, to about 0.011? What if they merely double n to 1000?
Strategy. Use σ_p̂ = sqrt(p(1−p)/n). Because n is under a square root, halving the SD requires quadrupling n.
Solution.
To halve the SD: multiply n by 4 → n = 4 × 500 = 2000.
Check: σ_p̂ = sqrt(0.5 × 0.5 / 2000) = sqrt(0.000125) ≈ 0.0112. ✓ (About half of 0.022.)
Doubling to n = 1000:
σ_p̂ = sqrt(0.5 × 0.5 / 1000) = sqrt(0.00025) ≈ 0.0158.
That's a reduction from 0.022 to 0.0158 — only about a 29% drop, not 50% (it's a factor of 1/√2 ≈ 0.707).
Interpretation. To make your estimate twice as precise, you must collect four times as much data. Doubling the sample gives a meaningful but modest improvement. This is why national polls rarely exceed a few thousand respondents — the precision gains from going larger don't justify the cost.
Problem. A website wants to estimate the proportion of all its users who are satisfied. It posts a voluntary pop-up survey. In a pilot, 200 users respond and 70% report being satisfied. A manager says, "Our estimate could be off. Let's fix it by leaving the survey up until 5,000 users respond — a much bigger sample will give us the true satisfaction rate." Evaluate the manager's reasoning.
Strategy. Identify whether the problem is variability (fixable by larger n) or bias (a property of the method).
Solution. A voluntary response survey is a biased sampling method: people with strong opinions — often the most dissatisfied or the most enthusiastic — are more likely to respond, so the responding group is not representative of all users. This bias is built into how the data are collected. Collecting 5,000 responses instead of 200 will reduce the variability of p̂ (the estimate will be more stable from one run to the next), but it will not move the center of the sampling distribution toward the true satisfaction rate. The estimate will simply be a more precise version of the same biased number.
Interpretation. The manager is wrong. A larger sample shrinks variability, not bias. To estimate the true satisfaction rate, the company needs a random sample of all users (a better method), not a bigger voluntary-response sample.
Problem. Two methods estimate the mean commute time μ (in minutes) for a city. Repeated sampling produces these sampling-distribution summaries:
| Method | Mean of sampling distribution | SD of sampling distribution |
|---|---|---|
| Method A | 31.0 | 4.5 |
| Method B | 34.2 | 1.8 |
The true mean commute time is μ = 31.0 minutes. Which method would you recommend, and why?
Strategy. Compare bias (compare each center to μ = 31.0) and variability (compare SDs).
Solution. Method A is centered at 31.0, which equals μ, so Method A is unbiased. Method B is centered at 34.2, about 3.2 minutes above the truth, so Method B is biased (it systematically overestimates). Method B has lower variability (SD 1.8 vs. 4.5), so it's more precise — but it's precisely aiming at the wrong value.
Interpretation. Recommend Method A. An unbiased estimator that's a bit noisy is generally preferable to a precise estimator that's systematically wrong, because Method A's bias can't be reduced by a better method (it's already correct) and its variability can be reduced by increasing the sample size. Method B's bias, by contrast, would persist at any sample size.
Mistake 1 — Thinking a bigger sample removes bias.
Students often write "increase the sample size to reduce the bias." Why it's wrong: bias is determined by the sampling/measurement method and lives in the center of the sampling distribution; n only affects the spread. A larger biased sample is just a more precise wrong answer. Get it right: to reduce bias, fix the method (use random sampling, improve coverage, calibrate instruments). To reduce variability, increase n.
Mistake 2 — Confusing bias with variability.
Saying "the estimates are spread out, so the method is biased," or "the estimates are off-center, so there's high variability." Why it's wrong: spread = variability; off-center = bias. They are independent. Get it right: ask two separate questions — Where is the center? (bias) and How wide is the spread? (variability).
Mistake 3 — Thinking population size drives variability.
"We sampled a huge population, so our estimate must be very variable / we need a bigger fraction of the population." Why it's wrong: variability depends on the sample size n, not the population size N (given the 10% condition). A sample of n = 1000 is about equally precise for a city or a whole country. Get it right: focus on n, not on the fraction n/N.
Mistake 4 — Treating "low variability" as "good estimate."
A tightly clustered set of estimates feels trustworthy, but if they're clustered around the wrong value, precision is a trap. Get it right: an estimate needs to be both unbiased and low-variability to be reliable.
Mistake 5 — Forgetting the √n, expecting linear gains.
Thinking "double the sample, half the error." Why it's wrong: the SD has √n in the denominator, so doubling n cuts the SD by a factor of 1/√2 ≈ 0.707 (about 29%), not 50%. You must quadruple n to halve the SD.
p̂ without changing its bias?σ_x̄ = σ/√n, increasing the sample size from n = 100 to n = 400 will change the standard deviation of x̄ by a factor of:p̂ has standard deviation σ_p̂ = sqrt(p(1−p)/n). This standard deviation depends on:n = 600 has σ_p̂ ≈ 0.020. Approximately what sample size is needed to reduce this to σ_p̂ ≈ 0.010?9. (Short answer, in context.) A streaming service estimates the average number of hours its subscribers watch per week. It emails a survey only to subscribers who watched in the last 24 hours, and 3,000 of them respond. Explain whether the resulting estimate is likely to be biased, and whether increasing the number of respondents to 30,000 would correct the problem.
10. (Short answer, in context.) Two methods estimate the proportion p of a town's residents who recycle. Method X is unbiased with σ_p̂ = 0.05. Method Y is biased (centered 0.08 above p) with σ_p̂ = 0.01. A colleague prefers Method Y "because it's more precise." Respond, and state how each method's weakness could (or could not) be addressed.
n reduces the standard deviation of x̄ by approximately what percent?12. (Short answer.) Explain, using the target/dartboard analogy, the difference between an estimator that is "accurate but imprecise" and one that is "precise but inaccurate." Identify which type a larger sample size can improve.
Statistical Practices: 3 (Analyze Data) + 4 (Interpret Results)
A regional health department wants to estimate p, the true proportion of adults in a large county (population about 800,000) who got a flu shot this season.
A research team proposes two different methods for estimating p:
n = 400 adults from the county's complete adult registry and record the proportion p̂₁ who got a flu shot.n = 4,000 adults have responded and records the proportion p̂₂.Suppose, unknown to the team, the true proportion is p = 0.45.
(a) For Method 1, the statistic p̂₁ is an unbiased estimator of p. State what "unbiased" means in terms of the sampling distribution of p̂₁, and calculate the standard deviation of p̂₁. (3 points)
(b) Explain why Method 2 is likely to produce a biased estimate of p. In your answer, identify the type of sampling problem and describe the likely direction or nature of the bias. (3 points)
(c) The team argues that because Method 2 uses ten times as many people (n = 4,000 vs. n = 400), it will give a more trustworthy estimate of p. Evaluate this claim. Discuss what the larger sample size does and does not fix. (2 points)
(d) Interpret what it would mean, in the context of this study, to recommend a method that is both low-bias and low-variability. Which of the two methods better fits this goal, and how could its main weakness be reduced? (2 points)
(a) "Unbiased" means that the mean of the sampling distribution of p̂₁ equals the true parameter p — that is, the sampling distribution of p̂₁ is centered at p = 0.45. Over all possible SRSs of size 400, the estimates p̂₁ average out to the true proportion.
Standard deviation (the 10% condition holds: 400 < 0.10 × 800,000 = 80,000):
σ_p̂₁ = sqrt( p(1 − p) / n ) = sqrt( 0.45 × 0.55 / 400 )
= sqrt( 0.2475 / 400 ) = sqrt( 0.00061875 ) ≈ 0.0249So σ_p̂₁ ≈ 0.025.
(b) Method 2 uses a voluntary response (self-selected) sample. People who choose to visit and report on the website are not representative of all adults in the county — those with strong opinions or specific characteristics (for example, people who are especially health-conscious and got the shot, or people who are particularly engaged with the topic) are more likely to participate. Because the responding group systematically differs from the full population, the sampling distribution of p̂₂ will be centered away from p = 0.45 — it is biased. A plausible direction: health-engaged adults who got vaccinated may be overrepresented, so p̂₂ would tend to overestimate p. (Either a clearly justified direction, or a clear statement that the center is shifted off p, earns the point.)
(c) The claim is incorrect. The larger sample size in Method 2 reduces the variability of p̂₂ (its estimates will be more tightly clustered from one run to the next), but it does not reduce the bias, because bias is a property of the sampling method, not the sample size. A voluntary-response sample of 4,000 is drawn from the same self-selected, non-representative group as a smaller one would be, so its sampling distribution is still centered off p. Method 2 just gives a more precise estimate of the wrong value. A larger sample fixes variability, not bias.
(d) A method that is low-bias means its estimates are centered on the true vaccination proportion p = 0.45 (accurate — on target), and low-variability means repeated samples give nearly the same estimate (precise — tightly clustered). In context, this is a method whose p̂ reliably lands close to the true county vaccination rate. Method 1 (the SRS) better fits this goal because it is unbiased. Its main weakness is variability (σ_p̂₁ ≈ 0.025), which could be reduced by increasing the sample size n (for example, quadrupling n to 1,600 would roughly halve the standard deviation to about 0.0125).
Part (a) — 3 points
p̂₁ is centered at p / the mean of the sampling distribution equals p (= 0.45).sqrt(0.45 × 0.55 / 400).σ_p̂₁ ≈ 0.025 (accept 0.0249).Part (b) — 3 points
p̂₂ is centered away from p (states a justified direction OR clearly that the center is shifted off the true value).Part (c) — 2 points
n (a bigger sample is a more precise estimate of the wrong value).Part (d) — 2 points
p = 0.45, accurate) AND low-variability (estimates consistent/precise) in context.n.Where students lose points:
σ (the mean version) instead of sqrt(p(1−p)/n) for a proportion, or forgetting the square root.1. (B) An unbiased estimator's sampling distribution is centered at the parameter. (A) describes low variability, not unbiasedness; (C) shape is unrelated to bias; (D) sample size affects variability, not whether an estimator is centered correctly.
2. (B) Increasing n shrinks σ_p̂ = sqrt(p(1−p)/n) without affecting the center. (A) voluntary response adds bias; (C) a systematically high instrument adds bias; (D) the fraction sampled does not drive variability — n does.
3. (B) Going from n = 100 to n = 400 multiplies n by 4, so the SD changes by 1/√4 = 1/2. (A) 1/4 confuses the factor on n with the factor on the SD; (C) 1/√2 would correspond to doubling n; (D) the SD does change.
4. (C) The cuff's 5 mmHg error is a method/calibration bias that every reading inherits, so more patients won't reduce it — but more readings (larger n) do reduce variability. (A) and (B) wrongly claim bias is fixable by sample size; (D) ignores that variability does fall.
5. (B) A small SRS is unbiased (low bias) but its small n makes estimates scatter widely (high variability). (A) and (D) are biased instruments (off-center); (C) voluntary response is biased.
6. (B) σ_p̂ depends on p and n. (A) and (C) population size N does not appear (given the 10% condition); (D) repeating the survey doesn't change the SD of a single p̂.
7. (C) To halve the SD (0.020 → 0.010), quadruple n: 4 × 600 = 2,400. (A) 1,200 only doubles n (SD falls by 1/√2); (B) 1,800 triples n; (D) 6,000 over-corrects (factor 1/√10).
8. (B) Bias is a property of the method, untouched by sample size. (A) and (D) wrongly tie bias to n; (C) conflates two distinct concepts.
9. (Short answer.) Yes, the estimate is likely biased. By emailing only subscribers who watched in the last 24 hours, the survey systematically excludes lighter or inactive viewers (undercoverage) and oversamples heavy viewers, so the responding group is not representative of all subscribers. The sampling distribution of the mean watch-time estimate is centered above the true subscriber average. Increasing respondents from 3,000 to 30,000 would reduce variability (a more precise estimate) but would not correct the bias, because the same skewed sampling frame produces the same off-center result. The fix is a random sample of all subscribers, not more respondents from the active group.
10. (Short answer.) The colleague is mistaken to prefer Method Y on precision alone. Method X is unbiased — centered on the true recycling proportion p — but has higher variability (σ_p̂ = 0.05); this weakness can be reduced by increasing the sample size (e.g., quadrupling n roughly halves the SD to 0.025). Method Y is precise (σ_p̂ = 0.01) but biased by 0.08, systematically overestimating p; this bias is a property of the method and cannot be reduced by a larger sample — only by fixing the design. A precise estimate of the wrong value is not trustworthy, so Method X is preferable.
11. (B) Doubling n multiplies the SD by 1/√2 ≈ 0.707, a reduction of about 29%. (A) 50% would require quadrupling n; (C) 71% is the remaining fraction, not the reduction; (D) the SD does change.
12. (Short answer.) On a dartboard where the bullseye is the true parameter: an estimator that is accurate but imprecise has darts centered on the bullseye but widely scattered — low bias, high variability (e.g., a small SRS). An estimator that is precise but inaccurate has darts tightly clustered but off to one side — low variability, high bias (e.g., a miscalibrated instrument). A larger sample size can improve the imprecise (high-variability) estimator by tightening the cluster, but it cannot fix the inaccurate (biased) one, because bias depends on the method, not on n.
StatsIQ · Lesson 17 of 30 · Unit 2: Probability, Random Variables, and Probability Distributions · Phase 3: Sampling Distributions
This material is for exam-preparation purposes and is not affiliated with or endorsed by the College Board. AP® is a registered trademark of the College Board.
Content pending statistical-accuracy review (Isaac).