Lesson 6: Sampling Methods

(a) Warm-Up

Imagine your principal wants to know what percentage of your school's 1,800 students support moving lunch 30 minutes later. She can't ask everyone — that would take all week. So she stands by the cafeteria door at 11:50 a.m. and asks the first 50 students who walk in.

Pause and think: who is missing from those 50? Students who eat later, who have off-campus lunch privileges, who skip lunch to study, who arrive late. The 50 she gets are real students with real opinions — but they are not a fair snapshot of the whole school. They are simply the students who were easiest to reach.

Here is the uncomfortable truth at the heart of this lesson: a sample can be wrong before you ever do a single calculation. No amount of fancy statistics later can fix a sample that was collected unfairly. In this lesson you'll learn how to collect a sample that actually represents the population — and how to spot the sneaky ways samples go wrong.

(b) Core Concept

Population, frame, and sample

Three words that sound similar but mean different things — and the AP exam loves to test the difference.

• The population is the entire group you want to learn about. (All 1,800 students at the school.)
• The sampling frame is the actual list of individuals from which you draw your sample. Ideally it matches the population exactly, but often it doesn't. (The school's enrollment roster — which might be missing students who enrolled last week.)
• The sample is the subset you actually collect data from. (The 50 students you survey.)

A parameter is a number describing the population (the true percent who support the change). A statistic is a number describing the sample (the percent who support it among your 50). We use the statistic to estimate the parameter — but only if the sample was collected well.

A census collects data from every individual in the population. The U.S. Census attempts this every 10 years. A census gives you the parameter directly with no sampling error — but it is usually impractical (too expensive, too slow, sometimes impossible, like testing every battery in a factory by draining it). That's why we sample.

Why random selection matters

The single most important idea: random selection removes bias from the choice of who gets sampled. When chance — not the researcher's convenience, not a volunteer's eagerness — decides who is in the sample, every type of individual has a fair shot at being included. That is what lets us generalize from the sample back to the population. Without randomness, you can compute a beautiful statistic that estimates the wrong number.

The Simple Random Sample (SRS)

A simple random sample (SRS) of size n is a sample chosen so that every possible group of n individuals has an equal chance of being selected. (Equivalently, every individual is equally likely, AND every combination is equally likely — both conditions matter.)

How to take an SRS, step by step:

1. Label every individual in the sampling frame with a distinct number, usually 1 to N.
2. Randomize — use a random process (calculator or random digit table) to pick numbers.
3. Select the individuals whose labels come up, ignoring repeats (don't select the same person twice).
4. Stop when you have n distinct individuals.

On the TI-84, suppose N = 1800 and you want n = 50:

TI-84: MATH → PROB → 5:randInt(
Type: randInt(1, 1800, 50)
Output (example): {417 1192 88 ...}

If any number repeats, skip it and generate another. (Tip: randInt(1,1800,60) generates a few extras so you still land 50 after removing duplicates.)

Using a random digit table works the same way conceptually. Because labels run up to 1800 (a 4-digit number), read digits in groups of 4 across a row. Keep any group from 0001 to 1800; discard 0000, anything 1801–9999, and repeats — until you have 50 distinct labels.

Other good sampling methods

These all use randomness, just with extra structure.

Stratified random sampling. Divide the population into strata — groups of individuals who are similar to each other on something relevant — then take a separate SRS within each stratum and combine them. Example: split the 1,800 students by grade level (9, 10, 11, 12), then take an SRS of, say, 15 from each grade. You use stratification when you believe the strata differ in opinion and you want to guarantee each is represented. Stratifying on a variable related to the response reduces variability in your estimate.

Cluster sampling. Divide the population into clusters — groups that each resemble the whole population (each cluster is a mini-mixture) — then randomly select whole clusters and sample everyone in the chosen clusters. Example: the school has 60 homeroom sections, each a mix of grades and types. Randomly pick 8 homerooms and survey every student in those 8. You use clustering mainly for convenience/cost when the population is naturally grouped and traveling to each individual is hard.

The key distinction (memorize this):

- Stratified → groups are internally similar, different from each other; you sample some from every group.

- Cluster → each group is internally diverse (a mini-population); you sample all of a few groups.

Stratified spreads your sample across all groups; cluster concentrates it in a few.

Systematic random sampling. Choose a random starting point, then select every k-th individual. Example: line up all 1,800 students alphabetically, randomly pick a start between 1 and 36, then take every 36th person → 50 students. It's easy to administer and is random as long as the starting point is random and the list order has no hidden pattern tied to the response.

The bad methods (use random selection, NOT these)

Convenience sampling. Select individuals who are easiest to reach (the principal's 50 at the cafeteria door). Almost always biased, because "easy to reach" is rarely representative.

Voluntary response sampling. Let people choose themselves by responding to a general appeal (a poll posted on the school website, a call-in radio survey, online reviews). Biased because people with strong opinions — often negative — are far more likely to respond. Bigger voluntary samples are not better; they're just bigger and still biased.

Sources of bias

Bias means a sampling method that systematically favors certain outcomes — the statistic tends to be off in the same direction every time you'd repeat it. Four big sources:

• Undercoverage — some groups of the population are left out of the sampling frame entirely (a phone survey misses people without phones).
• Nonresponse — individuals are selected but cannot be reached or refuse to answer, and the ones who respond differ from those who don't (only the people home at 2 p.m. answer the landline).
• Response bias — respondents answer inaccurately: they lie (about embarrassing behavior), misremember, or shade answers toward what's socially acceptable, or an interviewer's manner influences them.
• Question-wording bias — the phrasing of the question pushes people toward an answer. "Don't you agree that overworked students deserve a later lunch?" leads the respondent more than "Do you support moving lunch 30 minutes later?"

Crucial: bias is about the method, not the size. A poorly designed survey of 1 million people is still biased; a well-designed SRS of 50 is not. Bias ≠ small sample. (Small samples give you more variability — more random scatter — but a large sample collected badly is still systematically wrong.)

(c) Worked Examples

Example 1 — Identify the method (easy)

Problem. A quality manager at a bottling plant inspects every 100th bottle that comes off the line, starting at a randomly chosen bottle among the first 100. Which sampling method is this?

Strategy. Look for the signature: a fixed interval (every k-th) with a random start.

Solution. Selecting every 100th item after a random start is systematic random sampling.

Interpretation. This is efficient on an assembly line. It's valid as long as there's no repeating pattern in production tied to every-100th position (e.g., a particular machine head always filling the 100th bottle).

Example 2 — Identify the bias type (medium)

Problem. A magazine mails a survey to 20,000 subscribers asking about their satisfaction. Only 1,400 mail it back, and satisfaction comes out very high. The editor brags about the result. What is the most serious problem?

Strategy. Ask: who actually ended up in the data, and how might they differ from those who didn't respond?

Solution. Only 7% (1,400 / 20,000) responded. This is nonresponse bias: the subscribers who bothered to return the survey are likely those with the strongest feelings (or the most loyal), who may differ systematically from the 18,600 who didn't respond.

Interpretation. The 93% silent majority could easily hold different views, so the high satisfaction figure cannot be trusted as representative of all subscribers. The large mailing size does not rescue it.

Example 3 — Design an SRS (medium)

Problem. A college has 640 faculty members and wants an SRS of 25 to interview about a new policy. Describe how to select the sample using a calculator.

Strategy. Label → randomize → select, ignoring repeats.

Solution.

1. Label the faculty 1 through 640 using the official directory.
2. On the TI-84, compute randInt(1, 640, 25). (Generate a few extra: randInt(1,640,30) is safer.)
3. Select the faculty whose labels appear. If any label repeats, skip it and use the next new number until you have 25 distinct people.
4. Interview those 25 faculty members.

Interpretation. Because every group of 25 faculty is equally likely, the sample is unbiased by selection, so results can be generalized to all 640 faculty.

Example 4 — Design a stratified sample (AP-style)

Problem. A district has 3,000 elementary, 2,000 middle, and 1,000 high school students (6,000 total). A researcher believes opinions on a new bus schedule differ by school level and wants a stratified sample of 120 students. Describe the plan, and explain why stratifying helps here.

Strategy. Strata = school levels (similar within, different between). Allocate proportionally, then SRS within each.

Solution. Use school level as the strata. With proportional allocation (each level gets a share matching its size — 120/6000 = 0.02, or 2% of each):

• Elementary: 0.02 × 3000 = 60 students
• Middle: 0.02 × 2000 = 40 students
• High school: 0.02 × 1000 = 20 students

Take a separate SRS within each level (e.g., label elementary students 1–3000 and run randInt(1,3000,60), and similarly for the others), then combine the 60 + 40 + 20 = 120 students.

Interpretation. Because opinions are expected to differ between levels but be more similar within a level, stratifying guarantees all three levels are represented and reduces the variability of the overall estimate compared with a single SRS of 120.

(d) Common Mistakes

1. Mixing up stratified and cluster. Students reverse them constantly. Remember: stratified = sample some from every group, and groups are similar inside (like grade levels). Cluster = sample all of a few groups, and each group is a diverse mini-population (like whole homerooms). Wrong: "We picked 8 homerooms and surveyed everyone — that's stratified." Right: that's cluster.

2. Calling a "random-sounding" method random. Phrases like "we surveyed people as they walked by" or "we posted a poll online" sound active and even effortful, but they are convenience and voluntary response — biased, not random. Asking "the first 30 students" or "whoever volunteers" is never an SRS. Look for an actual chance mechanism (labels + randInt, a random table, a drawing).

3. Confusing bias with small sample size. A small SRS isn't biased — it's just more variable (its estimate bounces around more). Conversely, a huge convenience or voluntary-response sample is still biased. Don't write "the sample is biased because it's too small," and don't write "the large sample makes the bias go away." Bias is about method; size is about variability.

4. Forgetting to ignore repeats in an SRS. When using randInt or a digit table, a label can come up twice. You must skip duplicates so you end with n distinct individuals. Stating this in an FRQ earns the point.

5. Vague SRS descriptions. "Pick people randomly" earns nothing. You must say label the individuals, use a random device (name it: randInt or a random digit table), and ignore repeats. Specifics score points.

(e) Practice Problems

11. (Short answer, in context) A gym has 850 members and wants to survey 40 of them about new equipment. Describe in detail how to select a simple random sample of 40 members using a TI-84 calculator. Include all key steps.

12. (Short answer, in context) A university newspaper posts an online poll asking, "Should the library stay open 24 hours?" After one week, 3,500 students have voted and 92% say yes. Identify the sampling method, name the type of bias, and explain why the 92% figure is unreliable even though the sample is large.

13. (Short answer) Explain the difference between stratified and cluster sampling. Give one situation where you would prefer each.

A researcher wants to estimate the average number of hours per week that adults in a large town exercise. She interviews shoppers leaving a health-food store on Saturday morning. Identify the sampling method and explain why her estimate is likely biased, and state the likely direction of the bias.

True or False, with one sentence of justification: "Because nonresponse bias and undercoverage both leave certain people out of the data, they are the same source of bias."

---

## (f) FRQ Practice

> This is a full 10-point free-response question in the new exam format, focused on Practice 2 (Collect Data). Give yourself about 18 minutes. Write in context.

FRQ. A statewide education agency wants to estimate the proportion of the state's 9,000 public high school teachers who feel they have adequate classroom technology. The agency has a complete, up-to-date directory of all 9,000 teachers, including each teacher's school and subject area.

(a) The agency first considers contacting every teacher in the directory (a census). State one practical reason the agency might choose to sample instead of conducting a census. (2 points)

(b) Describe how the agency could select a simple random sample (SRS) of 200 teachers from the directory. Be specific enough that another person could carry out your procedure. (4 points)

(c) Instead, a staff member proposes the following plan: "Post a link to the survey on the agency's social media page for two weeks and use all the responses we get." Identify the type of sampling this plan uses, name the most serious source of bias it introduces, and explain in context how that bias would likely affect the estimated proportion. (4 points)

---

### MODEL RESPONSE

Part (a). A census would require reaching all 9,000 teachers, which is expensive and time-consuming; many would be hard to contact or slow to respond, so the agency can get a reliable estimate faster and more cheaply by sampling. (Any one valid practical reason — cost, time, effort, difficulty contacting everyone — earns full credit.)

Part (b).

1. Label the 9,000 teachers in the directory 1 through 9000.

2. Use a random device — on the TI-84, compute randInt(1, 9000, 200) (generating a few extra numbers to allow for duplicates) — or use a random digit table reading 4-digit groups, keeping 0001–9000.

3. Ignore any repeated labels (skip a number that has already appeared) until 200 distinct labels are obtained.

4. The 200 teachers whose labels were selected make up the sample; survey those teachers.

Part (c). This plan is a voluntary response sample — teachers choose themselves by deciding whether to respond. The most serious source of bias is voluntary response bias: teachers with strong opinions (especially those frustrated by inadequate technology) are far more likely to respond than those who are satisfied or indifferent. As a result, the sample would overrepresent dissatisfied teachers, so the estimated proportion who feel they have adequate technology would likely be biased too low (an underestimate of the true proportion). The large number of responses does not fix this.

---

### POINT-BY-POINT RUBRIC (sums to 10)

Part (a) — 2 points

- +1 Identifies a valid practical drawback of a census (cost, time, labor, or difficulty contacting all 9,000).

- +1 Frames it as a reason to sample instead (sampling is faster/cheaper/easier while still giving a good estimate).

Part (b) — 4 points

- +1 Labels/numbers the 9,000 teachers (e.g., 1–9000) using the directory.

- +1 Names a legitimate random device (randInt, random number generator, or random digit table) and applies it to the labels.

- +1 States that duplicate/repeated labels are ignored.

- +1 Selects 200 distinct teachers and identifies them as the sample to be surveyed.

Part (c) — 4 points

- +1 Correctly identifies the method as voluntary response sampling.

- +1 Names voluntary response bias (or nonresponse / self-selection bias) as the source.

- +1 Explains the mechanism in context: people with strong opinions (dissatisfied teachers) are more likely to respond.

- +1 States the likely effect/direction on the estimate (overrepresents dissatisfied teachers → estimated proportion with adequate technology is too low), and/or notes that a large response count does not remove the bias.

Where students lose points:

- In (b), writing "pick 200 teachers at random" with no labeling, no named random device, and no mention of ignoring repeats — vague descriptions earn at most 1 of 4. Each specific step is a separate point.

- In (b), forgetting the "ignore repeats / 200 distinct" step is the single most commonly missed point.

- In (c), naming the method but giving no contextual mechanism — you must say who responds and why (strong-opinion teachers), not just "it's biased."

- In (c), failing to state a direction for the bias, or incorrectly claiming the large sample size makes the result trustworthy.

- In (a), giving a reason that isn't practical (e.g., "a census is biased" — it isn't) earns 0.

---