A news headline announces: "People who drink coffee live longer — new study of 200,000 adults." Your friend, a devoted coffee drinker, declares victory. But before he brews another pot, ask two questions.
First: Who was studied, and who is this headline really about? If the 200,000 adults were a random sample of a country's population, the finding might describe that whole country. If they were 200,000 volunteers from one health-app, the finding describes app users — and maybe nobody else.
Second — and bigger: Did anyone assign people to drink coffee? Almost certainly not. People chose to drink coffee. And coffee drinkers might also exercise more, smoke less, or have higher incomes. Any of those could be the real reason they live longer.
This lesson is about turning a study's design into a precise statement of what we are allowed to conclude. Two design features — random sampling and random assignment — control the two questions above. Get them straight and you will never be fooled by a headline again.
Everything in Phase 1 — sampling methods (Lesson 6), experiments (Lesson 7) — comes together in one idea: the scope of inference. The scope of inference is the set of conclusions a study's design actually supports. It answers two separate questions:
The crucial insight of this lesson: these two questions are controlled by two different design choices, and they are independent of each other.
Random sampling means the individuals studied were selected from a population using chance (an SRS, stratified, cluster, or systematic sample — Lesson 6). Random sampling makes the sample representative, so statistics computed from it estimate the population's parameters. Random sampling is what permits generalizing to the population.
Random assignment means that, among the individuals in the study, chance decided who received each treatment (Lesson 7). Random assignment balances out all other variables — known and unknown — across the treatment groups, so the groups are comparable before treatment. Random assignment is what permits a cause-and-effect conclusion.
Key distinction — do not confuse these. Random sampling is about getting individuals into the study (it controls generalization). Random assignment is about what you do with the individuals once they are in the study (it controls causation). A study can have one, both, or neither.
Because the two features are independent, there are four combinations. This table is the heart of the lesson — memorize the structure, not the examples.
| **Random assignment used** (experiment) | **No random assignment** (observational) | |
|---|---|---|
| Random sample from population | Cause-and-effect, generalizes to population. Ex: From a city's voter roll, 600 adults are randomly chosen; each is randomly assigned to receive a reminder text or not. Texted voters turn out more. → The texts caused higher turnout, for this city's adults. | Association only, generalizes to population. Ex: 600 adults randomly chosen from the voter roll are surveyed; those who use social media report higher turnout. → Social-media use is associated with turnout in this city, but we cannot say it causes it. |
| Not a random sample (volunteers / convenience) | Cause-and-effect, but only for these subjects. Ex: 80 volunteer students are randomly assigned to a new study app or the old one; the app group scores higher. → The app caused higher scores for these 80 students; cannot generalize beyond them. | Association only, only for these subjects. Ex: 80 volunteer students report their app use; heavy users score higher. → A link exists among these volunteers — neither causal nor generalizable. The weakest design. |
Read the table by its two axes:
So the four cells are read as a pair of independent decisions. The gold standard — top-left — is a randomized experiment on a random sample, which is rare and powerful: it supports a causal claim about an entire population. It is rare because random sampling typically requires a list of the whole population, while experiments usually rely on volunteers for ethical and practical reasons.
Suppose coffee drinkers live longer (an observational finding). The problem is a confounding variable: a variable associated with both the explanatory variable and the response, so that its effect cannot be separated from the explanatory variable's effect. Maybe coffee drinkers exercise more. Exercise is linked to coffee drinking and to lifespan, so we cannot tell whether coffee or exercise extends life. A closely related idea is a lurking variable — a variable not measured in the study that influences the relationship. (A lurking variable becomes a confounder once we recognize it is tangled with the explanatory variable.)
Random assignment defeats confounding. If we randomly assign people to "drink coffee" or "don't," then exercisers, smokers, the wealthy, and everyone else are spread roughly evenly across both groups by chance. The groups start out alike in every respect except the treatment, so any difference in the response can be attributed to the treatment. No such protection exists in an observational study, where people sort themselves into groups by their own characteristics — which is exactly how confounders sneak in.
This is why only an experiment with random assignment supports cause-and-effect, and why an observational study — even one built on a beautiful random sample — supports association only.
On the AP exam, your verb must match your design:
A real association can arise for several reasons: X causes Y, Y causes X (reverse causation), a confounder causes both, or pure coincidence. An observational study cannot tell these apart — which is the whole reason the causal verb is off-limits.
How data are collected raises ethical obligations, and the AP CED expects you to recognize them. Five pillars:
Ethics is not a side note. A study with a flawless design but no consent, or one whose author hides inconvenient results, is statistically and morally compromised — and the AP exam will ask you to spot it.
Problem. A nutrition researcher obtains a list of all 4,000 students at a high school and selects an SRS of 100. She surveys them about daily breakfast habits and grades. Students who eat breakfast daily have higher mean GPA. What can she conclude?
Strategy. Locate the study in the 2×2. Random sample? Yes — SRS from the full student list. Random assignment? No — she only observed habits; nobody was assigned to eat breakfast. So: top row, right column.
Solution. Random sample → generalize to the population (all 4,000 students). No random assignment → association only.
Interpretation. "Among students at this high school, eating breakfast daily is associated with higher GPA. We cannot conclude that breakfast causes higher GPA, because students were not randomly assigned to eat breakfast — a confounder such as overall self-discipline or home stability could drive both." The result generalizes to the school but stops short of causation.
Problem. A gym recruits 60 members who volunteer for a study. Each is randomly assigned to a high-intensity interval program or a steady-cardio program for 8 weeks. The interval group loses significantly more weight. What is the scope of inference?
Strategy. Random sample? No — volunteers, a convenience group. Random assignment? Yes — chance assigned the programs. So: bottom row, left column.
Solution. Random assignment → cause-and-effect. No random sample → only for these subjects.
Interpretation. "For these 60 volunteers, the interval program caused greater weight loss than steady cardio. Because the subjects were volunteers rather than a random sample, we cannot generalize this causal finding to all gym members or all adults." The causal claim is valid but its reach is limited to the people in the experiment.
Problem. Two teams investigate whether a meditation app reduces anxiety.
For each team, state what may be concluded.
Strategy. Classify each in the 2×2. Team A: random sample, no assignment → top-right. Team B: no random sample, random assignment → bottom-left.
Solution & interpretation.
The payoff: neither study is "better" outright. Team A generalizes but can't claim cause; Team B claims cause but can't generalize. To get both, you would need a randomized experiment on a random sample — the rare top-left cell.
Problem. A company studying employee productivity secretly logs which websites its workers visit, links each record to the worker's name, and publishes a report claiming "social-media use lowers productivity" — omitting the 30% of workers whose data contradicted the claim. Identify the ethical problems.
Strategy. Check each pillar against the scenario.
Solution.
Interpretation. Beyond the ethics, the cherry-picking also destroys the study's validity: a conclusion built on a hand-picked subset is not trustworthy. Note too that even with clean data, this observational design could only show association, not that social media causes lower productivity.
Mistake 1: Claiming causation from observational data. Students see a strong link in survey data and write "X causes Y." Why it's wrong: without random assignment, a confounding variable can produce the link. Fix: if no one was randomly assigned to treatments, use "associated with" — never "causes."
Mistake 2: Over-generalizing from a convenience or volunteer sample. Students conclude something about "all teenagers" from a study of 50 volunteers. Why it's wrong: volunteers and convenience groups are not representative, so statistics from them needn't match the population. Fix: without random sampling, restrict the conclusion to the subjects actually studied.
Mistake 3: Conflating random sampling with random assignment. Students think "the study was random, so we can say it causes the outcome" — or "it was randomized, so it generalizes." Why it's wrong: the two randomizations do different jobs. Sampling controls generalization; assignment controls causation. Fix: check for both separately, and match each to its conclusion.
Mistake 4: Forgetting that an experiment alone doesn't generalize. A perfect randomized experiment on volunteers earns a causal claim but not a population claim. Fix: keep the two questions separate even when a strong design tempts you to claim everything.
Mistake 5: Ignoring ethics because the design "looks fine." A clean experiment with no informed consent is still a violation. Fix: always check consent, confidentiality, oversight, and honest reporting separately from the statistical design.
1 (MC). Random assignment to treatments in a study is what primarily allows researchers to:
2 (MC). Random selection of individuals from a population primarily allows researchers to:
3 (MC). A researcher takes an SRS of 300 employees from a large company and surveys them, finding that those who telecommute report higher job satisfaction. The most appropriate conclusion is:
4 (MC). Which study design supports BOTH a cause-and-effect conclusion AND generalization to a population?
5 (MC). A variable that is associated with both the explanatory and response variables, so its effect cannot be separated from the explanatory variable's, is called a:
6 (MC). 200 patients with chronic pain volunteer for a trial and are randomly assigned to a new drug or a placebo. The drug group reports less pain. The valid conclusion is:
7 (MC). A bar graph of two companies' revenues uses a y-axis starting at $90 million instead of $0, making Company A's bar look three times taller than Company B's though revenues are $98M and $95M. This is an example of:
8 (MC). A study links each subject's responses to their name but promises never to release identifiable data publicly. This study is best described as:
9 (Short answer). A psychologist randomly samples 400 adults from a city and finds that those who sleep at least 7 hours report better mood. Can she conclude that more sleep causes better mood? Can she generalize to the city's adults? Justify both answers.
10 (Short answer). Researchers randomly assign 90 volunteer gardeners to use Fertilizer A or Fertilizer B and find Fertilizer A produces taller plants. State the scope of inference (causation? generalization?) and justify each part.
11 (Short answer, in-context). A school newspaper reports: "Students in band have higher GPAs, so joining band raises your GPA." The data came from comparing all band members' GPAs to non-members' GPAs. Identify the flaw in the causal claim and name a plausible confounding variable.
12 (Short answer, in-context). A pharmaceutical company runs a well-designed randomized experiment but publishes only the trials where its drug performed well, quietly discarding two trials where it did not. Name the ethical pillar violated and explain why the published conclusion is untrustworthy.
13 (Short answer). Explain, in your own words, why random assignment — not random sampling — is what allows a cause-and-effect conclusion.
Directions: This question is worth 10 points. Show your reasoning and answer in context.
The Study. A city's public-health department wants to know whether a new 6-week mindfulness program reduces stress among the city's high-school teachers. From a complete roster of the city's 1,800 high-school teachers, researchers select a random sample of 120 teachers, all of whom agree to participate after being told what the study involves. The 120 teachers are then randomly assigned: 60 to complete the mindfulness program and 60 to a control group that continues their usual routine. After 6 weeks, each teacher completes a validated stress questionnaire. The mindfulness group has a significantly lower mean stress score than the control group. All responses are recorded with ID numbers, and the link between ID and name is kept private by the research team.
(a) Identify the two randomizations used in this study and state what each one permits the researchers to conclude. (4 points)
(b) State the complete scope of inference: can the researchers conclude that the mindfulness program caused the reduction in stress, and can they generalize the result to the city's high-school teachers? Justify each answer. (3 points)
(c) Suppose instead the researchers had simply surveyed the 120 randomly sampled teachers, asking whether they already practiced mindfulness, and found that those who did had lower stress. How would this change what they could conclude, and why? (2 points)
(d) Identify one way the study protects participants ethically. (1 point)
(a) The study uses random sampling and random assignment.
(b) Yes to both.
(c) With a survey only (no random assignment), the study becomes observational. The researchers could conclude only that practicing mindfulness is associated with lower stress among the city's teachers — not that it causes lower stress — because a confounding variable (e.g., teachers with lighter workloads might both practice mindfulness and have less stress) could explain the link. Generalization to the city's teachers would still hold, because the sample is still random.
(d) Any one of: teachers gave informed consent (they agreed after being told what the study involved); the data are confidential (the ID-to-name link is kept private by the team).
Part (a) — 4 points:
Part (b) — 3 points:
Part (c) — 2 points:
Part (d) — 1 point:
Where students lose points:
1. (B). Random assignment balances confounders across treatment groups, enabling cause-and-effect. (A) is the job of random sampling, not assignment. (C) assignment doesn't change required sample size. (D) a control group is still needed for comparison.
2. (B). Random selection makes the sample representative, enabling generalization. (A) is the job of random assignment. (C) confounding is controlled by random assignment, not sampling. (D) sampling doesn't fix group sizes.
3. (C). Random sample → generalize to the company; no random assignment → association only. (A) and (B) wrongly claim causation from an observational study. (D) wrongly limits a random-sample result to just the 300 surveyed.
4. (C). Only a randomized experiment (→ causation) on a random sample (→ generalization) supports both — the top-left cell. (A) observational → no causation. (B) volunteers → no generalization. (D) neither.
5. (B). That is the definition of a confounding variable. (A) the response is the measured outcome. (C) a treatment is what's applied in an experiment. (D) a parameter describes a population.
6. (B). Random assignment → causation; volunteers → only these 200 subjects. (A) over-generalizes beyond the volunteers. (C) "associated" understates a randomized experiment, and over-generalizes. (D) too strong — a valid causal claim exists for the subjects.
7. (B). A y-axis not starting at zero exaggerates a tiny difference — a classic misleading graphic. (A) confounding is about variables, not graphs. (C) and (D) are unrelated concepts.
8. (B). Researchers can link responses to names but promise not to release identifiable data → confidential. (A) anonymous would mean even the researchers can't identify respondents. (C)/(D) nothing in the description indicates these failures.
9. Causation — no. It is an observational study (no random assignment); a confounder such as overall health or work schedule could cause both more sleep and better mood. So sleep is only associated with better mood. Generalization — yes. The 400 adults are a random sample of the city's adults, so the association generalizes to the city's adult population.
10. Causation — yes: gardeners were randomly assigned to fertilizers, so the groups were comparable and the difference in plant height can be attributed to the fertilizer. Generalization — no: the gardeners were volunteers, not a random sample, so the causal result applies only to these 90 gardeners and cannot be extended to all gardeners.
11. The claim asserts causation from observational data (band members were not randomly assigned to join band), so a confounding variable can explain the GPA gap. Plausible confounder: students with stronger academic motivation/time-management (or supportive families) may be both more likely to join band and more likely to earn high GPAs. Correct claim: band membership is associated with higher GPA.
12. Violated pillar: honest reporting (selective/cherry-picked reporting, hiding unfavorable trials). The published conclusion is untrustworthy because suppressing the failed trials biases the overall picture — readers see only successes and overestimate the drug's effect; the full set of trials might show no real benefit.
13. Random assignment makes the treatment groups comparable on every other variable (known and unknown) by spreading those variables evenly across groups by chance. So when the groups differ in the response, the treatment is the only systematic difference left, and we can attribute the effect to it — confounding is ruled out. Random sampling only ensures the studied individuals represent a population; it does nothing to make treatment groups comparable, so by itself it cannot rule out confounding and cannot establish cause.
FRQ rubric (restated): (a) 4 pts — 1 each for naming random sampling, its generalization role, random assignment, and its causal role. (b) 3 pts — causation-yes (justified by random assignment), generalization-yes (justified by random sample), contextual population conclusion. (c) 2 pts — observational → association only (no random assignment), and generalization still holds (random sample intact). (d) 1 pt — one valid ethical protection (informed consent or confidentiality) tied to the scenario. Total: 10.
---
StatsIQ · Lesson 8 of 30 · Unit 1: Exploring One-Variable Data & Collecting Data · Phase 1: Data & Design
This lesson is study material for the May 2027 AP Statistics exam (new 5-unit CED) and is not affiliated with or endorsed by the College Board. AP® is a registered trademark of the College Board.
Accuracy review: All scope-of-inference classifications, causation/association language, and ethics definitions in this lesson have been checked against the 2×2 design framework. No numerical computations appear in this conceptual lesson. Reviewed for statistical accuracy by a retired actuary.