In 1936, the most respected poll in America, run by Literary Digest magazine, predicted that Republican Alf Landon would crush President Franklin Roosevelt. The magazine mailed out ten million straw-poll ballots and got 2.4 million back — a staggering sample. Roosevelt won 46 of 48 states. The poll wasn't just wrong; it was one of the most lopsided misses in polling history.
That same year, a young pollster named George Gallup predicted a Roosevelt win using a sample of only about fifty thousand — a tiny fraction of the Digest's. He was right. The magazine folded soon after.
How did 2.4 million responses lose to about fifty thousand? Today's lesson is about the answer — and it is the single most important idea in reading any poll: how you choose the people you ask matters far more than how many you ask. Get that backwards and a giant sample is worthless.
Public opinion is the collection of attitudes, beliefs, and preferences that citizens hold about politics, government, policies, and public figures. It is not one thing — it is millions of individual views, and it can be intense or shallow, stable or shifting. Because no one can interview an entire nation, we estimate public opinion by surveying a small slice of it and generalizing to the whole. That estimation is the science of polling, and whether the estimate is any good depends entirely on the method.
A poll is scientific when it is built to let a small group accurately represent a much larger one. Four features make that possible.
1. A random (probability) sample. The foundation. In a random sample — also called a probability sample — every member of the target population has a known, nonzero chance of being selected. Randomness is what prevents the pollster's (or the respondents') choices from skewing who gets asked. The classic image is drawing names from a hat where every name is in the hat. Modern pollsters approximate this with techniques like random-digit dialing or randomly drawn address lists. Randomness is non-negotiable: it is the mathematical thing that lets you generalize from a sample to a population at all.
2. A representative sample. A good sample mirrors the population it is meant to describe — similar mix of age, race, gender, region, education, and party. A random draw tends to produce a representative sample, but pollsters check and correct it (see weighting, below). If a poll meant to describe all U.S. adults is 80% senior citizens, it is not representative, and its numbers are suspect no matter how randomly those seniors were chosen.
3. An adequate sample size. You need enough people for the estimate to be statistically stable. Most national polls survey around 1,000–1,500 respondents — which sounds small for a country of over 330 million but is plenty, because precision depends on the number sampled, not the share of the population. A well-drawn sample of 1,000 describes the United States about as precisely as it would describe a single city.
4. Careful question wording. Even a perfect sample is ruined by bad questions (see below).
In Practice. A poll that nails the first three pillars can still mislead if it asks 1,000 people only in one wealthy suburb, or interviews them only on landlines at 2 p.m. (missing younger, mobile, working voters). The pillars work together: a random method, applied to a representative frame, at an adequate size, with neutral questions. Drop any one and the estimate wobbles.
Because a poll asks a sample rather than everyone, its result is an estimate, not the exact truth. The margin of error (also called sampling error) measures how much the sample's result could differ from the true value in the whole population, purely by the chance of who got drawn into the sample.
A poll reporting "52% approval, margin of error ±3 points" is telling you: the true approval rating in the full population is most likely somewhere between 49% and 55% (52 minus 3 to 52 plus 3). The precise statistical meaning, at the usual 95% confidence level, is that if the same poll were run over and over with fresh random samples, about 95% of those polls would land within ±3 points of the true value. It is not a guarantee that the truth is in that band, and it does not mean 95% of people agree — it is a statement about the reliability of the sampling method.
Bigger samples shrink the margin. Margin of error gets smaller as sample size grows — but slowly. It shrinks with the square root of the sample size, so to cut the margin in half you must roughly quadruple the sample. A sample of ~1,000 yields a margin of about ±3 points; pushing to ±1.5 points takes about 4,000 respondents. That diminishing return is why pollsters usually stop around 1,000–1,500: chasing a smaller margin costs a fortune for little gain.
What the margin does NOT cover. This is the trap almost everyone falls into. The margin of error captures only sampling error — the random luck of the draw. It says nothing about errors from a biased sample, a leading question, people who refuse to answer, or respondents who lie. A poll can report a tiny ±2 margin and still be wildly wrong if its sample is unrepresentative. The margin measures one kind of uncertainty, not all of it.
A poll of 1,000 likely voters finds Candidate A at 49% and Candidate B at 46%, with a margin of error of ±3 points. A leads by 3. Is A really ahead?
Apply the margin to each figure. A's true support could be anywhere from 46% to 52%; B's could be anywhere from 43% to 49%. Those ranges overlap — A could truly be at 46% while B is truly at 49%, which would make B the leader. Because the 3-point gap is no larger than the margin of error, we cannot be confident A is actually ahead. Pollsters call this a statistical tie.
A useful rule of thumb: if the gap between two figures is within the margin of error, treat the race as tied. (A technical note for the careful student: the uncertainty in the difference between two candidates is actually somewhat larger than the ±3 on a single number, so a lead should be comfortably bigger than the margin — not just one point over it — before you call it solid. The simple "is the gap inside the margin?" check is the version the AP exam expects, and it errs on the safe side.)
How you ask changes what you hear. Pollsters guard against several distortions: - Leading questions push respondents toward an answer ("Don't you agree that wasteful spending should be cut?"). - Framing changes responses by emphasis: "death tax" versus "estate tax" describe the same policy but poll differently. - Social-desirability bias makes people give the answer they think is acceptable rather than their true view (under-reporting unpopular attitudes, over-reporting things like voting and church attendance).
Reputable pollsters use neutral, balanced wording and rotate answer order to limit these effects.
A non-scientific poll abandons random sampling. Straw polls and self-selected online or call-in polls (sometimes labeled "SLOP" — self-selected listener opinion polls) let people choose to participate, so only the motivated respond. They are entertainment, not measurement — exactly the flaw that doomed the Literary Digest. To correct for samples that come in slightly skewed, scientific pollsters use weighting: statistically adjusting the data so under-represented groups (say, young voters who were harder to reach) count in proportion to their true share of the population.
Public opinion influences policymakers, who track polls to gauge what voters will reward or punish, and it can grant or withhold the public mandate behind a policy. But it also constrains them, and its power has limits. Opinion is often uninformed, unstable, or contradictory (people may want lower taxes and more services); it can be manipulated by how questions and issues are framed; and the framers deliberately built a republic, not a direct democracy, so that elected representatives filter raw opinion rather than simply obey each week's poll. Reading polls critically — checking the method before trusting the number — is the real skill this lesson teaches.
Context. Federalist No. 10, written by James Madison in 1787, is best known for its argument about controlling factions. But it also contains the founders' theory of what to do with public opinion — and that theory is why polling matters in the particular way it does in the United States.
Key quote (Madison, 1787):
The effect of [a republic] is "to refine and enlarge the public views, by passing them through the medium of a chosen body of citizens, whose wisdom may best discern the true interest of their country."
What it means. Madison drew a sharp line between a pure democracy, where citizens govern directly, and a republic, where the people govern through elected representatives. He argued the republic is safer precisely because it does not translate raw public opinion straight into law. Representatives are supposed to refine opinion — deliberate, weigh long-term interests, and resist the momentary passions that a snapshot poll captures. Public opinion is the ultimate source of authority (the principle of popular sovereignty — government rests on the consent of the governed), but it reaches policy filtered through institutions, not unmediated.
How it's used on the AP exam. Use Fed 10 to explain the tension a polling question often raises: should a representative follow every poll? Madison's answer is no — and that connects directly to the delegate vs. trustee debate (Lesson 8). A "delegate" mirrors current opinion; a "trustee" exercises independent judgment, exactly the "refining" Madison praised. On an FRQ about whether officials should govern by poll, Fed 10 is your evidence that the Constitution intentionally puts a deliberative buffer between public opinion and law — while still grounding all authority in popular sovereignty. Pair it with the delegate/trustee models for a complete answer.
No required Supreme Court case attaches to public opinion and polling — this is a methods-and-beliefs lesson, not a constitutional-law lesson. One pointer worth holding onto for the comparison skill: the federal judiciary is deliberately insulated from public opinion. Federalist No. 78 defends life tenure precisely so judges can rule on the law rather than the latest poll, and cases like Engel v. Vitale (1962) — which struck down school-sponsored prayer when most Americans favored it — show the Court acting in a counter-majoritarian way, against majority opinion. When a question asks you to compare a polling scenario to a case, that counter-majoritarian theme (Fed 78; an unpopular-but-constitutional ruling) is the connection to reach for.
Use the move: Identify the issue → State the principle → Apply it → Reach a conclusion.
Scenario 1 — Evaluating a poll's quality. A website posts a poll: "We asked our visitors, and 78% oppose the new tax. Click to vote!" Over 200,000 people responded. A news anchor cites it as proof the public opposes the tax.
Scenario 2 — Is the lead real? A scientific poll of 1,200 likely voters shows Candidate X at 47% and Candidate Y at 45%, margin of error ±3 points. A headline declares "X Takes the Lead."
Scenario 3 — Question wording. A group opposed to a highway project commissions a poll asking, "Do you support wasting millions of taxpayer dollars on an unnecessary highway?" It reports 70% opposition.
Scientific vs. non-scientific polls. A poll is scientific only if it uses a random (probability) sample. Call-in, write-in, click-to-vote, and "ask our visitors" polls are self-selected and non-scientific — useless for measuring the public, regardless of how many respond. Sample quality beats sample size.
Margin of error — the 2-point "lead." A 2-point lead with a ±3-point margin is a statistical tie, not a lead, because the candidates' ranges overlap. To be meaningful, a lead must exceed the margin of error (and, strictly, exceed it comfortably). Don't report a within-margin gap as a real lead — that's the single most common polling misread.
Sampling error vs. sampling bias. Sampling error is the random, unavoidable variation from sampling instead of counting everyone; it shrinks with a bigger sample and is measured by the margin of error. Sampling bias is a systematic skew from a non-representative sample (self-selection, undercoverage, nonresponse); a bigger sample does not fix it — the Literary Digest's 2.4 million responses were still biased. The margin of error measures sampling error only; it tells you nothing about bias.
Correlation vs. causation in opinion data. If a poll shows churchgoers lean one way and you conclude churchgoing causes that view, you've confused correlation with causation. Polls reveal associations, not causes; a third factor (region, age, upbringing) may drive both. On data questions, describe the relationship the numbers show — don't claim one variable caused the other unless the data establish it.
1. B. A scientific poll requires a random (probability) sample. A overrates size; C describes a self-selected (non-scientific) poll; D would be unrepresentative.
2. A. Polling precision depends on the number sampled, not the population share, so ~1,000 is adequate for any large population. B is false; C is not guaranteed and not required by law; D is arithmetically false (1,000 is far less than 1% of 330 million).
3. B. With a ±3 margin, the true value most likely lies within 51–57% (54 ± 3). A ignores sampling uncertainty; C and D misstate the margin.
4. B. The Literary Digest sample was self-selected and unrepresentative (drawn from car and phone owners, then only those who chose to mail back ballots), skewing it toward wealthier, anti-Roosevelt voters. A is the opposite of the truth (2.4 million is huge); the failure was bias, not wording (C) or a large margin (D).
5. B. A larger sample decreases the margin of error (by roughly the square root of the sample size). It does not fix sampling bias (D) — that's the key distinction.
6. B. Weighting statistically adjusts data so under-represented groups count in proportion to their true share. A and C are unrelated; D is a non-scientific poll.
7. C. A benchmark poll establishes a baseline at the start of a campaign. A tracks change over time; B surveys voters leaving the polls; D is a non-scientific straw poll.
8. B. "Don't you agree that wasteful spending should be cut?" is a leading question with loaded framing. A, C, and D use neutral wording.
9. B. Over-reporting socially approved behavior (voting, religious attendance) is social-desirability bias. A and C concern sampling uncertainty; D is a sampling method.
10. B. Sampling error is random and shrinks with a larger sample; sampling bias is systematic and is not cured by a larger sample. A, C, and D all misstate the relationship (note C reverses it — the margin measures error, not bias).
| Candidate | Support | Margin of error |
|---|---|---|
| Candidate R | 48% | ±3 points |
| Candidate S | 46% | ±3 points |
Which conclusion is best supported by the data?
11. B. The 2-point gap is within the ±3 margin of error, so the race is a statistical tie. A and C overstate a within-margin gap; D is false — closeness does not make a poll non-scientific.
| Month | Approval | Margin of error |
|---|---|---|
| January | 44% | ±3 points |
| February | 45% | ±3 points |
| March | 51% | ±3 points |
| April | 52% | ±3 points |
Which statement is most defensible?
12. B. The roughly 6–7-point rise from February (45%) to March/April (51–52%) exceeds the ±3 margin, so it is a defensible real increase. A ignores that change; C is wrong because the gap exceeds the margin; D claims causation the data do not establish.
13. B. Inferring that aging causes the preference from a mere age–support association confuses correlation with causation; a generational or other factor could drive both. A is wrong (no causal proof); C and D misname the issue.
14. B. The Court ruling against a clear majority's preference illustrates Federalist No. 78's defense of an independent judiciary that decides on the law, not on public opinion (the counter-majoritarian role). A and D misstate their sources; C is unrelated to judicial independence.
15. B. "Refine and enlarge the public views" supports representatives exercising independent (trustee) judgment rather than mechanically following polls. A is the delegate model Madison's phrase moves beyond; C overstates it (representatives still answer to voters); D is not in Fed 10.
This is FRQ 2, Quantitative Analysis: read the data, then identify, conclude, and connect to a course concept. It is scored here on 3 points (Parts A, B, and C).
The data. The table below reports results from a series of scientific national polls of U.S. adults on the question "Do you favor or oppose stricter funding for public infrastructure?" Each poll surveyed roughly 1,000 respondents and carried a margin of error of ±3 percentage points at the 95% confidence level. (Figures are illustrative and rounded for instructional use.)
| Group / Year | % Favor (2024) | % Favor (2026) | Margin of error |
|---|---|---|---|
| Adults age 18–29 | 61% | 68% | ±3 points |
| Adults age 65+ | 49% | 50% | ±3 points |
Prompt. Using the data in the table, respond to the following.
(A) Identify the age group that showed the larger change in support between 2024 and 2026.
(B) Using the data and the margin of error, draw a defensible conclusion about whether each group's change in support reflects a real shift in opinion or could be a statistical tie.
(C) Explain how the concept of political socialization or public opinion's influence on policymakers could account for, or follow from, a difference shown in the data.
(A) — 1 point. Adults age 18–29 showed the larger change. Their support rose 7 points (61% to 68%), while support among adults 65+ rose only 1 point (49% to 50%).
(B) — 1 point. Apply the ±3-point margin of error to each figure. - For the 18–29 group, the 2024 value could be as high as 64% and the 2026 value as low as 65% — but the 7-point gap exceeds the ±3 margin (in fact it exceeds twice the margin), so the increase is larger than sampling error can explain. This is a defensible real shift toward greater support. - For the 65+ group, the change is only 1 point. With a ±3 margin on each year, the 2024 and 2026 ranges (46–52% and 47–53%) overlap almost entirely, so the difference is well within the margin of error — a statistical tie, not a measurable change.
A correct conclusion: support among young adults rose meaningfully, while support among older adults was essentially flat.
(C) — 1 point. Political socialization — the lifelong process by which people acquire political values from agents such as family, schools, peers, and media — can account for the divergence. Younger adults' opinions are typically less fixed and more responsive to recent events, peer networks, and new information, so a shift in the broader environment moved the 18–29 group while the more settled views of the 65+ group held steady. Alternatively, a student may connect to public opinion's influence on policymakers: a clear, above-margin surge in support among young voters gives elected officials a real electoral incentive to back infrastructure funding, since they track which positions a voting bloc will reward — whereas the flat senior numbers create no such pressure.
Total: 3 points.
Common point-loss: - Part A: Failing to actually identify a group, or naming the 65+ group. A correct identification needs no explanation but must be the 18–29 group; citing the specific changes (7 points vs. 1 point) makes it airtight. - Part B: Drawing a conclusion without using the margin of error, or misusing it. The question explicitly requires students to weigh the ±3 margin: the 7-point change is real because it exceeds the margin; the 1-point change is a tie because it falls within it. Reversing this — calling the 1-point change "real" or the 7-point change "within the margin" — earns no credit. So does describing the numbers with no reference to the margin at all. - Part C: Naming a concept without linking it to the data. "Political socialization is how people learn values" earns nothing on its own; the response must connect the concept to why one group moved and the other did not (or how the gap would shape policy). Vague answers that don't tie back to the table do not score.
1. B. A scientific poll requires a random (probability) sample. A overrates size; C describes a self-selected (non-scientific) poll; D would be unrepresentative.
2. A. Polling precision depends on the number sampled, not the population share, so ~1,000 is adequate for any large population. B is false; C is not guaranteed and not required by law; D is arithmetically false (1,000 is far less than 1% of 330 million).
3. B. With a ±3 margin, the true value most likely lies within 51–57% (54 ± 3). A ignores sampling uncertainty; C and D misstate the margin.
4. B. The Literary Digest sample was self-selected and unrepresentative (drawn from car and phone owners, then only those who chose to mail back ballots), skewing it toward wealthier, anti-Roosevelt voters. A is the opposite of the truth (2.4 million is huge); the failure was bias, not wording (C) or a large margin (D).
5. B. A larger sample decreases the margin of error (by roughly the square root of the sample size). It does not fix sampling bias (D) — that's the key distinction.
6. B. Weighting statistically adjusts data so under-represented groups count in proportion to their true share. A and C are unrelated; D is a non-scientific poll.
7. C. A benchmark poll establishes a baseline at the start of a campaign. A tracks change over time; B surveys voters leaving the polls; D is a non-scientific straw poll.
8. B. "Don't you agree that wasteful spending should be cut?" is a leading question with loaded framing. A, C, and D use neutral wording.
9. B. Over-reporting socially approved behavior (voting, religious attendance) is social-desirability bias. A and C concern sampling uncertainty; D is a sampling method.
10. B. Sampling error is random and shrinks with a larger sample; sampling bias is systematic and is not cured by a larger sample. A, C, and D all misstate the relationship (note C reverses it — the margin measures error, not bias).
11. B. The 2-point gap is within the ±3 margin of error, so the race is a statistical tie. A and C overstate a within-margin gap; D is false — closeness does not make a poll non-scientific.
12. B. The roughly 6–7-point rise from February (45%) to March/April (51–52%) exceeds the ±3 margin, so it is a defensible real increase. A ignores that change; C is wrong because the gap exceeds the margin; D claims causation the data do not establish.
13. B. Inferring that aging causes the preference from a mere age–support association confuses correlation with causation; a generational or other factor could drive both. A is wrong (no causal proof); C and D misname the issue.
14. B. The Court ruling against a clear majority's preference illustrates Federalist No. 78's defense of an independent judiciary that decides on the law, not on public opinion (the counter-majoritarian role). A and D misstate their sources; C is unrelated to judicial independence.
15. B. "Refine and enlarge the public views" supports representatives exercising independent (trustee) judgment rather than mechanically following polls. A is the delegate model Madison's phrase moves beyond; C overstates it (representatives still answer to voters); D is not in Fed 10.
| Pt | Part | Awarded when the response… |
|---|---|---|
| 1 | A — Identify | Correctly identifies the 18–29 group as showing the larger change (citing 7 points vs. 1 point strengthens but is not required). |
| 2 | B — Draw a conclusion using the margin of error | Uses the ±3 margin to conclude the young-adult change is a real shift (gap exceeds the margin) while the senior change is within the margin / a statistical tie. |
| 3 | C — Connect to a course concept | Explains how political socialization (or public opinion's influence on policymakers) accounts for or follows from a difference in the data, explicitly linked to the table. |
Always defer to the official College Board rubric for your exam year. The AP Gov Quantitative Analysis FRQ asks students to identify data, describe/draw a conclusion from it, and explain how it connects to a political principle, institution, or behavior — using the numbers, not just naming them.
GovIQ · Lesson 20 of 25 · Unit 4: American Political Ideologies & Beliefs
This lesson is exam-prep material and is not affiliated with, endorsed by, or sponsored by the College Board, which produces the AP® US Government and Politics exam. AP® is a registered trademark of the College Board. Polling figures in the practice problems and FRQ are illustrative and rounded for instruction; the Literary Digest (1936) and Gallup examples are historical. Foundational document quotations are drawn from public-domain texts.
Content pending external review (government/poli-sci reviewer).