Lesson 23: Two-Sample Inference for Proportions

(a) Warm-Up

In Lessons 20 and 22 you worked with a single proportion: one sample, one p̂, one population. But most interesting questions in the real world are comparisons. Does the new vaccine lower the infection rate compared to a placebo? Does platform A get a higher click-through rate than platform B? Do men and women support a policy at different rates?

These are all questions about two proportions, and the thing we actually care about is the difference, p₁ − p₂.

Here's a quick gut-check before we dive in. Suppose a study finds that 14% of a vaccine group got infected versus 22% of a placebo group. The difference in the samples is 0.14 − 0.22 = −0.08. Does that 8-percentage-point gap prove the vaccine works? Or could a gap that size show up just from random chance, even if the vaccine did nothing?

That single question — "is the observed difference real, or is it noise?" — is what this entire lesson answers. We have two tools: a confidence interval (how big is the difference?) and a significance test (is there a difference at all?). The catch that trips up half of all students: the two tools compute their standard error differently. Keep that in your sights.

(b) Core Concept

The parameter and the statistic

We now have two populations (or two groups), each with its own true proportion: p₁ and p₂. From each we take a sample and compute a sample proportion, p̂₁ and p̂₂. Our estimate of the difference p₁ − p₂ is simply:

p̂₁ − p̂₂

That's the point estimate for both the interval and the test. Everything else is about measuring its uncertainty.

The conditions (check ALL of them, for BOTH groups)

Before any two-proportion procedure, verify:

1. Random: Both samples are random samples, OR both groups come from a randomized experiment.
2. Independent (the 10% condition): When sampling without replacement, each sample is less than 10% of its population — n₁ ≤ 0.10·N₁ and n₂ ≤ 0.10·N₂. The two groups must also be independent of each other (not paired).
3. Large Counts — for BOTH groups, all four counts ≥ 10:

n₁p̂₁ ≥ 10, n₁(1−p̂₁) ≥ 10, n₂p̂₂ ≥ 10, n₂(1−p̂₂) ≥ 10.

That last one is the most-missed condition in this whole lesson. You must check four counts, not two. One group passing does not let the other off the hook.

The standard error: this is where pooling lives

Here is the single most important distinction in Lesson 23.

For the CONFIDENCE INTERVAL — use the UNPOOLED standard error. Each group keeps its own p̂:

SE = sqrt( p̂₁(1−p̂₁)/n₁ + p̂₂(1−p̂₂)/n₂ )

Why unpooled? When we build an interval, we are not assuming the two proportions are equal — we're trying to estimate how different they are. So each group's variability is computed from its own sample proportion.

For the SIGNIFICANCE TEST — use the POOLED standard error. The test starts from the null hypothesis H₀: p₁ = p₂. If that's true, the two groups share one common proportion, and our best estimate of that shared value combines all the successes from both groups:

p̂_c = (x₁ + x₂) / (n₁ + n₂)

Then the standard error uses that single pooled proportion for both groups:

SE = sqrt( p̂_c(1−p̂_c)(1/n₁ + 1/n₂) )

Why pooled? Because under H₀ we are assuming the proportions are equal, so it would be inconsistent to estimate two separate variances. We pool to get the best single estimate of the variability the null model predicts.

The one-sentence rule: Interval → unpooled (two separate p̂s). Test → pooled (one p̂_c). Mix them up and your SE — and everything downstream — is wrong.

The confidence interval

(p̂₁ − p̂₂) ± z·SE where SE is unpooled and z is the critical value (1.96 for 95%, 1.645 for 90%, 2.576 for 99%).

Interpretation template: "We are C% confident that the interval from ___ to ___ captures the true difference in proportions, p₁ − p₂."

The "0 in the interval" idea: If the interval contains 0, then "no difference" (p₁ = p₂) is a plausible value — we don't have convincing evidence the proportions differ. If the interval is entirely above 0, we have evidence p₁ > p₂; entirely below 0, evidence p₁ < p₂. The sign and width tell you direction and magnitude.

The significance test

Hypotheses (the parameter is the difference):

• H₀: p₁ = p₂ (equivalently p₁ − p₂ = 0)
• Hₐ: p₁ ≠ p₂ (two-sided), or p₁ > p₂ / p₁ < p₂ (one-sided)

Test statistic (pooled SE, and note the null says the true difference is 0, so it drops out of the numerator):

z = (p̂₁ − p̂₂) / SE

Find the p-value from the standard normal curve, compare to α, and conclude in context.

FULL PANIC — Example A: a CONFIDENCE INTERVAL (unpooled SE)

Scenario. A clinical trial randomly assigns 600 volunteers to two groups. Of 300 who received a new vaccine, 42 got infected over the season. Of 300 who received a placebo, 66 got infected. Build a 95% confidence interval for the difference in infection rates, p_vaccine − p_placebo.

P — Parameter. Let p₁ = true infection proportion for the vaccine group and p₂ = true infection proportion for the placebo group. We want to estimate p₁ − p₂.

A — Assumptions/Conditions.

• Random: Volunteers were randomly assigned to the two groups (randomized experiment). ✓
• Independent: The two treatment groups are independent; random assignment supports this. (10% condition is about random sampling from a population; in a randomized experiment we rely on the randomization.) ✓
• Large Counts (all four ≥ 10):

n₁p̂₁ = 42 ≥ 10, n₁(1−p̂₁) = 258 ≥ 10, n₂p̂₂ = 66 ≥ 10, n₂(1−p̂₂) = 234 ≥ 10. ✓

N — Name the procedure. A two-sample z-interval for a difference of proportions (2-PropZInt).

I — Interval.

p̂₁ = 42/300 = 0.14
p̂₂ = 66/300 = 0.22
p̂₁ − p̂₂ = 0.14 − 0.22 = −0.08

UNPOOLED SE = sqrt( 0.14(0.86)/300 + 0.22(0.78)/300 )
            = sqrt( 0.0004013 + 0.0005720 )
            = sqrt( 0.0009733 )
            = 0.03120

95% CI = −0.08 ± 1.96(0.03120)
       = −0.08 ± 0.06115
       = (−0.1412, −0.0189)

TI-84: STAT → TESTS → 2-PropZInt
Input: x1=42, n1=300, x2=66, n2=300, C-Level=0.95
Output: (−0.14115, −0.01885)
        p̂1 = 0.14,  p̂2 = 0.22

C — Conclusion in context. We are 95% confident that the interval from −0.1412 to −0.0189 captures the true difference in infection proportions (vaccine − placebo). Because the entire interval is below 0, we have convincing evidence that the vaccine group has a lower infection rate than the placebo group — by somewhere between about 1.9 and 14.1 percentage points.

FULL PANIC — Example B: a SIGNIFICANCE TEST (pooled SE)

Scenario. A company tests two versions of an ad. Of 250 users shown ad A, 58 clicked; of 240 users shown ad B, 40 clicked. Is there a significant difference in click-through rates? Use α = 0.05.

P — Parameter. Let p₁ = true click-through proportion for ad A and p₂ = true click-through proportion for ad B. We test the difference p₁ − p₂.

A — Assumptions/Conditions.

• Random: Users were randomly assigned to see ad A or ad B. ✓
• Independent: The two groups are independent; each user saw only one ad. ✓
• Large Counts (all four ≥ 10):

n₁p̂₁ = 58 ≥ 10, n₁(1−p̂₁) = 192 ≥ 10, n₂p̂₂ = 40 ≥ 10, n₂(1−p̂₂) = 200 ≥ 10. ✓

N — Name the procedure. A two-sample z-test for a difference of proportions (2-PropZTest).

• H₀: p₁ = p₂
• Hₐ: p₁ ≠ p₂ (two-sided — we just ask whether they differ)

I — Test (POOLED SE).

p̂₁ = 58/250 = 0.232
p̂₂ = 40/240 = 0.16667
p̂₁ − p̂₂ = 0.06533

POOLED proportion:
p̂_c = (58 + 40)/(250 + 240) = 98/490 = 0.20000

POOLED SE = sqrt( 0.20(0.80)(1/250 + 1/240) )
          = sqrt( 0.16 × 0.0081667 )
          = sqrt( 0.00130667 )
          = 0.036148

z = (0.232 − 0.16667) / 0.036148 = 0.06533 / 0.036148 = 1.807

p-value = 2 · P(Z > 1.807) = 2(0.03535) = 0.0707

TI-84: STAT → TESTS → 2-PropZTest
Input: x1=58, n1=250, x2=40, n2=240, p1: ≠ p2
Output: z = 1.8074,  p = 0.0707
        p̂1 = 0.232,  p̂2 = 0.16667,  p̂_c = 0.20

C — Conclusion in context. Since the p-value 0.0707 > α = 0.05, we fail to reject H₀. We do not have convincing evidence of a difference in click-through rates between ad A and ad B. (Note: this is not proof the ads are identical — see Common Mistake #4.)

(c) Worked Examples

Worked Example 1 (build & interpret a difference interval)

Problem. An education researcher compares pass rates in an online course versus an in-person course. Of 200 online students, 171 passed; of 180 in-person students, 153 passed. Construct a 90% confidence interval for p_online − p_inperson and interpret it.

Strategy. Difference interval → unpooled SE, z* = 1.645 for 90%.

Solution.

p̂₁ = 171/200 = 0.855      p̂₂ = 153/180 = 0.850
p̂₁ − p̂₂ = 0.005

UNPOOLED SE = sqrt( 0.855(0.145)/200 + 0.850(0.150)/180 )
            = sqrt( 0.00061988 + 0.00070833 ) = sqrt(0.00132821) = 0.036445

90% CI = 0.005 ± 1.645(0.036445) = 0.005 ± 0.05995 = (−0.0549, 0.0649)

2-PropZInt with x1=171, n1=200, x2=153, n2=180, C-Level=0.90 → (−0.0549, 0.0649).

Interpretation. We are 90% confident the true difference in pass rates (online − in-person) is between −0.055 and 0.065. The interval contains 0, so we do not have convincing evidence that the two pass rates differ — "no difference" is plausible.

Worked Example 2 (run a two-proportion test — decide pooled vs unpooled)

Problem. In a poll, 120 of 200 men and 152 of 220 women support a new policy. Is there a significant difference between men's and women's support? Use α = 0.05.

Strategy. We're testing H₀: p₁ = p₂ → use the pooled proportion in the SE.

Solution.

p̂₁ = 120/200 = 0.600 (men)      p̂₂ = 152/220 = 0.6909 (women)
p̂₁ − p̂₂ = −0.09091

p̂_c = (120 + 152)/(200 + 220) = 272/420 = 0.64762
POOLED SE = sqrt( 0.64762(0.35238)(1/200 + 1/220) )
          = sqrt( 0.228222 × 0.0095455 ) = sqrt(0.0021785) = 0.046673

z = −0.09091 / 0.046673 = −1.948
p-value = 2 · P(Z < −1.948) = 2(0.02572) = 0.0514

2-PropZTest (x1=120, n1=200, x2=152, n2=220, p1 ≠ p2) → z = −1.948, p = 0.0514.

Conclusion. Since p = 0.0514 > 0.05, we fail to reject H₀. We do not have convincing evidence of a difference in support between men and women — though it's a very close call (just barely above 0.05).

Worked Example 3 (interpret "0 in the interval" vs a test)

Problem. A 95% CI for p_A − p_B comes out to (0.003, 0.197). A colleague claims "since the interval doesn't include 0, a two-proportion test at α = 0.05 would reject H₀: p_A = p_B." Is the colleague right? What does the interval say in plain language?

Strategy. Connect the interval to the two-sided test at the matching confidence level.

Solution. The colleague is essentially right in spirit. A 95% CI corresponds to a two-sided test at α = 0.05. Since the interval (0.003, 0.197) lies entirely above 0, the value p_A − p_B = 0 is not in the plausible range, so a two-sided test would reject H₀ at the 5% level. (One subtlety for the exam: the interval uses the unpooled SE and the test uses the pooled SE, so the correspondence is very close but not algebraically exact when the result sits right on the boundary. Don't claim the test "must" reject from the interval on the AP exam — run the test if asked.)

Plain language. We're 95% confident p_A exceeds p_B by between 0.3 and 19.7 percentage points. Group A's proportion is convincingly higher than group B's.

Worked Example 4 (catch the pooled/unpooled trap)

Problem. A student builds a confidence interval for p₁ − p₂ but writes SE = sqrt( p̂_c(1−p̂_c)(1/n₁ + 1/n₂) ) using a pooled proportion. What's wrong, and what should they have written?

Solution. Pooling is only justified under H₀: p₁ = p₂ — an assumption we make in a test. A confidence interval makes no such assumption; it's estimating how different the proportions are. The correct interval SE keeps each p̂ separate:

SE = sqrt( p̂₁(1−p̂₁)/n₁ + p̂₂(1−p̂₂)/n₂ ).

Using the pooled SE in an interval will give the wrong margin of error and can cost points on the FRQ.

(d) Common Mistakes

1. Using the wrong SE (pooled vs unpooled). The single biggest error. Interval → unpooled (two separate p̂s). Test → pooled p̂_c. If you write a pooled SE for an interval, or two separate p̂s in a test, your standard error is wrong and every number after it is wrong too.

2. Checking Large Counts for only one group. You must verify all four counts: n₁p̂₁, n₁(1−p̂₁), n₂p̂₂, n₂(1−p̂₂), each ≥ 10. A common slip is checking the two successes and forgetting the two failures — or checking group 1 and assuming group 2 is fine.

3. Interpreting the difference backwards. If you define p₁ − p₂ as vaccine − placebo and get a negative interval, that means the vaccine rate is lower. Students often flip the direction in their sentence. State up front which group is "1" and which is "2," and keep that order all the way through.

4. Concluding "no difference" / "the proportions are equal" from failing to reject. Failing to reject H₀ means we lack convincing evidence of a difference — it does not prove the proportions are equal. Never write "the proportions are the same" or "we proved p₁ = p₂." Say: "we do not have convincing evidence of a difference."

5. Forgetting context and direction in the conclusion. A bare "reject H₀" earns little. You must name the groups, the variable, and (for an interval entirely above/below 0 or a one-sided test) the direction of the difference.

(e) Practice Problems

1 (MC). For a confidence interval for p₁ − p₂, which standard error is correct?

• (A) sqrt( p̂_c(1−p̂_c)(1/n₁ + 1/n₂) )
• (B) sqrt( p̂₁(1−p̂₁)/n₁ + p̂₂(1−p̂₂)/n₂ )
• (C) sqrt( p̂₁(1−p̂₁)/n₁ ) + sqrt( p̂₂(1−p̂₂)/n₂ )
• (D) sqrt( p̂(1−p̂)/n ) with p̂ = (p̂₁+p̂₂)/2

2 (MC). A two-proportion significance test pools the proportion. With x₁=80, n₁=200, x₂=60, n₂=200, the pooled p̂_c is:

• (A) 0.30 (B) 0.35 (C) 0.40 (D) 0.70

3 (MC). A 95% CI for p₁ − p₂ is (−0.04, 0.12). The correct conclusion is:

• (A) p₁ is convincingly greater than p₂.
• (B) p₂ is convincingly greater than p₁.
• (C) There is no convincing evidence of a difference, since 0 is in the interval.
• (D) The proportions are exactly equal.

4 (MC). A two-proportion z-test gives z = 1.95, Hₐ: p₁ ≠ p₂. The p-value is closest to:

• (A) 0.025 (B) 0.051 (C) 0.975 (D) 0.95

5 (MC). Which is NOT required for two-proportion inference?

• (A) Both samples are random / from a randomized experiment.
• (B) All four of n₁p̂₁, n₁(1−p̂₁), n₂p̂₂, n₂(1−p̂₂) are ≥ 10.
• (C) n₁ = n₂.
• (D) The two groups are independent of each other.

6 (MC). Failing to reject H₀: p₁ = p₂ means:

• (A) We have proven p₁ = p₂.
• (B) We lack convincing evidence that p₁ ≠ p₂.
• (C) p₁ and p₂ are definitely different.
• (D) The sample sizes were too small to use.

7 (MC). For x₁=45, n₁=150, x₂=30, n₂=150, the unpooled SE for a CI equals (to 3 d.p.):

• (A) 0.037 (B) 0.050 (C) 0.071 (D) 0.099

8 (Short answer). A nutrition study finds 88 of 400 people on treatment had high cholesterol vs 124 of 400 on control. State H₀ and Hₐ for a two-sided test, and identify which SE (pooled or unpooled) you'd use.

9 (Short answer, in context). A streaming service tests two homepage layouts. Layout A: 210 of 500 users subscribed. Layout B: 168 of 480 users subscribed. Construct a 95% CI for p_A − p_B and state, in context, whether there's convincing evidence of a difference.

10 (Short answer, in context). Using the data in #9, suppose instead you want to test whether the layouts differ at α = 0.05. Compute the pooled p̂_c, the pooled SE, the z-statistic, and the p-value, then conclude in context.

11 (Short answer). Explain in one or two sentences why a confidence interval uses an unpooled SE but a significance test uses a pooled SE.

12 (MC). A 99% CI for p₁ − p₂ is (0.02, 0.18). If you instead built a 95% CI from the same data, the new interval would be:

• (A) Wider, and might now include 0.
• (B) Narrower, and still entirely above 0.
• (C) The same width.
• (D) Centered at a different point.

(f) FRQ Practice (10 points)

A quality-control engineer wants to know whether a new manufacturing process reduces the defect rate compared with the old process. In a randomized trial, 400 units were produced with the new process and 88 were defective. Another 400 units were produced with the old process and 124 were defective.

(a) Do these data provide convincing evidence at the α = 0.05 level that the new process has a lower defect rate than the old process? Carry out an appropriate significance test. (8 points)

(b) Based on your conclusion in part (a), which type of error — Type I or Type II — could you have made, and what would that error mean in this context? (2 points)

Model Response

Part (a).

P — Parameter. Let p₁ = true defect proportion for the new process and p₂ = true defect proportion for the old process. We test:

• H₀: p₁ = p₂
• Hₐ: p₁ < p₂ (the new process has a lower defect rate)

A — Assumptions/Conditions.

• Random: Units were produced in a randomized trial. ✓
• Independent: The two groups of units are independent. ✓
• Large Counts (all four ≥ 10): 400(0.22)=88, 400(0.78)=312, 400(0.31)=124, 400(0.69)=276 — all ≥ 10. ✓

N — Name. Two-sample z-test for a difference of proportions (2-PropZTest), pooled SE.

I — Test.

p̂₁ = 88/400 = 0.22       p̂₂ = 124/400 = 0.31
p̂₁ − p̂₂ = −0.09

p̂_c = (88 + 124)/(400 + 400) = 212/800 = 0.265
POOLED SE = sqrt( 0.265(0.735)(1/400 + 1/400) )
          = sqrt( 0.194775 × 0.005 ) = sqrt(0.000973875) = 0.031207

z = −0.09 / 0.031207 = −2.884
p-value = P(Z < −2.884) = 0.0020   (one-sided)

2-PropZTest (x1=88, n1=400, x2=124, n2=400, p1 < p2) → z = −2.884, p = 0.0020.

C — Conclusion. Since the p-value 0.0020 < α = 0.05, we reject H₀. We have convincing evidence that the new manufacturing process has a lower true defect rate than the old process.

Part (b). Because we rejected H₀, the only error we could have made is a Type I error — rejecting a true null. In context: concluding the new process has a lower defect rate when in reality the two processes have the same defect rate.

Rubric (10 points total)

Part (a) — 8 points:

Part (b) — 2 points:

Where students lose points:

• Using the unpooled SE in the test (the interval SE) — costs the SE point and usually the z point. The test requires pooling.
• Forgetting to verify all four counts, or checking only the defective counts — loses a condition point.
• Wrong direction in Hₐ (writing ≠ or > instead of <) or doubling a one-sided p-value — loses points for hypotheses and/or p-value.
• Bare conclusion ("reject H₀") with no context or direction — loses the conclusion point.
• Part (b): saying Type II error. Since we rejected, only a Type I error is possible. Naming the error without the contextual meaning earns only 1 of 2.