Lesson 24: Chi-Square Test for Homogeneity

(a) Warm-Up

You surveyed three separate random samples — 200 teenagers, 200 working adults, and 200 retirees — and asked each person how they mainly watch TV: Streaming, Cable, or Neither. You make a side-by-side bar graph and the bars look different across the three age groups. Teens lean heavily toward streaming; retirees lean toward cable.

Here's the question that should be nagging you: could those differences just be sampling noise? If the three age groups really watched TV the same way, you'd still expect three random samples to come out a little different. So how different is "too different to be chance"?

In Lesson 23 you compared two proportions. But now you have three groups and a categorical response with three categories — too much to squeeze into a single difference of proportions. You need one test that compares entire distributions across several groups at once. That test is the chi-square (χ²) test for homogeneity. Let's build it.

(b) Core Concept

When do you use this test?

Use the chi-square test for homogeneity when you have one categorical response variable measured on two or more groups (or populations) that came from separate samples — or from a randomized experiment with several treatment groups — and you want to know whether the distribution of that response is the same across all the groups.

Three signals that it's a homogeneity test:

• Several separate samples, one from each group (you fixed the group sizes in advance).
• One categorical response with two or more categories.
• The question is about comparing distributions, not about association within one sample.

Heads up — this is NOT a goodness-of-fit test. Goodness-of-fit (one sample, one variable, compared to a hypothesized set of proportions) is not on the new exam. And it's not a test for independence either (that's Lesson 25 — one sample, two variables measured on the same people). Homogeneity = several samples, one response variable, compare the distributions.

The data: a two-way table

Lay the data out as a two-way table of counts (never percents — the test runs on raw counts). Put the groups in the rows and the response categories in the columns:

The hypotheses (in words)

For homogeneity, write hypotheses in words about distributions, not symbols:

• H₀: The distribution of TV-watching preference (Streaming / Cable / Neither) is the same for teens, adults, and seniors.
• Hₐ: The distribution of TV-watching preference is not the same for all three age groups (at least one group differs).

Note: Hₐ is just "the distributions are not all the same." It does not say which group differs or how. The chi-square test is a single yes/no question about the whole table.

Expected counts: what H₀ predicts

If H₀ is true, every group has the same distribution — so the best single estimate of that common distribution comes from the column totals (the pooled response). Overall, 270/600 = 45% stream, 210/600 = 35% cable, 120/600 = 20% neither. Under H₀, each group of 200 should split in those same proportions.

The shortcut formula gives the same answer:

E = (row total × column total) / grand total

Work one cell fully — Teens / Streaming:

E = (row total × column total) / grand total
E = (200 × 270) / 600
E = 54000 / 600
E = 90

Do this for all nine cells. Because all three row totals equal 200, every row gets the same expected counts here:

The chi-square statistic

Now measure how far the observed (O) counts fall from the expected (E) counts:

χ² = Σ (O − E)² / E

Compute the contribution of each cell, (O − E)² / E, then add them all up. Two cells worked fully:

Teens / Streaming:  (120 − 90)² / 90 = (30)²/90 = 900/90  = 10.000
Teens / Cable:      (50 − 70)²  / 70 = (−20)²/70 = 400/70  =  5.714

Doing all nine cells:

χ² = 10.000 + 5.714 + 2.500 + 0 + 0 + 0 + 10.000 + 5.714 + 2.500
χ² = 36.429

Degrees of freedom

df = (r − 1)(c − 1) where r = number of rows (groups) and c = number of columns (response categories). The grand total does not matter for df.

df = (3 − 1)(3 − 1) = (2)(2) = 4

The p-value

The chi-square test is always right-tailed — a big χ² means big mismatch. Find the area to the right of your statistic:

p-value = χ²cdf(χ², 1E99, df)

χ²cdf(36.429, 1E99, 4) ≈ 0.0000002   (about 2.4 × 10⁻⁷)

Conclusion in context

Because p ≈ 0.0000002 < α = 0.05, we reject H₀. We have very strong evidence that the distribution of TV-watching preference is not the same across teens, adults, and seniors.

Which cells drove the result?

The chi-square test alone won't tell you how the groups differ — but the cell contributions will. The two biggest contributors (10.000 each) are Teens / Streaming (far more streamers than expected) and Seniors / Streaming (far fewer than expected). The "Adults" row contributed essentially nothing — adults landed right at the expected counts. So the differences are driven mainly by teens over-streaming and seniors under-streaming. Always report this when asked to describe the nature of the difference; never claim a specific group "differs" from a significant χ² alone without pointing to the contributions.

(c) Worked Examples

Worked Example 1 (easy) — Just the expected counts

Problem. Two high schools each drew a random sample of 100 seniors and recorded whether the student passed a state exam. Find the expected counts under H₀.

Strategy. Apply E = (row total × column total) / grand total to each cell.

Solution.

North/Pass: (100 × 140)/200 = 70      North/Fail: (100 × 60)/200 = 30
South/Pass: (100 × 140)/200 = 70      South/Fail: (100 × 60)/200 = 30

Interpretation. Under "the pass-rate distribution is the same at both schools," each school of 100 should produce about 70 passes and 30 fails (the pooled rate is 140/200 = 70%). North did better than expected, South worse. All four expected counts are ≥ 5, so the test will be valid.

Worked Example 2 (medium) — χ², df, and p-value with the full PANIC test

Problem. A chain took independent random samples of 100 customers at each of three cafés and asked their preferred drink: Coffee, Tea, or Other. Test at α = 0.05 whether the distribution of drink preference is the same across the three cafés.

P — Parameter / hypotheses.

• H₀: The distribution of drink preference (Coffee/Tea/Other) is the same at cafés A, B, and C.
• Hₐ: The distribution of drink preference is not the same across the three cafés.
• α = 0.05.

A — Assumptions (conditions).

• Random: Independent random samples were taken at each café. ✓
• 10%: Each sample of 100 is less than 10% of that café's customers. ✓
• Large counts: All expected counts ≥ 5 (computed below). ✓

Expected counts (E = row × col / grand):

Smallest expected count is 26.67 ≥ 5. ✓ Condition met.

N — Name the procedure. Chi-square test for homogeneity.

I — Inference (compute).

Largest contributions:
A/Coffee:  (45 − 33.33)²/33.33 = 4.083
C/Coffee:  (25 − 33.33)²/33.33 = 2.083
A/Other:   (20 − 26.67)²/26.67 = 1.667
... (all nine cells summed)
χ² = 10.25
df = (3 − 1)(3 − 1) = 4
p = χ²cdf(10.25, 1E99, 4) ≈ 0.0364

C — Conclusion in context. Since p ≈ 0.0364 < α = 0.05, we reject H₀. There is convincing evidence that the distribution of drink preference is not the same across the three cafés. The largest cell contributions come from Café A (more coffee than expected) and Café C (less coffee than expected), so the difference is driven mainly by how coffee preference varies between A and C.

Worked Example 3 (AP-style) — A randomized experiment, and a "fail to reject" finish

Problem. Researchers randomly assigned 180 patients with chronic pain to three treatments (Drug X, Drug Y, Placebo), 60 per group, and recorded each patient's outcome after 8 weeks: Improved, No Change, or Worse. Test at α = 0.05 whether the distribution of outcomes is the same across the three treatments.

P. H₀: the distribution of outcomes (Improved/No Change/Worse) is the same for all three treatments. Hₐ: it is not the same for all three. α = 0.05.

A. Random — patients were randomly assigned to treatments (a randomized experiment), so the Random condition is met. The 10% condition does not apply in a randomized experiment (we are not sampling without replacement from a population — note that). Large counts — expected counts below.

Expected counts: every treatment has row total 60, so each row is identical:

E(Improved) = 60 × 90/180 = 30
E(No Change) = 60 × 50/180 = 16.67
E(Worse)    = 60 × 40/180 = 13.33

Smallest expected count = 13.33 ≥ 5. ✓

N. Chi-square test for homogeneity.

I.

Biggest contributions:
X/Improved:       (38 − 30)²/30   = 2.133
Placebo/Improved: (22 − 30)²/30   = 2.133
Placebo/Worse:    (18 − 13.33)²/13.33 = 1.633
... (all nine cells)
χ² = 8.947
df = (3 − 1)(3 − 1) = 4
p = χ²cdf(8.947, 1E99, 4) ≈ 0.0624

C. Since p ≈ 0.0624 > α = 0.05, we fail to reject H₀. We do not have convincing evidence that the distribution of outcomes differs across the three treatments. (Notice the pattern looks suggestive — Drug X improved most, Placebo least — but with this sample size the differences are not strong enough to rule out chance. "Fail to reject" never means "the distributions are the same"; it means we lack evidence that they differ.)

(d) Common Mistakes

1. Running the test on percents instead of counts. Students convert each row to percentages and feed those in. The chi-square statistic depends on sample size — percents throw the counts away. Always use raw observed counts, and remember expected counts also come out as counts (decimals are fine: 26.67 is a legal expected count).

2. Using the wrong degrees of freedom. A common error is df = number of cells − 1 or df = n − 1. For a two-way table it is (rows − 1)(columns − 1). For the 3×3 tables above that's (2)(2) = 4, not 8 and not 599. The grand total never enters df.

3. Forgetting the "all expected counts ≥ 5" condition — and checking observed instead. The Large-Counts condition is about expected counts, not observed. A cell can have an observed count of 2 and still be fine if its expected count is ≥ 5. Always compute the expected counts and cite the smallest one.

4. Claiming which group differs (or "how") straight from a significant χ². A small p-value tells you the distributions differ somewhere — it does not identify the group or direction. To say how they differ you must point to the cell contributions (O−E)²/E and compare observed vs. expected. Don't write "teens prefer streaming more" as a conclusion of the test; write it as a follow-up supported by the contributions.

5. Mixing up homogeneity with independence or goodness-of-fit. Several separate samples + one response = homogeneity. One sample + two variables on the same people = independence (Lesson 25). One sample vs. a claimed distribution = goodness-of-fit, which is not on this exam. The math for homogeneity and independence is identical; the design and hypotheses wording differ.

(e) Practice Problems

MC 1. A homogeneity test compares one categorical response across 4 groups, with the response having 3 categories. The degrees of freedom are:

(A) 2 (B) 6 (C) 11 (D) 12

MC 2. In a two-way table the cell with row total 100, column total 80, and grand total 200 has expected count:

(A) 16 (B) 40 (C) 50 (D) 80

MC 3. A χ² test for homogeneity gives χ² = 14.2 with df = 6. Using χ²cdf(14.2, 1E99, 6) ≈ 0.0276, at α = 0.05 the correct decision is:

(A) Fail to reject H₀; the distributions are the same.

(B) Reject H₀; there is convincing evidence the distributions differ.

(C) Fail to reject H₀; there is convincing evidence the distributions differ.

(D) Reject H₀; the distributions are the same.

MC 4. Which scenario calls for a chi-square test for homogeneity?

(A) One random sample of 500 adults classified by both political party and region.

(B) One sample of 200 dice rolls compared to the expected uniform distribution.

(C) Separate random samples from three cities, each classified by preferred news source.

(D) Two means compared with a two-sample t-test.

MC 5. For the χ² test for homogeneity, the alternative hypothesis states:

(A) All group distributions are equal.

(B) A specific named group differs from the others.

(C) The distribution of the response is not the same for all groups.

(D) The two variables are independent.

MC 6. A test produces these cell contributions (O−E)²/E: 0.1, 0.2, 5.8, 0.3, 6.1, 0.0. The χ² statistic is:

(A) 6.1 (B) 5.8 (C) 12.5 (D) 2.08

MC 7. Increasing every observed count in a homogeneity table by a factor of 10 (10× more data, same proportions) will:

(A) Leave χ² unchanged. (B) Multiply χ² by about 10.

(C) Divide χ² by 10. (D) Multiply χ² by 100.

MC 8. A homogeneity test has one expected count of 3.4 while all observed counts exceed 8. The correct response is:

(A) Proceed; observed counts are all large enough.

(B) The Large-Counts condition is not met; the result may be unreliable.

(C) Switch to a goodness-of-fit test.

(D) Increase the degrees of freedom to compensate.

Short Answer 9 (in context). Three independent random samples of 100 commuters in three neighborhoods were asked their main commute mode: Car, Transit, or Bike. The table:

(a) State H₀ and Hₐ in words. (b) Compute the expected counts. (c) Find χ², df, and the p-value. (d) Conclude at α = 0.05.

Short Answer 10 (in context). Explain, in one or two sentences, why a researcher who gets a significant χ² for homogeneity across four brands cannot immediately conclude "Brand C is the one that's different" — and what they should examine instead.

(f) FRQ Practice (10 points)

Practices assessed: 3 — Analyze Data; 4 — Interpret Results.

A school district wants to know whether students' preferred social media platform differs by grade level. From three grade levels, the district takes separate independent random samples of 100 students each and records each student's single most-used platform: App-V (short video), App-P (photos), or App-T (text/chat).

(a) State the hypotheses for an appropriate test. (2 pts)

(b) Identify the test and verify that the conditions for inference are met. (3 pts)

(c) Calculate the test statistic, degrees of freedom, and p-value. (3 pts)

(d) State your conclusion in context at α = 0.05, and identify which cells contribute most to the result. (2 pts)

Model Response

(a) Hypotheses.

• H₀: The distribution of preferred platform (App-V / App-P / App-T) is the same across 9th, 10th, and 11th graders.
• Hₐ: The distribution of preferred platform is not the same for all three grade levels.

(b) Procedure and conditions. Chi-square test for homogeneity (one categorical response compared across three separately sampled groups).

• Random: Three independent random samples (one per grade) were taken. ✓
• 10%: Each sample of 100 students is less than 10% of all students in that grade. ✓
• Large Counts: All expected counts ≥ 5. Expected counts (each row total is 100, so all rows identical):

Smallest expected count = 25 ≥ 5. ✓

(c) Test statistic, df, p-value.

χ² = Σ (O − E)²/E
   = (55−40)²/40 + (30−35)²/35 + (15−25)²/25
   + (40−40)²/40 + (35−35)²/35 + (25−25)²/25
   + (25−40)²/40 + (40−35)²/35 + (35−25)²/25
   = 5.625 + 0.714 + 4.000 + 0 + 0 + 0 + 5.625 + 0.714 + 4.000
χ² = 20.68
df = (3 − 1)(3 − 1) = 4
p = χ²cdf(20.68, 1E99, 4) ≈ 0.00037

(d) Conclusion. Since p ≈ 0.00037 < α = 0.05, we reject H₀. There is convincing evidence that the distribution of preferred social media platform is not the same across the three grade levels. The largest cell contributions (about 5.6 each) come from 9th/App-V (far more short-video fans than expected) and 11th/App-V (far fewer than expected), so the difference is driven mainly by a decline in App-V preference and a rise in App-T preference from 9th to 11th grade.

Scoring Rubric (10 points)

Total: 10 points.

Where Students Lose Points

• Part (a): Writing Hₐ as "11th graders prefer App-T more" (too specific) or stating hypotheses about a single proportion instead of the whole distribution. Using symbols with no "in context" wording.
• Part (b): Calling it a goodness-of-fit or independence test; checking observed counts instead of expected; skipping the Random condition.
• Part (c): df = 8 (cells − 1) or df = 299; arithmetic slips in the χ² sum; reporting a left-tail probability instead of χ²cdf(χ², 1E99, df).
• Part (d): "Accept H₀" language; concluding the distributions are the same; rejecting but never tying p to α; or stopping at "they differ" without pointing to the cell contributions when asked how.