Lesson 29: Regression Analysis

(a) Warm-Up

You've spent most of this course on one variable at a time — one proportion, one mean, one distribution. But the most interesting questions in the real world link two quantitative variables. Does studying more raise your quiz score? Does an older car sell for less? Does more screen time go with less sleep?

Here's a quick gut check. A coach records, for eight players, the hours per week each one practices free throws and how many of 50 free throws they then make. The data trend upward: more practice tends to go with more makes. Two natural questions follow. First, how strong is that relationship, and can we put a single number on it? Second, if a new player practices 5 hours a week, what should we predict they'll make — and how far can we trust that prediction?

This lesson gives you the whole regression toolkit: scatterplots, the correlation r, the least-squares line, predictions, residuals, and r². The calculator will hand you the numbers in seconds. Your job — and the AP exam's whole focus — is to give those numbers meaning, in context, with units.

(b) Core Concept

Step 1 — Make and describe a scatterplot

When you have two quantitative variables, the first move is always a scatterplot. We put the explanatory variable (the one we think predicts or explains) on the x-axis and the response variable on the y-axis.

We describe a scatterplot with three (sometimes four) features:

• Form — Is the overall pattern roughly a straight line (linear) or does it curve (nonlinear)? Regression with a line is only appropriate if the form is linear.
• Direction — As x increases, does y tend to increase (positive association) or decrease (negative association)?
• Strength — How closely do the points cluster around the pattern? Strong (tight), moderate, or weak (loosely scattered).
• Unusual features — Note any points that fall far from the overall pattern, or any gaps/clusters. Describe them; on this exam you do not apply special rules to remove or reclassify them.

Let's use a worked dataset for the whole section: ten students' hours studied (x) and quiz score (y, out of 100).

[GRAPH: Scatterplot of Quiz Score vs. Hours Studied. X-axis: "Hours Studied" (0–11). Y-axis: "Quiz Score" (50–95). Ten points trending upward in a roughly straight band: (1,57), (2,56), (3,67), (4,62), (5,72), (6,73), (7,82), (8,76), (9,89), (10,86). The least-squares line ŷ = 52.07 + 3.62x is drawn through the cloud, rising from about 56 at x=1 to about 88 at x=10. Association is positive, linear, and strong.]

Description: The association is positive (more hours → higher score), the form is linear (a straight line fits the trend), and it is strong (points cluster tightly around the line).

Step 2 — The correlation r

Correlation r measures the strength and direction of a linear relationship between two quantitative variables. Memorize its properties — the AP exam loves them:

• r is always between −1 and 1: −1 ≤ r ≤ 1.
• The sign of r matches the direction: positive r = positive association, negative r = negative.
• |r| near 1 = strong linear pattern; near 0 = weak/no linear pattern.
• r has no units and does not change if you change units (inches → cm) or swap which variable is x.
• r only measures linear strength. A perfect U-shaped curve can have r ≈ 0 even though x and y are perfectly related.
• r is not resistant — a single far-off point can pull it dramatically.
• Correlation ≠ causation. A strong r from observational data never, by itself, proves that x causes y.

For our data, the calculator gives r = 0.9471. Interpretation: there is a strong, positive, linear relationship between hours studied and quiz score.

Step 3 — The least-squares regression line

The least-squares regression line (LSRL) is the line that makes the sum of the squared residuals as small as possible. We write it

ŷ = a + bx

where ŷ ("y-hat") is the predicted response, a is the y-intercept, and b is the slope.

On the TI-84:

TI-84: enter x into L1, y into L2
STAT → CALC → 8:LinReg(a+bx)   Xlist:L1  Ylist:L2
(turn on DiagnosticOn first so r and r² show)
Output:  a = 52.0667   b = 3.6242   r² = 0.8971   r = 0.9471

So the LSRL is ŷ = 52.07 + 3.62x (rounding for reporting).

Step 4 — Interpret the slope and intercept in context, with units

This is where points are won and lost. Generic templates:

• Slope: "For each additional 1 [x-unit], the predicted [y-variable] increases/decreases by b [y-units], on average."
• Intercept: "When [x] = 0 [x-units], the predicted [y-variable] is a [y-units]."

Slope b = 3.62: For each additional hour studied, the predicted quiz score increases by about 3.62 points, on average.

Intercept a = 52.07: A student who studies 0 hours is predicted to score about 52.07 points on the quiz. (Notice x = 0 is just barely inside our data, so this interpretation is reasonable here — but always ask whether x = 0 makes sense.)

Step 5 — Make a prediction

To predict, plug an x-value into the line. For a student who studies 7 hours:

ŷ = 52.07 + 3.62(7) = 52.07 + 25.34 = 77.41 points

Extrapolation warning. Our data only covers 1 to 10 hours. Predicting outside that range — say, for 20 hours — is extrapolation, and the linear pattern may simply not hold out there. Never trust a prediction far beyond the observed x-values.

Step 6 — Residuals

A residual is how far a real data point sits above or below the line:

residual = observed − predicted = y − ŷ

The student who studied 7 hours actually scored 82. Their predicted score was 77.41, so:

residual = 82 − 77.41 = +4.59

A positive residual means the point sits above the line (the model under-predicted). A negative residual means the point is below the line. The LSRL is built so the residuals always sum to zero.

Step 7 — Residual plots: is a line even appropriate?

A residual plot graphs the residuals (y-axis) against x (or against ŷ). It is the single best tool for checking whether a linear model fits:

• Random scatter above and below zero, with no pattern → a linear model is appropriate.
• A clear pattern (a curve, a U-shape, fanning out) → a linear model is not appropriate; the true relationship is nonlinear or the spread changes.

[GRAPH: Residual plot for the study-hours data. X-axis: "Hours Studied" (0–11). Y-axis: "Residual" (−6 to +6) with a horizontal dashed line at 0. Points: (1,−0.6),(2,−3.3),(3,+4.1),(4,−4.6),(5,+1.8),(6,−0.8),(7,+4.6),(8,−5.1),(9,+4.3),(10,−2.3). Residuals scatter randomly above and below zero with no clear curve or trend.]

This residual plot shows random scatter with no pattern, confirming that a linear model is appropriate for these data.

Step 8 — The coefficient of determination r²

Square the correlation to get r², the coefficient of determination. Here r² = (0.9471)² = 0.8971, about 89.7%. The required interpretation:

r² = the proportion (or percent) of the variation in the response variable y that is explained by the linear relationship with x.

In context: About 89.7% of the variation in quiz scores is explained by the linear relationship with hours studied. (The remaining ~10.3% is due to other factors and natural scatter.) Note r² is always between 0 and 1 and is never negative — so it can't tell you the direction; only r and the slope do that.

(c) Worked Examples

Example 1 (easy) — Describe a scatterplot and name r

A used-car dealer records the age (years) and price ($1000s) of eight cars of the same model.

[GRAPH: Scatterplot of Price vs. Age. X-axis: "Age (years)" (0–9). Y-axis: "Price ($1000s)" (5–30). Eight points falling steadily from (1,28) down to (8,9) in a tight straight line. Strong negative linear association.]

Strategy. Describe form, direction, strength; then anticipate the sign of r.

Solution. The pattern is linear, negative (older cars cost less), and strong (very tight). The calculator gives r = −0.994. Because the association is negative, r is negative; because it's nearly perfect, |r| is near 1.

Interpretation. There is a strong, negative, linear relationship between a car's age and its price.

Example 2 (medium) — Interpret slope and intercept, then predict + residual

Using the same car data, the TI-84 returns:

LinReg(a+bx):  a = 30.000   b = −2.583   r² = 0.987

So ŷ = 30.00 − 2.58x (price in $1000s, age in years).

Slope b = −2.58: For each additional year of age, the predicted price decreases by about 2.58 thousand dollars ($2,580), on average.

Intercept a = 30.00: A car that is 0 years old (brand new) is predicted to cost about $30,000.

Predict the price of a 5-year-old car:

ŷ = 30.00 − 2.58(5) = 30.00 − 12.90 = 17.10  → about $17,100

Residual for the actual 5-year-old car, which sold for $17,000 (y = 17):

residual = y − ŷ = 17 − 17.10 = −0.10  → about −$100

The negative residual means that car sold for slightly less than the line predicted; it sits just below the line.

Example 3 (medium) — Interpret r and r² correctly

A study of 40 cities finds a correlation of r = 0.6 between average daily coffee shops per capita and average commute time.

(a) Interpret r. There is a moderate, positive, linear relationship: cities with more coffee shops per capita tend to have longer commutes.

(b) Compute and interpret r². r² = (0.6)² = 0.36. About 36% of the variation in commute time is explained by the linear relationship with coffee shops per capita.

(c) Can we conclude more coffee shops cause longer commutes? No. This is observational data, so correlation does not imply causation. A lurking variable — like city population/density — could drive both more coffee shops and longer commutes.

Example 4 (AP-style) — Read a residual plot to judge the model

A biologist fits a least-squares line predicting a plant's height from the number of days since planting and produces the residual plot below.

[GRAPH: Residual plot of Height residuals vs. Days. X-axis: "Days Since Planting" (0–30). Y-axis: "Residual (cm)" (−8 to +8) with a horizontal line at 0. Points form a clear ∩-shaped curve: negative residuals at low days, rising to strongly positive residuals in the middle, then falling back to negative residuals at high days.]

Strategy. Decide whether the residual plot shows random scatter (linear OK) or a pattern (linear not OK).

Solution. The residuals show a clear curved (∩-shaped) pattern, not random scatter: the line under-predicts in the middle and over-predicts at both ends.

Interpretation. A linear model is not appropriate for these data. The strong pattern in the residual plot signals that the true relationship between days and height is nonlinear (it curves), so predictions from the line would be systematically off. (On this exam, your job stops at this judgment — you are not asked to transform the data or fit a curved model.)

(d) Common Mistakes

1. Interpreting the slope with no context or no units. Writing "the slope is 3.62, so it goes up by 3.62" earns nothing. You must name the variables and units and say "predicted… on average": "for each additional hour studied, the predicted quiz score rises about 3.62 points." Also include the word predicted — the slope describes the line's behavior, not a guarantee for any one student.

2. Claiming causation from correlation. A strong r from an observational study never proves x causes y. Only a randomized experiment supports a cause-and-effect claim. Watch for lurking variables.

3. Confusing r and r². r = 0.9 is a strong correlation; r² = 0.81 means 81% of the variation is explained. They are different numbers with different interpretations. Also, r² is never negative, so it can't tell you direction — if asked for direction, report r or the slope's sign.

4. Extrapolating. Plugging in an x far outside the data range and trusting the answer. The line may be nonsense out there (e.g., a free-throw model predicting "62 made out of 50"). State the data's x-range and stay inside it.

5. Misreading a residual plot. Students see any residual plot and say "linear is fine," or they panic at random scatter. The rule: random, patternless scatter → linear model is appropriate; a clear curve/pattern → linear model is not appropriate. A strong-looking scatterplot is not enough on its own; the residual plot is the deciding evidence.

(e) Practice Problems

MC 1. A scatterplot shows points falling tightly from upper-left to lower-right. Which value of r is most plausible?

(A) r = 0.95 (B) r = 0.10 (C) r = −0.92 (D) r = −0.15

MC 2. A regression line is ŷ = 2.1 + 0.85x. What is the predicted y when x = 20?

(A) 17.0 (B) 19.1 (C) 22.0 (D) 0.85

MC 3. Using ŷ = 2.1 + 0.85x, a data point at x = 20 has an observed value of y = 17. What is its residual?

(A) +2.1 (B) +19.1 (C) −2.1 (D) −19.1

MC 4. Which is the correct interpretation of a slope b = −4.5 in a model predicting gas mileage (mpg) from vehicle weight (1000 lbs)?

(A) Heavier cars have a correlation of −4.5 with mileage.

(B) For each additional 1000 lbs of weight, predicted mileage decreases by 4.5 mpg, on average.

(C) A car weighing 0 lbs gets 4.5 mpg.

(D) 4.5% of the variation in mileage is explained by weight.

MC 5. A correlation between two variables is r = −0.7. What is r², and what does it mean?

(A) 0.49; 49% of the variation in y is explained by the linear relationship with x.

(B) −0.49; the relationship is negative.

(C) 0.49; 49% of points fall on the line.

(D) 0.84; the relationship is strong.

MC 6. Which statement about correlation r is true?

(A) r has the same units as the response variable.

(B) r is resistant to unusual points.

(C) A strong r from an observational study proves causation.

(D) r measures the strength and direction of a linear relationship and lies between −1 and 1.

MC 7. A residual plot of y − ŷ versus x shows a clear U-shaped (curved) pattern. This indicates that:

(A) the correlation must be exactly 0.

(B) a linear model is not appropriate for the data.

(C) the slope of the regression line is negative.

(D) there is a calculation error, since residuals must be random.

MC 8. A study reports a strong positive correlation between number of firefighters sent to a blaze and the damage in dollars. The best explanation is:

(A) Firefighters cause the damage.

(B) The correlation is a mistake.

(C) A lurking variable — the size of the fire — drives both.

(D) Reducing firefighters would reduce damage.

Short Answer 9. A scatterplot of hours of sleep (x) vs. reaction time in ms (y) for 12 people is roughly linear and slopes downward, with the points loosely scattered. The calculator gives r = −0.62.

(a) Describe the association (form, direction, strength).

(b) Interpret r² in context.

Short Answer 10. For a model ŷ = 14.68 + 3.15x predicting free throws made (out of 50) from weekly practice hours:

(a) Interpret the slope in context with units.

(b) Predict the makes for a player who practices 5 hours.

(c) Explain why predicting the makes for a player who practices 15 hours per week would be unreliable.

(f) FRQ Practice (10 points)

Practices assessed: 3 — Analyze Data; 4 — Interpret Results.

A basketball coach believes that players who practice free throws more during the week make more of them in games. For a random sample of 8 players on the team, she records the number of hours each practices free throws per week (x) and the number of free throws each makes out of 50 attempts the following game (y).

A least-squares regression analysis on the calculator produces:

LinReg(a+bx):  a = 14.68   b = 3.15   r = 0.961   r² = 0.923

A scatterplot shows a positive, linear, strong pattern, and the residual plot is shown below.

[GRAPH: Residual plot of Makes residuals vs. Practice Hours. X-axis: "Practice Hours" (0–9). Y-axis: "Residual" (−4 to +4) with a horizontal line at 0. Points: (1,+0.2),(2,−1.0),(3,+2.9),(4,−3.3),(5,+2.5),(6,−2.6),(7,+1.2),(8,+0.1). Residuals scatter randomly above and below zero with no clear pattern.]

(a) Interpret the slope and the y-intercept of the regression line in context. (3 pts)

(b) Predict the number of free throws made for a player who practices 3 hours per week, and compute the residual for the actual player who practiced 3 hours (and made 27). (3 pts)

(c) Interpret the value of r² in context. (2 pts)

(d) The coach wants to predict the makes for a player who practices 15 hours per week. Using both the residual plot and the idea of extrapolation, explain whether the linear model is appropriate and whether this prediction would be trustworthy. (2 pts)

Model Response

(a) Slope and intercept.

• Slope b = 3.15: For each additional hour of weekly practice, the predicted number of free throws made (out of 50) increases by about 3.15, on average.
• Intercept a = 14.68: A player who practices 0 hours per week is predicted to make about 14.68 free throws out of 50.

(b) Prediction and residual.

ŷ = 14.68 + 3.15(3) = 14.68 + 9.45 = 24.13 free throws (predicted)
residual = observed − predicted = 27 − 24.13 = +2.87

The positive residual means this player made about 2.87 more free throws than the model predicted — the point sits above the line.

(c) r². r² = 0.923, so about 92.3% of the variation in free throws made is explained by the linear relationship with weekly practice hours. The remaining ~7.7% is due to other factors and natural variation.

(d) Model appropriateness + extrapolation. The residual plot shows random scatter with no clear pattern, so a linear model is appropriate for the observed range of data. However, the data only cover 1 to 8 practice hours. Predicting at 15 hours is extrapolation — far outside that range — so the linear pattern may not hold there. (In fact ŷ = 14.68 + 3.15(15) ≈ 61.9, which is more than 50 makes out of 50 — impossible.) The prediction would not be trustworthy.

Scoring Rubric (10 points)

Total: 10 points.

Where Students Lose Points

• Part (a): Slope with no units or no context ("goes up by 3.15"); omitting "predicted/average"; interpreting the intercept without saying it's at 0 hours.
• Part (b): Computing predicted but forgetting the residual; flipping the residual formula to ŷ − y (sign error); using the exact unrounded equation and marking the rounded answer "wrong" — consistency with shown work earns the point.
• Part (c): Saying "92.3% of the data points are on the line" or "92.3% of free throws"; dropping "variation"; dropping context. A bare "92.3% explained" with no variables is incomplete.
• Part (d): Saying the model is bad because the residual plot looks scattered (scatter is good); ignoring extrapolation entirely; concluding the prediction is fine because r² is high.