A USA Today article yesterday made some exaggerated claims about the effect of gasoline prices on President Bush’s approval ratings:
When it comes to President Bush’s approval rating — the number that measures his political health — one factor seems more powerful than any Oval Office address or legislative initiative.
It’s the price of a gallon of gas.
Statisticians who have compared changes in gas prices and Bush’s ratings through his presidency have found a steady relationship: As gas prices rise, his ratings fall. As gas prices fall, his ratings rise.
For some Americans, analysts speculate, gas prices provide a shorthand reading of the general state of the economy. Even though prices at the pump are largely outside the president’s control, he gets credit when they fall — and blame when they rise.
“Gas prices are a price everybody knows because it hangs on the street in big letters,” says Stuart Thiel, an economist at DePaul University in Chicago who has been tracking the trend for several years.
A statistical analysis by Doug Henwood, editor of the liberal newsletter Left Business Observer, found that an “uncanny” 78% of the movement in Bush’s ratings could be correlated with changes in gas prices. Based on trends in crude oil prices, Henwood predicted last Thursday that it “wouldn’t be surprising to see his approval numbers rise into the mid-40s.”
In a USA TODAY/Gallup Poll taken Friday through Sunday, Bush’s rating rose to 44%, his highest in a year. Average gas prices, which peaked at more than $3 a gallon in August, had dropped under $2.50, the lowest since March.
The problem, however, is that Henwood’s analysis is not correct, nor is he a statistician as Page’s lede implies (he actually did graduate work in English). To explain why, I’ll have to go through some statistics.
Henwood’s finding that “78% of the movement in Bush’s ratings could be correlated with changes in gas prices” is based on an incorrectly specified statistical model. He used the logarithm of nominal gas prices to predict President Bush’s approval rating. There are two problems with this, one conceptual and one technical. The conceptual problem is simple: 9/11. It boosted President Bush’s approval ratings into the stratosphere, and they’ve more or less declined consistently since then. Meanwhile, gas prices have trended upward over Bush’s presidency. The two series are correlated, but any variable that trended upward during this time period would show a similar relationship (I can “explain” 62 percent of the variance in Bush approval using a variable that just counts the number of months he has been in office). Second, the model is incorrectly specified (it suffers from what’s called serial correlation, which means the errors at time t and time t+1 are correlated).
When we estimate a quick-and-dirty model for presidential approval from January 2001-June 2006 using a lag of approval in the previous month (which corrects the serial correlation problem), a variable capturing the 9/11 approval boost in Sept./Oct. 2001, a variable for the Iraq invasion in March 2003, average hourly wages, total payroll employment, and the logarithm of inflation-adjusted gas prices, we find that the effect of gas prices is negative but not quite statistically significant at conventional levels. 9/11 and the Iraq invasion, by contrast, have highly significant effects. (A more complex model that takes account of the long, slow decline in the 9/11 boost would likely wipe out the gas effect completely.)
Now it’s certainly plausible that gasoline prices have some effect on approval, but not to the extent that Harwood or Page suggest. Given the weak relationships that existed during past administrations (which Harwood acknowledges), the relationship that we observe during Bush’s presidency seems likely to be a statistical artifact. It is very premature to call gas prices the “strongest factor” affecting approval. (For those who are interested, The Macro Polity is to my mind the definitive political science work on the factors influencing presidential approval.)
Update 9/22 10:47 AM: Using sophisticated techniques and better data than my quick-and-dirty model above, the distinguished political scientists Nathaniel Beck, Simon Jackman, and Howard Rosenthal report results for the determinants of Bush approval that mirror my analysis above (PDF):
Immediately note that the approval series is dominated by the long, almost uninterrupted decline after the peak in the 9/11 aftermath. Hence, any variable that trends in a similar way will emerge as a good predictor of approval, at least in these data. Figures 9 and 10 plot the
relationship between weekly gasoline prices and approval and cumulative U.S. deaths in Iraq,
and presidential approval, respectively. The latter variable trends up, by construction, and
gas prices also generally trend up over the post 9/11 phase of Bush’s presidency. We entered
these variables, plus the log of first time unemployment claims in an augmented version of our
transition model… The coefficient on the
covariates are something of a mixed bag, with the estimated effects on changes in gas prices
not unambiguously signed at conventional levels of statistical significance (i.e., the posterior
probability that changes in gas prices drive approval down is just .77). Cumulative U.S. deaths
in Iraq appears to drive approval down (again, if for no other reason than both variables trend
in the same direction over much of the time series). Similarly, first time unemployment claims
generally trend down over the course of the Bush presidency, around seasonal variation, and
so picks up an unambiguously positive coefficient.