Brendan Nyhan

What is John Zogby talking about?

Zogby International, the polling firm of John Zogby, announced its latest poll yesterday with a press release headlined “No Bounce.” The release states that “President Bush’s televised address to the nation produced no noticeable bounce in his approval numbers, with his job approval rating slipping a point from a week ago, to 43%, in the latest Zogby International poll.”

But if you read the details of the poll, you will find out that it was conducted from June 27-29. Bush gave his speech during the evening of June 28. So how could Zogby tell if Bush got a bounce when a substantial portion of his respondents were interviewed before the speech? This seems like another example of Zogby’s questionable professional ethics to me.

Yet despite this obvious flaw, the poll was touted by Bloomberg as showing that Bush did not receive a bounce:

Bush’s Iraq Speech Fails to Rally Support, Poll Finds

U.S. President George W. Bush’s public
approval rating remained in negative territory after a nationally
televised address on Iraq, a new poll has found.

The poll by Utica, New York-based Zogby International
reported that 43 percent of those surveyed approved of Bush’s job
performance and 56 percent disapproved. The poll of 905 people was
taken June 27-29 and had a margin of error of 3.3 percentage
points.

What’s especially strange is that reporter Ken Fireman notes later in the article that Bush’s speech took place on June 28, but doesn’t see fit to mention the contradiction between that date and the period in which the poll was conducted. (Atrios also picked up the release and quoted it without comment.)

As I wrote before, Bush is ultimately unlikely to get any significant boost from the speech. But to demonstrate that, we need a poll conducted after the speech. This is not complicated. So why can’t Zogby and Bloomberg get it right?

Clarification 7/2: As a commenter correctly points out, I somehow missed this statement in the press release:

The Zogby America survey includes calls made both before and after the President’s address, and the results show no discernible “bump” in his job approval, with voter approval of his job performance at 45% in the final day of polling.

Apologies for the error. But it doesn’t change my conclusion. The press release initially touts the overall finding of an approval level of 43% as evidence of a lack of a bounce. That’s just wrong. And the fact that the approval level in the third day is close to his previous one isn’t proof of anything. Assuming a proportional number of calls across the three days, Zogby made about 300 calls in the third day. As such, his margin of error for polling on that day is much larger than the +/- 3.3% margin of error for the survey as a whole, which means that he can’t conclude much of anything from the 45% figure.

Update 7/2: A Gallup poll conducted after Bush’s speech (6/29-6/30) shows Bush’s job approval at 46%, which is not statistically distinct from the previous level of 45% (6/24-6/26). It also shows that responses to questions about Iraq appear to be relatively stable, though pro-war sentiment bumped up 4% on two questions, which is right at the limit of the poll’s margin of error.