PERILS OF PUSH POLLING FOREIGN POLICY

John Zogby, the well known political pollster is in the news today due to a poll he ran for a left-wing peace group that was published in the Christian Science Monitor that purports to show a remarkable shift in sentiment by American troops in Iraq to favor immediate withdrawal and a deep division between regular troops and reservists. Serious questions are being raised on both the Left and Right about the methodology used by Zogby as well as the phrasing of the questions themselves.

For starters, a proper representative sampling appears not to have been used by Zogby in this poll; Bruce Kesler at the Democracy Project writes:

"Today, Mystery Pollster -- who is liberal, a reputable and experienced evaluator of polls, and who had access to some of the “secret” methodology – reveals his judgments [ ed. Note " Mystery Pollster"communicated with John Zogby directly]. They parallel mine.

First, to be clear, the Center for Peace and Global Studies is in effect a "partisan" sponsor in that, according to Zogby, they oppose the war in Iraq.

Second, while Zogby says his interviewers selected respondents randomly at various locations, he makes no claim of random selection with respect to the locations involved. I apologize for being so vague, but the most I can say is that the method Zogby used to gain access to those locations constrained his ability to make random selections.

Third, even if consumers of this data knew all that I know about how Zogby's interviewers "walked up to troops" (as commenter Karen puts it), they would still have questions about the impact of such an interaction might have on the kinds of troops most likely to agree to participate in the survey. Consider the exit poll example again. Even though exit pollsters have disclosed the procedures they use to train interviewers and select respondents, we still debate the effect of those procedures on the kinds of voters that choose to participate. Disclosure in that case cannot resolve all questions, but it at least enables an informed debate. Unfortunately, such discussion and debate is impossible in this case. "

Aside from the problem of of the Zogby poll not being properly representative in general there may be a particular selection bias at work here. As a military affairs writer for The Chicago Tribune, Colonel E. W. Chamberlain III has noted, the troops that civilian journalists have the easiest access to in a war zone are also those most likely to have the lowest morale and loudest gripes. Garrison or routine duty is inherently demotivating compared to more exciting missions in the field that far fewer journalists ever see outside of the Robert Kaplan types.

A definitive answer on the accuracy of the Zogby poll is not possible unless John Zogby chooses to make such information regarding his exact methodology available. If it is accurate, that poll would be a critical data point for field commanders, policy makers and the Congress to know. It would indicate very serious problems in terms of the troops morale and mission objectives and a need for the administration to take action.

If it is not accurate then it is either bad data honestly arrived at or an attempt to drive public debate and thus foreign policy under false pretenses to a position favored by the Zogby client and perhaps Mr. Zogby himself, given his personal interest in America adopting a more evenhanded or pro-Arab policy for the Middle-East. Either way the poll would not be adding illumination to the events in Iraq but throwing up dust and distorting the OODA loop.

John Zogby is an influential and important pollster but credibility once lost, is lost. He owes it to both himself and the American troops in Iraq to offer the public some clarification.  ¶ 12:08 PM