There’s something wrong about our surveys!

March 2, 2010
Written by J.A. de la Cruz / Coast-to-Coast
Monday, 01 March 2010 21:39
For the nth time, the local political survey firms Social Weather Stations (SWS) and Pulse Asia are under withering scrutiny. Not just by candidates and their political advisers, but by a growing number of observers, including members of academe, who are increasingly concerned about the methods, practices and, yes, the results of these surveys as purveyed by the firms and their adherents.

These sectors are coming around to the view that our local pollsters are doing a great disservice to the public and to the social-science profession by using flawed, even long-discarded, methods in their determination of public opinion, and then issuing the polling results in a skewed, helter-skelter way, without as much as offering the obligatory caveats about their work. They are also being taken to task for their adroit (some suggest deceptive) “marketing” operations as they actively promote their “studies” and seek “sponsors” (subscribers is how these firms call them) to cover the costs of their operations. The critics insist that these firms have taken a larger-than-life role in public life, promoting candidates and advocacies with hardly any accountability at all. They contend that, instead of becoming enhancers, they have become degraders of our democratic aspirations. Having taken roots in our democratic discourse and playing such a key role in the shaping of public opinion, it is time these firms’ own operations are scrutinized and subjected to the rigors of real, factual and scientific research with hardly any room for intervention of any kind from any source. These concerns have become even more telling in the run-up to the May elections as the country’s future gets so closely interlinked to the “polled” fortunes of the candidates. Instead of getting scrutinized for their views, their past performance and their character and standing in private and public life, the public is fed with, at best, less-than- exemplary polling results. That these firms have had their own share of “boo-boos” in the past makes such a scrutiny even more necessary and urgent at this time.

Tatad’s view

Comebacking Sen. Kit Tatad, who has been a victim of flawed survey results in the past, has actively sought greater transparency and accountability on the part of SWS and Pulse Asia. In our regular Kapihan sa Sulô forum last Saturday, Tatad asked that these firms refrain from surveying and purveying the results in the meantime until they can clear themselves, as it were, from past mistakes and indiscretions. Said Tatad: “SWS should first explain its fatally flawed exit poll of the 2004 elections in Metro Manila before it conducts yet another opinion poll related to the May 10 elections. For its part, Pulse Asia should disclose to the public how many candidates have paid how much in order to participate in and benefit from its surveys. I believe this is the irreducible minimum ethical and professional requirement before the two firms resume their unrestrained effort to shape public perceptions on the next presidential elections. Our people have a right to make this demand in light of the far-from-exemplary record of the two firms and the unaccountable power they now seem to possess.

“On May 11, 2004, within hours of the close of balloting, SWS announced that the incumbent President Gloria-Macapagal Arroyo got 31 percent of the votes in Metro Manila as against opposition candidate Fernando Poe Jr., who reportedly got 23 percent. The exit poll, commissioned by ABS-CBN, was conducted in the homes of 528 voters in the National Capital Region [NCR]. However, when the official Commission on Elections count came, Mr. Poe took the NCR with 1,452,380 votes or 36.67 percent of the votes, while Mrs. Arroyo got 1,049,016 votes or 26.46 percent of the votes. Mr. Poe won in all Metro Manila cities and towns except Las Piñas, where he lost by a mere 1,876 votes.

“This gross misreading of the results of the 2004 presidential elections in Metro Manila was far more devastating than the costly error of the Literary Digest in predicting President Franklin Delano Roosevelt’s defeat in the hands of Alf Landon in 1932, and George Gallup’s, Archibald Crossley’s and Elmo Roper’s common error in predicting President Harry Truman’s defeat in the hands of Thomas Dewey in 1948. Why so? Because while the American pollsters had erred in their respective pre-election surveys, the best of which could never be completely free from any mistake, SWS had messed up in an exit poll, where no professional pollster should.

“Similarly, I would ask Pulse Asia to make a full disclosure of the services it has sold to politicians who are eager to rate in its surveys. Contrary to what appears to be sound ethical practice, the firm has been inviting politicians to participate in its surveys at the rate of P400,000 per head, and to introduce ‘rider’ questions about their candidacies at P100,000 each. The politicians’ names have never been published, and neither have the ‘rider’ questions,” Tatad said. (Note: Actually, both firms and others conducting political surveys should make this disclosure).

Pulse Asia’s caveats

Indeed, these firms, being the leaders in the field, have a duty to make as much of their operations (and connections) open to the public. To his firm’s credit, Pulse Asia president Prof. Ronnie Holmes gamely answered questions about their polling methodology, their practices and, yes, their “subscribers” and other “clients.” Holmes noted that their methods and practices have adhered closely to the requirements of polling, and they are an active member of the Philippine Social Services Council, which ostensibly monitors their members’ operations as the umbrella organization of this largely self-regulatory body. He also noted that their records are open to public scrutiny and they will be open to discussions with the media and other sectors as far as their undertakings, including “marketing” activities, are concerned.

He also agreed with our other guest, veteran journalist Yen Makabenta, that it may, indeed, be necessary to change the survey question in the most sought-after issue at this point—who would one vote for president if the elections were held today—as such effectively suppresses the actual percentage of undecided. Quoting pollster David Moore, Makabenta noted that such “vote choice” (a forced choice) question glosses over voter indecision, which is likely in an election campaign as a good number of voters actually make their choice right at the precinct level or just days or hours before going to the polls. “The worst sin in poll reporting,” Moore noted, “was hedging”—which is what happens with a “forced choice” question. He also noted that in the US, the undecided can range from a low of 20 percent to as high as 70 percent—depending on how far away the election is.

Curiously, with three months to go before the May elections, both SWS and Pulse Asia are reporting very low “undecided,” i.e., from 2 percent to 4 percent only—almost negligible by polling standards. Yet, these results, which gloss over the huge “undecided,” are reported as if cast in stone, bringing the candidates and their advisers to moments of ecstasy or exasperation, depending on which side one is on. To avoid this skewed, if not totally discardable, question, Moore suggested a new question which, to my mind, better captures the opinion or sentiment of a respondent. Translated into the coming polls, it should read: “In the May election, who would you vote for president, or haven’t you yet made up your mind?” And to those who made a decision, a rider to be asked should be: “Is that a firm choice, or could you change your mind before Election Day?”

Moore’s point

In his book, The Opinion Makers, Moore makes the startling conclusion that pollsters “do not measure public opinion, they manufacture it.” He anchors this contention on the practice of polling firms to gloss over “voter indecision” during an election campaign. Moore notes:

“There is crisis in public-opinion polling today, a silent crisis that no one wants to talk about. The problem lies not in the declining response rates and increasing difficulty in obtaining representative sample, though these are issues the polling industry has to address. The problem lies, rather, in the refusal of media polls to tell the truth about those surveyed and about the larger electorate. Rather than tell us the essential facts about the public, they feed us a fairy-tale picture of a completely rational, all-knowing and fully engaged citizenry. They studiously avoid reporting on widespread public apathy, indecision and ignorance. The net result is conflicting poll results and a distortion of public opinion that challenges the credibility of the whole polling enterprise. Nowhere is this more often the case than in election polling.”

So there. To those who have taken on the polling firms as oracles, and their surveys unvarnished truth on the public’s opinion of the various candidates, we can only say: caveat emptor. And let us move on to make those surveys truly reflective of the public pulse, not a skewed or, worse, manufactured one.