There’s something wrong about our surveys!

March 2, 2010
BusinessMirror
Written by J.A. de la Cruz / Coast-to-Coast
Monday, 01 March 2010 21:39
For the nth time, the local political survey firms Social Weather Stations (SWS) and Pulse Asia are under withering scrutiny. Not just by candidates and their political advisers, but by a growing number of observers, including members of academe, who are increasingly concerned about the methods, practices and, yes, the results of these surveys as purveyed by the firms and their adherents.

These sectors are coming around to the view that our local pollsters are doing a great disservice to the public and to the social-science profession by using flawed, even long-discarded, methods in their determination of public opinion, and then issuing the polling results in a skewed, helter-skelter way, without as much as offering the obligatory caveats about their work. They are also being taken to task for their adroit (some suggest deceptive) “marketing” operations as they actively promote their “studies” and seek “sponsors” (subscribers is how these firms call them) to cover the costs of their operations. The critics insist that these firms have taken a larger-than-life role in public life, promoting candidates and advocacies with hardly any accountability at all. They contend that, instead of becoming enhancers, they have become degraders of our democratic aspirations. Having taken roots in our democratic discourse and playing such a key role in the shaping of public opinion, it is time these firms’ own operations are scrutinized and subjected to the rigors of real, factual and scientific research with hardly any room for intervention of any kind from any source. These concerns have become even more telling in the run-up to the May elections as the country’s future gets so closely interlinked to the “polled” fortunes of the candidates. Instead of getting scrutinized for their views, their past performance and their character and standing in private and public life, the public is fed with, at best, less-than- exemplary polling results. That these firms have had their own share of “boo-boos” in the past makes such a scrutiny even more necessary and urgent at this time.

Tatad’s view

Comebacking Sen. Kit Tatad, who has been a victim of flawed survey results in the past, has actively sought greater transparency and accountability on the part of SWS and Pulse Asia. In our regular Kapihan sa Sulô forum last Saturday, Tatad asked that these firms refrain from surveying and purveying the results in the meantime until they can clear themselves, as it were, from past mistakes and indiscretions. Said Tatad: “SWS should first explain its fatally flawed exit poll of the 2004 elections in Metro Manila before it conducts yet another opinion poll related to the May 10 elections. For its part, Pulse Asia should disclose to the public how many candidates have paid how much in order to participate in and benefit from its surveys. I believe this is the irreducible minimum ethical and professional requirement before the two firms resume their unrestrained effort to shape public perceptions on the next presidential elections. Our people have a right to make this demand in light of the far-from-exemplary record of the two firms and the unaccountable power they now seem to possess.

“On May 11, 2004, within hours of the close of balloting, SWS announced that the incumbent President Gloria-Macapagal Arroyo got 31 percent of the votes in Metro Manila as against opposition candidate Fernando Poe Jr., who reportedly got 23 percent. The exit poll, commissioned by ABS-CBN, was conducted in the homes of 528 voters in the National Capital Region [NCR]. However, when the official Commission on Elections count came, Mr. Poe took the NCR with 1,452,380 votes or 36.67 percent of the votes, while Mrs. Arroyo got 1,049,016 votes or 26.46 percent of the votes. Mr. Poe won in all Metro Manila cities and towns except Las Piñas, where he lost by a mere 1,876 votes.

“This gross misreading of the results of the 2004 presidential elections in Metro Manila was far more devastating than the costly error of the Literary Digest in predicting President Franklin Delano Roosevelt’s defeat in the hands of Alf Landon in 1932, and George Gallup’s, Archibald Crossley’s and Elmo Roper’s common error in predicting President Harry Truman’s defeat in the hands of Thomas Dewey in 1948. Why so? Because while the American pollsters had erred in their respective pre-election surveys, the best of which could never be completely free from any mistake, SWS had messed up in an exit poll, where no professional pollster should.

“Similarly, I would ask Pulse Asia to make a full disclosure of the services it has sold to politicians who are eager to rate in its surveys. Contrary to what appears to be sound ethical practice, the firm has been inviting politicians to participate in its surveys at the rate of P400,000 per head, and to introduce ‘rider’ questions about their candidacies at P100,000 each. The politicians’ names have never been published, and neither have the ‘rider’ questions,” Tatad said. (Note: Actually, both firms and others conducting political surveys should make this disclosure).

Pulse Asia’s caveats

Indeed, these firms, being the leaders in the field, have a duty to make as much of their operations (and connections) open to the public. To his firm’s credit, Pulse Asia president Prof. Ronnie Holmes gamely answered questions about their polling methodology, their practices and, yes, their “subscribers” and other “clients.” Holmes noted that their methods and practices have adhered closely to the requirements of polling, and they are an active member of the Philippine Social Services Council, which ostensibly monitors their members’ operations as the umbrella organization of this largely self-regulatory body. He also noted that their records are open to public scrutiny and they will be open to discussions with the media and other sectors as far as their undertakings, including “marketing” activities, are concerned.

He also agreed with our other guest, veteran journalist Yen Makabenta, that it may, indeed, be necessary to change the survey question in the most sought-after issue at this point—who would one vote for president if the elections were held today—as such effectively suppresses the actual percentage of undecided. Quoting pollster David Moore, Makabenta noted that such “vote choice” (a forced choice) question glosses over voter indecision, which is likely in an election campaign as a good number of voters actually make their choice right at the precinct level or just days or hours before going to the polls. “The worst sin in poll reporting,” Moore noted, “was hedging”—which is what happens with a “forced choice” question. He also noted that in the US, the undecided can range from a low of 20 percent to as high as 70 percent—depending on how far away the election is.

Curiously, with three months to go before the May elections, both SWS and Pulse Asia are reporting very low “undecided,” i.e., from 2 percent to 4 percent only—almost negligible by polling standards. Yet, these results, which gloss over the huge “undecided,” are reported as if cast in stone, bringing the candidates and their advisers to moments of ecstasy or exasperation, depending on which side one is on. To avoid this skewed, if not totally discardable, question, Moore suggested a new question which, to my mind, better captures the opinion or sentiment of a respondent. Translated into the coming polls, it should read: “In the May election, who would you vote for president, or haven’t you yet made up your mind?” And to those who made a decision, a rider to be asked should be: “Is that a firm choice, or could you change your mind before Election Day?”

Moore’s point

In his book, The Opinion Makers, Moore makes the startling conclusion that pollsters “do not measure public opinion, they manufacture it.” He anchors this contention on the practice of polling firms to gloss over “voter indecision” during an election campaign. Moore notes:

“There is crisis in public-opinion polling today, a silent crisis that no one wants to talk about. The problem lies not in the declining response rates and increasing difficulty in obtaining representative sample, though these are issues the polling industry has to address. The problem lies, rather, in the refusal of media polls to tell the truth about those surveyed and about the larger electorate. Rather than tell us the essential facts about the public, they feed us a fairy-tale picture of a completely rational, all-knowing and fully engaged citizenry. They studiously avoid reporting on widespread public apathy, indecision and ignorance. The net result is conflicting poll results and a distortion of public opinion that challenges the credibility of the whole polling enterprise. Nowhere is this more often the case than in election polling.”

So there. To those who have taken on the polling firms as oracles, and their surveys unvarnished truth on the public’s opinion of the various candidates, we can only say: caveat emptor. And let us move on to make those surveys truly reflective of the public pulse, not a skewed or, worse, manufactured one.

Advertisements

Tatad tells SWS, Pulse Asia to explain ‘flawed’ surveys

March 1, 2010

Written by Jonathan Mayuga / Correspondent
Sunday, 28 February 2010 19:43

THE Social Weather Stations (SWS) and Pulse Asia were asked to explain on Saturday their polling “errors and malpractices” before conducting yet another survey related to the May 10 elections.

Former senator Francisco “Kit” Tatad said SWS should explain its “fatally flawed” exit poll of the 2004 elections in Metro Manila, while Pulse Asia should make full public disclosure on the number of candidates that commissioned its services.

Speaking before the weekly forum “Kapihan sa Sulô Hotel,” Tatad said such is the “irreducible minimum ethical and professional requirement” before the two companies should resume their operations, which shape public perceptions on the next presidential election.

Tatad, who claimed to have never commissioned any survey before, said polling firms should be guided by ethical standards or norms because of the nature of the role they play in shaping the mind of the people.

“Our people have a right to make this demand in light of the far-from-exemplary record of the two firms and the unaccountable power they now seem to possess,” Tatad said.

He recalled that on May 11, 2004, within hours of the close of balloting, SWS announced that the incumbent President Arroyo got 31 percent of the votes in Metro Manila as against the opposition candidate Fernando Poe Jr., who reportedly got 23 percent. The exit poll, commissioned by network giant ABS-CBN, was conducted in the homes of 528 votes in Metro Manila.

However, when the official Commission on Elections count came, it was Poe who won in Metro Manila, garnering a total of 1,452,380 votes or 36.67 percent against Arroyo, who got 1,049,016 votes or 26.46 percent of the votes.

Poe, he said, won in all Metro Manila cities and towns, except in Las Piñas, where he lost by a mere 1,867 votes.

Tatad said the gross misreading of the results of the 2004 presidential elections in Metro Manila was far more devastating than the costly error of the Literary Digest in predicting President Franklin Delano Roosevelt’s defeat against Alf Landon in 1932, and Geore Gallup’s Archibald Crossley’s and Elmo Roper’s common error in predicting President Harry Truman’s defeat against Thomas Dewey in 1948.

SWS, according to Tatad, messed up badly in the 2004 exit poll, when no professional pollster should.

Pulse Asia, meanwhile, should disclose to the public of the services it had “sold” to politicians who are eager to rate in its surveys.

He said the firm has been inviting politicians to commission the company by participating in its surveys at the rate of P400,000 per head, and to introduce “rider” questions about their candidates at P100,000 each. This, he said, is contrary to sound ethical practice.

He added that the politicians’ names have never been published, and neither have the “rider” questions.

“Since the questionnaire forms the soul of any survey, it should be neutral and not biased for or against anyone. But the fact that paying politicians are allowed to contribute their own questions is not the best way of ensuring the neutrality and objectivity of the questions. And it does not prevent anyone from asking, what else is being sold aside from the questions?” he said.

Tatad was referring to the expanded segment of its “Ulat ng Bayan” series, which the firm claims to be “designed to provide an accurate reading of public views and sentiments on the most current national issues and events.”

Speaking during the same forum, Pulse Asia president and managing fellow Ronald Holmes said the company is, in fact, guided by ethical norms like all other poll survey firms.

In fact, he said Pulse Asia often refused clients who wish to “push-poll,” wherein question tend to be leading, which he said is more of a campaign strategy than a survey.

“We adhere to the highest standard and norms in the practice. Our questions are not leading or biased,” he said.

According to Holmes, Pulse Asia’s practice is legal as well as ethical, like any other “market research” commissioned by any company, wherein they are bound to follow or adhere to the “confidentiality clause” of a contract.