BY DAN MARIANO, Manila Times, Monday, 22 February 2010
When a candidate complains about survey results, people usually conclude that he must be a cellar-dweller in the opinion polls. And former Sen. Francisco “Kit” Tatad could be regarded as one such disgruntled aspirant.
A survey done by Social Weather Station (SWS) earlier this month, for instance, placed Tatad at a “significant distance” from the leading pack of senatorial candidates. The casual observer, thus, is tempted to dismiss his complaints against surveys in general as just another case of “sour grapes.”
Tatad, however, is not your typical Aesopian fox badmouthing the fruit that are beyond his reach. What he presents is an obviously well-researched critique that should give the public—especially the news media—pause.
Surveys do tend to influence, if not how people vote, then certainly how they size up candidates.
Aspirants who do well at the start of the race tend to get more attention—and conceivably more financial support from quarters that see campaign contributions as wagers on future political accommodation and favors.
As experience tells us, it does not exactly enhance the democracy that we have been trying to erect for the past several decades. Instead of making informed choices and wise decisions, the bulk of voters come under the deceptive spell of popularity.
During a media forum last week, Tatad described local surveys as “fatally flawed.” He said the outfits that conduct surveys in the Philippines use misleading methodologies, which have long been discredited in more advanced democracies.
“Local pollsters have used methodologies and techniques that are flawed and discredited, and have long been discarded in the US, where public opinion polling was invented and turned into a billion-dollar industry,” Tatad said at the Fernandina media forum last week.
These “flawed” methodologies include: face-to-face interviewing, quota and cluster sampling, loaded and lengthy questionnaires, trial heat polls and pick 3 polling, which are all stated in the methodologies disclosed by the pollsters.
Face-to-face interviewing is the standard method used by pollsters—such as SWS and Pulse Asia—for eliciting responses from survey participants. Respondents are tracked door-to-door and interviewed by the pollsters’ field personnel. They are asked to respond to the preset questionnaire and shown pictures of candidates.
In the past, personal or face-to-face interviewing was viewed as an appropriate method for conducting opinion surveys because it ostensibly allowed the pollster to select the “right” respondent to be interviewed.
After major flops, however—notably, the erroneous forecast of victory for New York Gov. Thomas E. Dewey over incumbent President Harry S. Truman in the 1948 US presidential race—this survey method was discarded, Tatad recalled.
Reputable pollsters in the United States have now totally abandoned face-to-face interviewing, he added.
Tatad cited experts Chava Frankfort-Nachnias and David Nachmias who in Research Methods in the Social Sciences wrote: “The very flexibility that is the interviewer’s chief advantage leaves room for the interviewer’s personal influence and bias.”
Tatad also quoted pollster Kenneth Warren who in his book, In Defense of Public Opinion Polling, wrote: “The cons of door-to-door interviews far outweigh the pros . . . Because of the sensitivity or personal nature of some questions, interviewers, because they were placed in face-to-face situations, have admitted that they sometimes guessed or fudged responses . . . These problems are a major source of bias in personal interviews, causing significant contamination of the poll data.”
These methodological and practical problems according to Warren have doomed face-to-face interviews.
By 1980 in the United States, nobody wanted to pay for this type of surveys which were “fatally flawed and grossly inaccurate anyway.”
But as Tatad noted, this “seems to have had no persuasive effect on our local pollsters.”
Another glaring weakness in local surveys, according to Tatad, is the extensive and general use of quota sampling to create a supposedly “representative sample” of the Philippine population.
In quota sampling, survey respondents are picked from different types of people—such as by age, sex, religion, income—and various predetermined areas, like region of the country, as well as urban or rural.
“This method is the most familiar form of non-probability sampling. It is supposed to mirror the same proportions in the targeted survey populations, but doesn’t,” Tatad said.
Quota sampling, he added, proved to be an earthshaking failure in 1948 after three leading US pollsters—Gallup, Roper and Crossley—erroneously called the US presidential election in favor of Dewey instead of Truman.
In the United Kingdom, where quota sampling persisted, it was blamed for the pollsters’ failure to predict Prime Minister John Majors’ victory in 1992.
Nowadays, reputable US pollsters rely almost exclusively on probability random sampling to create a “representative sample,” Tatad said.
“Why then do local pollsters continue to use quota sampling and face-to-face interviewing for their surveys?” Tatad asked. “Why haven’t they adopted probability random sampling, which has protected US opinion polls from using contaminated data?”
Tatad also said that local pollsters have ignored standards for professional and ethical practice of public opinion polling that elsewhere are regarded as sacred by polling associations and reputable pollsters.
“Professional standards are virtually nonexistent in the local opinion polling industry,” the comebacking senator said.
“No law regulating the conduct of opinion polling, and no professional association of pollsters either to set and enforce standards of conduct and standards of disclosure and ensure the reliability and validity of survey results,” he added.
Tatad further said that the news media have been an unsuspecting purveyor of dubious findings to the detriment of the election campaign and the public.
“The public would have had a better appreciation and understanding of public opinion polling had the media been a little more critical and vigilant,” he said.
Needless to say, Tatad’s critique requires a response of similar detail and authority from SWS, Pulse Asia and other local pollsters. Dismissing it as mere “sour grapes” will no longer suffice.
dansoy26@yahoo.com
SENATOR KIT TATAD 