We cannot complete this presentation without discussing briefly the unwitting part the media have played to allow opinion poll results to dominate public perceptions of the campaign. This would not have been possible if dubious opinion polls had not been reported energetically in the media without an iota of analysis. The public would have had a better appreciation and understanding of public opinion polling had the media been a little more critical and vigilant.
In the US, the media normally ask the following 20 questions before publishing the results of any opinion poll:
- Who did the poll?
- Who paid for the poll and why was it done?
- How many people were interviewed for the survey?
- How were those people chosen?
- What are (nation, state or region) or what group (teachers, lawyers, Democratic voters, etc.) were those people chosen from?
- Are the results based on the answers of all the people interviewed?
- Who should have been interviewed and was not? Or do response rates matter?
- When was the poll done?
- How were the interviews conducted?
- What about polls on the Internet or World Wide Web?
- What is the sampling error for the poll results?
- Who’s on first?
- What other kinds of factors can skew poll results?
- What questions were asked?
- In what order were the questions asked?
- What about “push polls”?
- What other polls have been done on this topic? Did they say the same thing? If they are different, why are they different?
- What about exit polls?
- What else needs to be included in the report of the poll?
- So I’ve asked the questions. The answers sound good. Should we report the results?
The reason for asking these questions is plain enough: there are good polls and bad polls. It appears that every poll should be judged guilty until proven otherwise. Of course, some polls are more reliable that others. But the following key ingredients are needed:
- Adherence to Professional Standards and Ethics
- A Well-developed, Intelligent, Yet Doable Research Design
- A Carefully Drawn and Used Representative Sample
- A Well-designed Questionnaire
- Well-trained and Professional Interviewers
- Careful Coding and Tabulation of Raw Poll Data
- Thorough and Insightful Analysis