Polling statistics: Sorting lies from damn lies

American novelist Cormac McCarthy once wrote of a group of Mexican cowhands that they weighed each piece of new information carefully, and disdained any knowledge not gained at first hand. McCarthy felt this trait was common among people who had become experts in a narrow field of endeavour.

American novelist Cormac McCarthy once wrote of a group of Mexican cowhands that they weighed each piece of new information carefully, and disdained any knowledge not gained at first hand. McCarthy felt this trait was common among people who had become experts in a narrow field of endeavour.

Twenty-first century Slovakia is a long way removed from McCarthy's 1930s Mexico, not least in that many of us profess knowledge of affairs we dimly understand. It's not that we wouldn't prefer to be sceptical cowhands still, but that we feel we can't afford to be. The field of human endeavour has widened, and none of us can keep up any more. The yearning for comprehension, however, burns as bright as ever.

That's a human need which is well understood by polling agencies, who feed us regular statistics on political preferences and social trends that give busy, confused people the feeling they still understand the country they live in.

Such surveys should not serve as lodestars for the questing intellect. We know too little about how these agencies get their results. We have too many doubts about the motives of people who pay for polls to be carried out. We fear that they are often used to shape public opinion, rather than to inform. We suspect some people conducting surveys know even less about their business than we know about ours.

Still, polling agencies don't lack customers: foreign aid workers goggling over 'data' that 60% of Slovaks support the segregation of the Roma minority; Internet firms either lamenting or exulting over the latest 'penetration rates'; political amateurs of every stripe troubling the world with predictions for elections that won't be held for another 16 months.

It seems that people are going to read polls no matter what, if only to have some snippets of information to contribute to casual conversations, or to give their biases and prejudices an ostensibly factual basis. This being the case, polling agencies, and the newspapers which publish the results of their work, should make sure they follow clear rules. The public should also be told what to look for in assessing the trustworthiness of polls. Here's a brief list of the information every poll should include:

1. The size of the group of respondents. No matter where in the world a poll is done, a group of at least 1,000 people is required if any survey is to claim to measure opinions that represent those of the wider population.

2. The degree or margin of possible error.

3. The date the poll was carried out.

4. The exact text of the question asked.

5. Whether the poll was conducted face-to-face or by telephone (a far less reliable method).

6. Whether respondents were selected at random, or by a quota system. Random surveys can sometimes yield higher-quality results, but they are more expensive, and thus less common in Slovakia. Quota surveys ensure that the respondents polled reflect the country's basic demographic trends, including sex, level of education, minority group, what size of town or village the person inhabits etc. It's not unusual for a survey compiler stationed in a particular town to be given quota guidelines which require him to poll three women and three men, two of whom have to be Hungarian, four of whom must have less than university education, and three of whom should be between 30 and 45.

7. Although this is not current practice, some social scientists feel the public should be told who paid for the poll to be done, and what kind of polls the agency in question specialises in.

The fact that this information is rarely provided in its entirety is mostly the fault of the media, who cut much of it because they don't think it's important. On other occasions, it's hidden by the agency itself, perhaps to conceal inferior methods.

But as a recent stir over a political survey conducted by the GfK agency shows, public trust in all polling groups can be damaged even by one suspect poll conducted according to unknown rules. It makes people wonder whether statistics are a waste of time, and whether they wouldn't be better off ignoring events and trends that don't directly affect their lives. It makes us aware of what we all sometimes resemble - guys in betting shops with clapped-out shoes, trying to predict the future and understand the world with the aid of a tabloid newspaper.

Which is why it's so important that polling agencies follow standard procedure and tell us the limits of their information. It's the only way we can begin to claim we understand things we don't have first-hand knowledge of. It's the only way we can regain a shadow of the self-sustaining certainty those Mexican cowhands knew.

Top stories

Stock image.

Twice as many Ukrainians work in Slovakia now than before the Russian invasion.


Píšem or pišám?

"Do ľava," (to the left) I yelled, "Nie, do prava" (no, to the right), I gasped. "Dolšie," I screamed. "Nie, nie, horšie..." My Slovak girlfriend collapsed in laughter. Was it something I said?


Matthew J. Reynolds
Czech biochemist Jan Konvalinka.

Jan Konvalinka was expecting a pandemic before Covid-19 came along.


SkryťClose ad