The value of anecdotal evidence

Robert Henderson

There is a general sneering at statements based on anecdotal evidence. This is wrong because although there are not sociological laws in the sense of  those in physics or chemistry, there are indubitably sociological phenomena which show that the behaviour  of humans is governed by more than their individual biology and experience. Opinion polls work on this assumption. Where the question asked is unambiguous and at least reasonably  uncontentious, the poll  of a thousand or so people is, when placed in the context of the superficially  atomistic nature  of human behaviour,  remarkably close to what the nation thinks. Polls of voting intention in general elections are generally accurate in terms of  the overall percentage vote  for each party if not in the constituency results.

Perhaps the neatest example of such a law in action is the voting at general elections. The voting patterns in a general election are generally uniform. If the swing from one party to another is 5% in the vast majority of constituencies it will be  close to that. Exceptionally some constituencies will return a different figure but invariably this
can be traced to factors such as a good or bad constituency MP, whether the MP is a minister and so on.

Of course, polls and market research are based on supposedly scientifically selected samples which remove bias and produce an answer which either applies to the population in general or whatever group is being polled.

When we collect anecdotal evidence we  automatically select from those within our social group which generally means people like us in terms of class  and education with age and sex also strong influences . That is no different in principle to  the pollster or market researcher polling , say, the members of a political party or middleclass women on childcare.  All we need to know about any anecdotal evidence is the likely group it has been culled from and then put it into context.

Some evidence arguably  does not even have to be put into social context. That is evidence which  consists of factual reports of actual behaviour. Take the case of  a riot. Those who witness it will by and large tell the same general story regardless of social status.

In one respect anecdotal evidence may be much closer to the truth than that gleaned by pollster. The more contentious a subject the less likely a person is to tell the truth to pollsters. They are much more likely to tell it in private conversation with friends, although there is peer pressure to conform to a particular view mitigate this advantage.

As to the objection that  anecdotal evidence will be  biased, of course it will be. The real question is  why should we believe it is generally more biased than that collected by pollsters?  Pollsters manipulate results by their questions and contentious questions often render poll results highly dubious for the reason given above. Moreover, we know that although a sample of  1,000  is generally reckoned to be the  size of sample beyond which little greater accuracy will be achieved, it is also true that much smaller samples  provide answers which are still pretty accurate. The average person assessing his or her view of an important matter will probably have taken in several dozens of  individual views through face to face social contact, the internet and the media before arriving at a judgement.  It is also true that the individual will bring all the normal human abilities to assess the views of others before  judgement is made, something polling does not do. That may actually be a more accurate way of assessing the general sociological mood of a population than scientific polling.  Finally, the sociological phenomenon of general change in population such as voting
behaviour will of itself  ensure a high degree of truth in the reporting of opinions because opinions will widely change through whatever sociological law governs such things.

To those still prone to sneer let them reflect on this: human beings actually run  their day to day  lives simply by basing their behaviour on the empirical evidence of what  others do and say, that is, anecdotal evidence.

Can  anecdotal evidence be quantified or evaluated objectively? Problematic to say the least, but perhaps the Rev Thomas Bayes (1703-61) can come to our rescue. A dictionary of philosophy (Pan)  states that Bayes developed a theorem “giving an expression for the probability of an hypothesis, h, if some evidence, e, is added to antecedent knowledge, a. The theorem states that the probability of  h relative to e and a is equal to the probability of  h relative to a multiplied  by the probability of e relative to h and a, and divided by the probability of e relative to a. This means that evidence improbable antecedently, but likely to obtain if the hypothesis is true, raises the probability of a hypothesis most. ”  The problem of assigning probabilities to antecedent evidence  exists, but in principle the theorem appears to be able to
deal with the type of information described as anecdotal.  It is worth adding that Bayes theorem  is widely used in science, engineering, computer modelling and robotics, so it has undeniable practical value.

Post a comment or leave a trackback: Trackback URL.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: