Using scientific methods to produce evidence is what makes psychology (together with medicine, climatology, biology, chemistry and physics, etc.) a ‘science’. A science isn’t defined by subject matter. It’s more about its methods.
We’ve all heard a lot about “following the science” during the coronavirus pandemic. So what does that actually mean in practice? How do psychologists create ‘scientific evidence’ so that others might use or “follow” it?
If you believe it, prove it
The cornerstone of science is to test our theories and opinions with controlled experiments. It’s not enough to believe, for example, that cats swish their tails because they are angry. If you want to present it as a ‘scientific fact’ you have to carry out an experiment and prove it. That’s science and the ‘scientific method’.
Using scientific methods is important because so many things in life turn out not to be true once we test them in an experiment. Cats don’t swish their tails because they’re angry, for example. Lemmings [spoiler alert] don’t jump off cliffs. And although it seemed bleedin’ obvious to countless generations of humans that the Earth is stationary in space – and the Sun moves across the sky – it turned out not to be true. So good science always tests the obvious, just in case our assumptions are wrong. So what does a psychology experiment look like?
Let’s say you have a theory that drinking alcohol will diminish driving performance. You can easily devise an experiment to test this. You set up a complex driving course, and then test the driving ability of a sober driver and compare it with the ability of a driver to whom you have administered alcohol. Did your drunken participant hit more cones on the driving course than the sober driver? If so, you can provisionally accept that the theory is true – and repeat the experiment.
The point about basing your facts in science is that you (try to) claim nothing to be true that sceptics cannot observe for themselves. But there are two important problems with using this type of experimental method with people.
Awkward people – and individual differences
The first problem is that people are all different. You might accidentally choose Lewis Hamilton for the ‘drunken condition’ – and find that his performance is actually better than that of the driver in the ‘sober condition’. This problem can be overcome fairly easily, by testing lots of people and using the average of their performance. Having one Lewis Hamilton in a sample of 1000 will not change the average much. So – if your theory about drunk drivers is true – then on average, drunk drivers will give those cones a more of a pasting than sober drivers .
It is for this reason than psychology is a science of statistics and probabilities – and not a science of hard facts like physics. We can’t test every single person, so we can’t predict what any particular individual will do or feel with 100% accuracy. All we can do in an experiment is work out what is very likely and what is very unlikely.
So we must be satisfied with having general laws and principles in psychology – that are probably true for most people in most circumstances. We cannot always predict what any specific individual will do. Although this limitation is a pain, it is not world-ending. Governments can base social policies on our understanding of what ‘most people’ think and do.
The second problem that arises from using experimental methods on people is much harder to deal with. It has to do with changing ethical values over the last fifty years.
Psychologists of the mid-twentieth century were often fairly cavalier about ethics. Today, ethics are centre stage. We can’t mess with people’s heads. Here’s an example to illustrate the problem…
Quite a few readers will have heard of Stanley Milgram’s now-classic study of conformity and obedience (Wikipedia has more details). His experiment showed that authority figures could coerce participants, remarkably easily, into administering very high electric shocks to unsuccessful ‘learners’. Today, this type of experiment would be unthinkable. Tricking participants, and coercing them into giving lethal electric shocks to other people (even though nobody actually received any shocks), would be considered grossly unethical. You could traumatise people for life. That kind of experiment is rarely seen today. Even psychological experiments that do not require participants to electrocute one another depend on disguising their real purpose from the participants – raising ethical questions about deception.
These ethical problems have meant that psychology’s ‘classic’ experimental methods have fallen out of favour. And because doing experiments on animals brings their own problems (some ethical and others concerned with problems of comparability between species), much psychological evidence now comes from questionnaire surveys and interviews. With questionnaires and interviews, psychologists measure everything from attitudes and personality characteristics to how well you’re sleeping. [There’s an article on questionnaire construction – coming soon.]
Questionnaires sidestep most ethical dilemmas and bring some other great advantages: They are much cheaper and easier to set up than experiments, and they can encompass vast sample sizes. They can be used to test thousands of people at once.
But there are problems with questionnaires…
Most of these problems are unavoidable. One serious problem is that psychological concepts (like personality types) cannot be seen or measured directly. We cannot see or directly measure agreeableness, empathy, introversion, narcissism or ADHD. This means that psychologists must infer these attitudes and types from indirect measures – especially questionnaires. So, for example, we might ask John if he likes helping other people. If John says, yes, he likes helping other people, he is assessed to be ’empathetic’). But of course, John could just be lying. We might test to see if people are ‘psychopaths’ by asking them if they cut up worms when they were children (the Empathy Quotient actually asks this question!) But, obviously, even the dimmest so-called psychopath can work out where questions about ‘cutting up worms’ are going.
Second, when questionnaires show relationships between two things, they can never tell us which of those things, if either, caused the other. If we find, for example, that the people who hate school also do badly at exams, we can’t be sure which caused which – or even that one caused the other at all. Questionnaire responses are also highly subjective. They rely on honesty and the ability of the participants to ‘know’ what they think. They also rely on the ability of participants to work out what the question is driving at. Psychology is littered with questionnaires that are – absurdly and amateurishly – ambiguous.
Unfortunately, the scientific difficulties caused by these problems are often magnified in practice, rather than reduced. Some psychologists are simply ignorant of the problems, while others just ignore them. Many high-profile questionnaires used in research today are designed very badly indeed. The recent and vast UCL Covid-19 survey, for example, was awash with shockingly simple and easily avoidable errors. [I’ll spell this out in the article about questionnaires.]
Another factor that compromises psychology as a science is that scientists are people. They are subject to the same psychological pressures and processes as everyone else. They want research grants, secure jobs and PhD qualifications – often far more than they want abstract ‘truth’. No one gets a PhD for doing technically brilliant research if it proves their own theory is wrong. The pressure to make ‘positive’ findings is intense. So if the data doesn’t support our theory, well, we can just quietly put that data in the bin.
The result of this dodgy methodology is that many recent findings using questionnaires cannot be repeated in follow up studies – where replication of original findings is the acid test of proper science. And it’s not that surprising. Caught between the relentless pressure to make brilliant new discoveries and increasing political pressure to discover socially acceptable things, some psychologists are starting to make the science up.
This paints a rather grim picture of modern psychology. And at a time when our explanatory rivals (especially psychiatry and neuroscience) are booming, there is a pressing need for a different approach.
Luckily, psychology is a hundred years old. And there is a vast reservoir of psychological science already out there – tested and proven over decades in countless replications – waiting to be sorted out and “followed”.
I hope to set some of that knowledge out in this website!