We all know about them – those legendary articles that transcend their fields and attain status as ‘compulsory annual reading for scientists.’ Their ideologies are pure, their arguments are well reasoned and their applications are manifold. Unfortunately, a small class of these articles maintain their theoretical impact because their practical impact is minimal, at least in certain fields. Whilst reading these masterpieces, we love them, but when we get back to work, we find it all too easy to slip into our bad old habits, comfortable in the knowledge that we know the principles of science even if the current project doesn’t best embody them.
Yesterday I read Strong Inference, an 8-page article by (bio)physicist John R. Platt published in Science in 1964 (doi:10.1126/science.146.3642.347). I don’t know if I was reading it for the first time, or if I had done so previously for an undergraduate Philosophy of Psychology class. Either way, I found coming across it and taking the time to read it (again) to be thoroughly invigorating. The basic argument is the Popperian tenet that there is no value in proof because data can prove multiple theories, there is only value in disproof and the subsequent disregard of the disproved theory. In the article, Platt supports this thesis by outlining a number of methods that afford this form of ‘strong inference’. These methods include an at first surprising dismissal of the testing of quantitatively distinct hypotheses in favour of testing qualitatively distinct hypotheses.
“Many-perhaps most-of the great issues of science are qualitative, not quantitative, even in physics and chemistry. Equations and measurements are useful when and only when they are related to proof; but proof or disproof comes first and is in fact strongest when it is absolutely convincing without any quantitative measurement.”
Reading the article provided an invigorating experience. It filled me with a ‘wonder of first-principles’ – a delight in the understanding of how and why something works – and it reminded me that the power of falsification forms the foundations of the scientific method. Then, when I examined my own position in relation to the article, and I found it less invigorating and a little more thought-provoking. I thought back to the situations where I lost sight of the goal of falsification and was content with trying to find evidence and a compelling argument to support a sensible idea… not a sin in itself, but it doesn’t represent strong inference as outlined by Platt. To take the metaphor of foundations further, maybe a house is first built over strong foundations, but as it expands it needs to stay true to those foundations. If it expands outwards and builds on its own structure without regard for whether or not the extensions are supported by the foundations, it will eventually display a very noticeable lack of structural integrity.
It is easy to spot that lack of structural integrity in the arguments of others, especially when they take a stance against your own position. It’s why we hate Reviewer 2 so much – “Of course he was going to recommend rejection” I might say, “How can he devote his career to Theory A when it’s totally inconsistent with finding F and any argument he makes to accommodate it makes his theory unfalsifiable?” But I must apply these same standards to my own research. What theories do my findings disprove? “Why am I still considering theory B,” I should also be saying, “when finding G and the H-effect disproved it nearly 20 years ago?” But of course this kind of self-critical thinking is uncomfortable for all of us, especially when we find ourselves to be (occasional) hypocrites.