1000 Hurts
Psychophysics is a human-facing science with interventions arguably more robust than medicine.
Anyone who’s read enough argmin dot net knows that I often write posts trolling for information. Monday’s post falls into that category, because I am legitimately curious about which human-facing sciences routinely find robust effects. And I got a legitimate, definitive answer from many passionate readers and friends: psychophysics. This is a compelling “exception to prove the rule” in human-facing science.
Psychophysics is the study of how humans perceive and process physical stimuli like light and sound. It is the science that maps out the limits of what we can perceive. Remarkably it has cataloged a long list of universal, robust principles of perception.
I feel dumb for not making this connection before because I actually know a lot of psychophysics. With the naive hope of making a career in music, I spent much of my youth studying the psychophysics of auditory perception. This subfield is so vital that it gets its own name: psychoacoustics. We can’t hear frequencies about 22,000 Hz, which is why we can’t hear dog whistles. We perceive loudness logarithmically, which is why we invented the decibel scale. There are more subtle effects, such as masking, in which a perceptible sound vanishes when played simultaneously with another sound of suitably higher volume. Even these curious, subtle effects are robustly replicable.
The laws of psychoacoustics are so robust and stable that we can build an entire engineering discipline on top of them. We remove the humans from the story and express the principles as mathematical equations and numerical tables. Psychology becomes signal processing. We can then translate these formulae into circuit diagrams and digital code. This code tells us how audio can be captured, compressed, decoded, and amplified without perceptual loss. The same principles enable technologies from telephones to concert loudspeakers. We now take for granted the ability to access high-quality audio from an infinitude of artists on our smartphones. Transmission of sound was somewhat more shocking one hundred years ago.
Visual psychophysics is similarly well established, and this field forms the backbone of our capture, compression, and reproduction of imagery. I mentioned a paper on Monday about memory and word confusion, and this had a similar psychophysical feel. We shouldn’t be surprised that they reported a highly significant effect in their replication study. Recent investigations into the psychophysics of sensorimotor control have also found robust principles that may lead to innovations in prosthetics and other human-machine interfaces. Konrad Kording pointed me to the contemporary sensorimotor psychophysics literature and noted that the results are so undeniable that the community rarely reports p-values. He’s right. The error bars are tiny. Because of its maturity and relatively small size, the field of psychophysics perhaps doesn’t find as many sigma interventions as medicine. But it’s still a place where consistent, robust interventions are discovered.
Why psychophysics is so much more robust than “soft” psychology is a question that psychologists have been torturing themselves about for a century. In one of the science reform community’s favorite early papers on the file drawer/publication bias problem, Anthony Greenwald explicitly calls out psychophysics as likely exempt from his considerations in “Consequences of prejudice against the null hypothesis.” Friend-of-the-blog Paul Meehl lists fifteen reasons why soft psychology fails to make progress in his 1978 magnum opus “Theoretical Risks and Tabular Asterisks: Sir Karl, Sir Ronald, and the Slow Progress of Soft Psychology.”
Go through Meehl’s list of 15 issues in soft psychology, and you’ll see none apply to psychophysics. You have very nicely defined quantitative outcomes: the number of objects, the location of a stimulus, the ordering of a list. You have precisely controlled interventions: you can finely vary the presentation stimuli, changing the color, amplitude, relative magnitude of background noise, and so on. Psychophysics often admits strongly predictive mathematical laws that robustly forecast responses of a diverse set of individuals. The quantifications have stable meanings across contexts, and we can transport them into engineering rules.
If anything, psychophysics is far better off scientifically than medicine. Disease pathology is often far more difficult to define in rigid terms. Few pharmaceutical interventions have clean mathematical models of action beyond the simple pharmacodynamics of absorption. Most chemicals are poison. Some chemicals are miracle cures. It’s astounding that we find anything that works at all.
And yet we do.
In addition to the precisely controlled interventions that are characteristic of psychophysical experimentation there are two other factors that contribute to the robustness and replicability of psychophysical experiments. Unlike in many other areas of psychology (and medicine), psychophysical experiments typically use a within subject design (comparing the responses of the same subject to two different interventions) which gets rid of the many complications involved in comparing across subjects. Related to this is that most psychophysical tasks can be done quickly and repeatedly by experimental subjects. That means it's possible to record hundreds or sometimes thousands of data points on an individual subject. It also means you don't need to recruit a lot of subjects. There was a common joke current in color science when I was first trying to master the basics of the discipline. A psychophysical experiment needs three subjects: the two authors plus the naive subject. This wasn't literally true but it did capture an important aspect of the literature back in the 1970s and 1980s. Thanks for an interesting post.
When I read your blog generally I understand what you're getting at almost immediately (and usually agree). In this series of posts you seem to be worried that statistics isn't a path to truth. As you have explained many times in earlier posts, often statistics is just a bureaucratic convention; a publication hurdle.
But bureaucracies, conventions and hurdles are not always useless even if imperfect. The rules of the road are just conventions, which are sometimes inefficient (e.g., I have to wait at a red light even if there's no traffic in the orthogonal direction) and certainly don't prevent all traffic accidents, but I'm pleased we have them. Likewise, while the p-value requirements are imperfect, I suspect we'd be overwhelmed with even worse papers if we didn't have them. While I expect we could improve both our statistical conventions and our traffic rules, actually doing both could be tricky.
I also see you hinting at the fact that statistical rigour has little to do with understanding something at a theoretical or pre-theoretical level. Control theory, causal models, and some economic modelling try to build models that capture aspects of the underlying phenomena. But as you've remarked in your posts, unstructured machine learning in the form of GenAI is where all the money is today.
The fields of psychology, psycho-linguistics and linguistics overlap a lot in terms of the phenomena they cover. Linguistics - perhaps because of Chomsky's influence - is very theoretical and famously numerophobic, while psychology tends to be very atheoretical and rely on statistical methods. I think linguistics has discovered many interesting facts about human language, but without any statistical information it is hard to tell if any specific claim is reliable.