Brain Games

Meehl's Philosophical Psychology, Lecture 6, part 3.

This post digs into Lecture 6 of Paul Meehl’s course “Philosophical Psychology.” You can watch the video here. Here’s the full table of contents of my blogging through the class.

There is an inherent tension between rules and play in the game of science.  Because scientists approach other scientists' results with skepticism, they weed out folklore from facts in a way that other modes of inquiry don’t. So we invent rules to make it easier to attack each other’s work. Show me the p-values. Show me the preregistration. Show me the identification strategy. Show me.

But where do the ideas come from? How do you decide which papers to write? If you want to discover new facts, being rigid and skeptical is a terrible starting point. You need to be creative. You need to play. Here’s Meehl:

“I don't know of a single really brilliant mathematician or physicist who has written a book saying ‘The way I made my discoveries was to shackle my leg to the chair like Hemingway writing for four hours before he could get drunk every day.’ They don't write about it that way. People like Poincare talk in terms of dreams, free associations, and going fishing. The typical attitude and physics is it should be a little wacky. There's a famous quote from Niels Bohr about somebody's theory of the nucleus: ‘The only thing I don't like about this is not crazy enough.’ 

“Physicists are much less spastic than we are. They can relax because they know they’ve got the best science there is. Blow up the damn world! That's how good theirs is. So they're very relaxed. They're not puffed and stuffy and pompous about their vocabulary. They talk about charm and color and strawberry flavors in the nucleus. We never would do that in psychology. If we had to talk about strawberry flavor, we’d look up the Latin word for strawberry.”

The scientific mind needs to have two conflicting personas. One for coming up with theories. One for testing them.

“When I characterize science as being tough-minded that means in corroborating or refuting the theory. That’s stage two. It's not in the stage of creating it. it's not in the stage of designing the experiments to test it. There you should be freewheeling. Let your hair down. Allow your creative spark to function.”

Herein lies the key tension. When reading everyone else’s work you need to be skeptical. In your own work, you need to be freewheeling. Skepticism requires rules.  Discovery requires play. Research is a game of inquiry. We’re not going to get anywhere if we don’t appreciate this tension. If we suck the fun out of it. If we make it too hard to play.

My affection for research play gets me into endless trouble. I’ve been chastised by many senior scientists for not taking research seriously enough or for having too many jokes in papers. “Science should be a serious business!” They tell me. But the tension between rules and play is part of what drives research forward. So middle-aged me has embraced that I get myself into trouble no matter what I do. I’m here to advocate for play, and I have constructive suggestions.

The first concerns graduate education. In K-12 science, we have all sorts of ways to make the topic captivating and fun. Egg drop experiments or growing crystals or whatever. In graduate school, we only teach rigid methods and host reading groups tearing apart papers. As Meehl notes, the blame rests on the shoulders of faculty.

“It's our fault. The faculty does this to you. We get your head fixed so that to not have a random sample is the worst conceivable thing a person could do.

“The mental set is to play it safe so that the peer group won't think you're dumb or the faculty won't think you're stupid. And the way you can play it safe is to make the usual at picking statistical criticisms. I mean that's it's always safe to [ask] “Was this a totally random sample of the western hemispheres schizophrenic?” It's bound to work, because it never is. So you can get by with that and get reinforced.”

We need to find ways to teach graduate students to occupy two personas—“Offense” and “Defense” if you will. We seldom do seminars on creativity and play. We should do more of them! We should encourage grad courses where projects can be speculative and weird and not necessarily targeted toward future education. We should run creative writing seminars. We should run reading groups on offbeat topics. Encouraging play has to be part of the process. 

My second suggestion is to abandon pre-publication peer review. I’ve written about this before, and I’ll write about it again, but reviewing is a practice whose time has come and gone.¹ It’s an unpaid waste of everyone’s time and gives undue authority to rules that hinder progress.

We could treat reviewing as a sanity check rather than a full-on adversarial attack. Is the paper written well? Is it plagiarism or fraud? Did they provide sufficient means for others to reproduce the work? If so, publish.

Ssince we’re not getting rid of peer review tomorrow, let me at least add some constructive criticism for how to approach reviewing. That we have all of these classes on methods and none cover how to be a good referee is pretty weird. I support Meehl’s proposal:

“I was a good [referee] in the sense that I tried to enter into the author's frame of reference. I asked the clinician in me, “What is this person trying to think about?” [I didn’t ask] what I'm thinking about! I mean I can write my own damn article. What is this author trying to say? My task as a referee is not whether he converts me, but whether he explains what he's up to so that I can see, as the Supreme Court says, “a rational mind being converted.” Even if I don't get converted!”

Make a steelman argument for every paper you read. Don’t think your primary role is gatekeeping. Remember that reviewing isn’t about you and your predilections. Remember that reviewing can’t determine whether a paper will be revolutionary. Figure out if the paper makes a cogent, clear argument. Can you make a case that someone might be convinced by whatever is written?

What deserves to be published, after all? What should the rules be? Should there be rules?

1. If you want a good argument in one place, I agree with almost all of what Adam Mastroianni says here.