I ended Friday’s post by asking what I should do with this blog for the coming weeks. The first option was to dive into more criticism of statistical methods. The second was to pursue a potential “fourth path” out of the randomized trialomania.
The vote was overwhelming and nearly unanimous. Only two people voted for criticism, and they are the two chaos agents in the group chat. Today, we start down the fourth path.
Rereading Friday’s blog, I suspect my phrasing was tipping the scales a bit. No one likes to admit they want to read bitter criticism. And yet Twitter exists. But I’m excited (and a bit terrified) to try to expand on some of these ideas I’ve been thinking about for the past couple of years here.
Let’s start the series off riddle.
I am a massive fan of Japanese food. For my next sabbatical, I decide that I’m going to move to Japan and master cooking Japanese cuisine. Before I go, I know I need to learn to speak Japanese. I start doing Duolingo. I take Japanese classes with undergrads at Berkeley. Not satisfied, I hire a private tutor. After two years of hard work, I meet my hosts in Tokyo. We have a conversation in Japanese where they tell me how impressed they are with my conversational abilities.
In this scenario, was I able to speak Japanese because of the Japanese lessons?
Of course, the answer is yes. The only person who might even pause when answering this question is someone who has spent the last two years learning causal inference rather than a new language.
Now think about a different hypothetical scenario. Let’s imagine that a person is feeling low and depressed to the point where they are having trouble functioning. They visit a psychiatrist who diagnoses their new patient as having clinical depression. The psychiatrist writes them a prescription for Prozac. The person diligently takes their prescribed dose every day. After two years, the patient is feeling great, invigorated with life, and excited about the future.
In this scenario, was the person able to overcome depression because of the Prozac?
Not to be presumptuous, but I’m sure you are considerably less sure about the answer to this second question.
But why? Why does it feel obvious that language acquisition is caused by lessons and diligent practice whereas it does not feel obvious that depression is cured by SSRIs?
I’ve asked a bunch of different people this question, and I always get interesting answers. And I think that’s because the answer is complicated. Tell me your answer the comments or by email. Tomorrow I’ll summarize your responses, give you my best guess of how to resolve this causal riddle, and lay out how this riddle will set a course for the future of the blog.
Very rarely does someone pick up a language without either studying it or living in a country where the language is spoken. But many depressed people feel better over time even if they don't take any medication. So the difference between the two cases is whether the outcome is common even for people who don't take the given action.
The first case feels obvious because my internal world model predicted that you would be able to speak Japanese with a high probability given the context. That world model prediction was based on directly observing and hearing about many different kinds of people who have learned to speak different languages through dedicated practice (and likewise observing that people who don't practice speaking different languages are not capable of speaking different languages). Me claiming that "You were able to speak Japanese because of the Japanese lessons." is communicating that, had I spoke to you before you learned Japanese, I would've been willing to make a large bet that you would indeed be able to speak Japanese in the future after following the regimen described.
On the flip side, my internal world model did not assign a high probability to the depressed person being happy two years in the future given the context, which is why the causal claim doesn't feel obvious. That world model prediction was based on directly observing and hearing about many different kinds of people who have taken antidepressants and not been able to pull out of their depression, although some have (and likewise observing that some depressed people who don't take antidepressants can overcome their depression, although many don't). Me claiming that "I don't know if the person was able to overcome their depression because of the Prozac." is communicating that, had we spoke before the person started taking Prozac, I wouldn't have been willing to bet anything but a relatively small amount of money on them being happy in two years.