Funny thing you should mention Rosenblatt. His 1956 PhD thesis at Cornell was on psychometrics, and starts with the following passage:
"All research psychologists are familiar with problems in which the simultaneous working of a large number of variables seems to determine a piece of behavior, or a personality trait, or the outcome of an experiment. Such complex relationships are not peculiar to psychology; they are equally true, for example, of the gas laws in physics. However, psychology more than the physical sciences must deal with these relationships statistically, rather than as perfect mathematical functions."
It's not at all obvious to me that the uncertainty in these examples is more epistemic than aleatoric. It could be, if you buy the claim that "If all the causes in his case were known, we could predict for him perfectly". But that claim seems at least as true of the outcome of a literal dice roll, the paradigmatic example (and etymological origin) of aleatoric uncertainty—if anything, the dice roll is probably _more_ predictable, since the physics governing it are less chaotic than those governing psychological causation.
Maybe a pedantic distinction, but possibly important if we're talking about the philosophical foundations of probability.
Well, it's certainly true that I don't need Meehl to convince me to use statistics in decision-making!
But IIUC, _you_ are saying "Even though these questions are about individual people, their answers have epistemic uncertainty." And I'm asking: if the uncertainty about the answers to these questions isn't aleatoric, then what the hell is? These are questions about things that haven't happened yet, whose outcomes will be determined in significant part by the outcomes of chaotic processes that are provably unpredictable even with near-perfect measurement of current conditions (to say nothing of quantum stuff). Statements like "If all the causes in his case were known, we could predict for him perfectly (barring environmental accidents)" are vacuous, because the probability of a relevant "environmental accident" is 1.
It's true that there's also probably epistemic uncertainty—there's always more relevant stuff that we could measure—but I think the reason these questions are hard is because there's no practical set of measurements that can bring our uncertainty close to zero. (At least, with our current theories!)
I may have misunderstood how you were using the word aleatoric. I use it to mean "random." But you seem to be using it to mean "unpredictable." Is that right? I don't consider these to be the same, but I am probably missing context.
I'm not sure I believe in randomness except as a useful modeling simplification. I find it challenging to come up with situations where the difference between "random" and "unpredictable" matters in any testable sense. But I'd be happy to be convinced otherwise!
The one thing the quote doesn't really account for is that making a prediction can change the underlying dynamics. Predictions can become self-fulfilling prophecies. Of course, the effect of this is not clear (and sometimes there are cases where maybe one is motivated to prove the prediction wrong in which case the effect is in the opposite direction). However, *if* the average effect of making a prediction was indeed that the outcome is more likely to align to the prediction, and if this effect is stronger when the prediction is made by some "rational", big system rather than an individual practitioner (very possible these two would have different effects on the psyche), then in a way we could say that measuring hit rates of statistical vs. clinical predictions is rigged in favor of the statistical approach. Of course, these are all very big ifs.
Funny thing you should mention Rosenblatt. His 1956 PhD thesis at Cornell was on psychometrics, and starts with the following passage:
"All research psychologists are familiar with problems in which the simultaneous working of a large number of variables seems to determine a piece of behavior, or a personality trait, or the outcome of an experiment. Such complex relationships are not peculiar to psychology; they are equally true, for example, of the gas laws in physics. However, psychology more than the physical sciences must deal with these relationships statistically, rather than as perfect mathematical functions."
Sort of a funny statement because the gas law is also statistical...
That's the allure of the functional form -- once you write one down, it is so tempting to forget that it may have had statistical origins.
It's not at all obvious to me that the uncertainty in these examples is more epistemic than aleatoric. It could be, if you buy the claim that "If all the causes in his case were known, we could predict for him perfectly". But that claim seems at least as true of the outcome of a literal dice roll, the paradigmatic example (and etymological origin) of aleatoric uncertainty—if anything, the dice roll is probably _more_ predictable, since the physics governing it are less chaotic than those governing psychological causation.
Maybe a pedantic distinction, but possibly important if we're talking about the philosophical foundations of probability.
I imagine that if you already believe that the uncertainty is aleatoric, you don't need Meehl to convince you to use statistics in decision-making.
What makes Meehl's argument powerful is his case for statistics in the face of epistemic uncertainty. More on this to come, of course.
Well, it's certainly true that I don't need Meehl to convince me to use statistics in decision-making!
But IIUC, _you_ are saying "Even though these questions are about individual people, their answers have epistemic uncertainty." And I'm asking: if the uncertainty about the answers to these questions isn't aleatoric, then what the hell is? These are questions about things that haven't happened yet, whose outcomes will be determined in significant part by the outcomes of chaotic processes that are provably unpredictable even with near-perfect measurement of current conditions (to say nothing of quantum stuff). Statements like "If all the causes in his case were known, we could predict for him perfectly (barring environmental accidents)" are vacuous, because the probability of a relevant "environmental accident" is 1.
It's true that there's also probably epistemic uncertainty—there's always more relevant stuff that we could measure—but I think the reason these questions are hard is because there's no practical set of measurements that can bring our uncertainty close to zero. (At least, with our current theories!)
Anyway, looking forward to the next part!
I may have misunderstood how you were using the word aleatoric. I use it to mean "random." But you seem to be using it to mean "unpredictable." Is that right? I don't consider these to be the same, but I am probably missing context.
I'm not sure I believe in randomness except as a useful modeling simplification. I find it challenging to come up with situations where the difference between "random" and "unpredictable" matters in any testable sense. But I'd be happy to be convinced otherwise!
FWIW, I think Wikipedia is on my side:
https://en.wikipedia.org/wiki/Uncertainty_quantification#Aleatoric_and_epistemic
What I love about that wikipedia distinction is we're both right.
The one thing the quote doesn't really account for is that making a prediction can change the underlying dynamics. Predictions can become self-fulfilling prophecies. Of course, the effect of this is not clear (and sometimes there are cases where maybe one is motivated to prove the prediction wrong in which case the effect is in the opposite direction). However, *if* the average effect of making a prediction was indeed that the outcome is more likely to align to the prediction, and if this effect is stronger when the prediction is made by some "rational", big system rather than an individual practitioner (very possible these two would have different effects on the psyche), then in a way we could say that measuring hit rates of statistical vs. clinical predictions is rigged in favor of the statistical approach. Of course, these are all very big ifs.
I also think the game is rigged here, but for different reasons. More on this in coming posts!