So many great points in this essay. It's interesting to think about your arguments from an econometrics perspective. Model choice in econometrics is guided by economic theory and concerns about causal inference, and econometricians constantly wrestle with issues like endogeneity, model misspecification, & valid instruments to ensure meaningful causal insights, not just statistical significance. The DGP needs to reflect realistic economic assumptions (e.g., market frictions, equilibrium, non-stationary processes, etc.). While both disciplines rely heavily on solvable methods, the econometric approach is more constrained by underlying theory and/or hypothesis, and as i wrote the other day, that often leads to tossing some real neat/efficient mathematics to more sleight of hand techniques.
Statistics is so messy because we conflate so many different applications. When models, measurement, probability, decisions, uncertainty, etc. all get funneled into computational tooling, we forget these are all very different things! Let me collect some thoughts over the weekend and then blog more about this conflation next week.
Ive always been of the opinion that anytime you think you can logic a tough decision making problem out with math yet the field is inherently full of subjective normative statements (i.e., “we should do xyz to change the current situation” or “i think it should be done this way because of xyz” note this is different than “if this is true, xyz must be true” which never happens when two humans are involved *see chapter 4-5 of Herbert Simon’s The Sciences of the Artificial*) is inevitably going to be fraught with biases and unfairness. Statistics is just one of the many means that can help further the discourse and argument in decision making. It’s just some people think we should use it as the ONLY decision making tool when making decisions within the justice system and medical system, which typically contains problems that are so complex (and dare i say wicked..? *see Horst Rittel*) that it is impossible to be rational and rarely contains the nice propositional logic framework necessary for it to be solved solely by math. In those cases, there are many good and bad solutions depending on how you turn your head and view the problems within those fields. Depending on how you feel about those problems, you can always find a statistical justification for one decision if you frame your problem a certain way. The only way to go about making decisions within those field of interest is to have discourse and debate and constantly iterate on the decision with different stakeholders rather than pointing at math as the only holy grail. Inevitably, people will disagree and thus there will always be “bias” and “unfairness” but that’s why we make compromises and make adjustments to decisions as we engage with more and more with stakeholders that are intimately affected by the outcome of a decision that other people make.
I am sorry for the long response. I feel like I am just yelling into a crowd of AI people who will inevitably ignore me and get confused as to why people cant stats their way out of world hunger when I make this point.
Love this post. I took EECS227AT before and am now taking Stat210A. I thought they were two very different fields at first, but it was really fascinating to learn that so many important statistical tools, like hypothesis testing, are actually optimization problems! Would you plan to cover those topics in the intersection of optimization and statistics, in Stat241B next semester?
Nice! I really, really liked this blogpost!
So many great points in this essay. It's interesting to think about your arguments from an econometrics perspective. Model choice in econometrics is guided by economic theory and concerns about causal inference, and econometricians constantly wrestle with issues like endogeneity, model misspecification, & valid instruments to ensure meaningful causal insights, not just statistical significance. The DGP needs to reflect realistic economic assumptions (e.g., market frictions, equilibrium, non-stationary processes, etc.). While both disciplines rely heavily on solvable methods, the econometric approach is more constrained by underlying theory and/or hypothesis, and as i wrote the other day, that often leads to tossing some real neat/efficient mathematics to more sleight of hand techniques.
Statistics is so messy because we conflate so many different applications. When models, measurement, probability, decisions, uncertainty, etc. all get funneled into computational tooling, we forget these are all very different things! Let me collect some thoughts over the weekend and then blog more about this conflation next week.
Here’s a rant against optimization of scalar objectives (convex or not) more generally: https://arxiv.org/abs/2006.02577
Ive always been of the opinion that anytime you think you can logic a tough decision making problem out with math yet the field is inherently full of subjective normative statements (i.e., “we should do xyz to change the current situation” or “i think it should be done this way because of xyz” note this is different than “if this is true, xyz must be true” which never happens when two humans are involved *see chapter 4-5 of Herbert Simon’s The Sciences of the Artificial*) is inevitably going to be fraught with biases and unfairness. Statistics is just one of the many means that can help further the discourse and argument in decision making. It’s just some people think we should use it as the ONLY decision making tool when making decisions within the justice system and medical system, which typically contains problems that are so complex (and dare i say wicked..? *see Horst Rittel*) that it is impossible to be rational and rarely contains the nice propositional logic framework necessary for it to be solved solely by math. In those cases, there are many good and bad solutions depending on how you turn your head and view the problems within those fields. Depending on how you feel about those problems, you can always find a statistical justification for one decision if you frame your problem a certain way. The only way to go about making decisions within those field of interest is to have discourse and debate and constantly iterate on the decision with different stakeholders rather than pointing at math as the only holy grail. Inevitably, people will disagree and thus there will always be “bias” and “unfairness” but that’s why we make compromises and make adjustments to decisions as we engage with more and more with stakeholders that are intimately affected by the outcome of a decision that other people make.
I am sorry for the long response. I feel like I am just yelling into a crowd of AI people who will inevitably ignore me and get confused as to why people cant stats their way out of world hunger when I make this point.
Love this post. I took EECS227AT before and am now taking Stat210A. I thought they were two very different fields at first, but it was really fascinating to learn that so many important statistical tools, like hypothesis testing, are actually optimization problems! Would you plan to cover those topics in the intersection of optimization and statistics, in Stat241B next semester?
How did you not go with "new noise" as the subtitle?