"The discovery of vitamins and deficiency diseases shows how a chaotic lack of reproducibility might be core to science itself"
Is this a descriptive or normative claim? If descriptive, then sure, nobody disagrees with your title "science has always been in crisis". But if it's normative, then no argument has been given, and hence, the second part of the title "This is fine" has been entirely ignored in substance. You've offered a single example of a successful discovery made pre-formal methods--what of the thousands of failures? To be sure, this post seems merely suggestive so maybe I am being too harsh. Looking forward to your substantive arguments in future posts then.
I recently read Carlos Chavez’s line that “the credibility revolution succeeded by developing research designs where the identifying assumptions were embedded in the structure of the data rather than imposed by the researcher.” (https://carloschavezp29.substack.com/p/why-macro-never-had-a-credibility).
To me, that framing captures the wrong turn that keeps the social sciences in permanent crisis. The identifying assumptions never vanish, they just move upstream into measurement, classification, sampling, and the choice of what even counts as "treatment," "unit," and "outcome." But many applied social scientists still convince themselves that data are somehow self-interpreting facts rather than acknowledge that they become evidence only through theory, and are already theory-laden categories. If the theory is thin or wrong, the categories can be wrong, and the statistics will deliver precise answers to a misconceived question. The fix is not to pretend we can design away judgment, it is to make judgment explicit, contestable, and honestly limited in scope.
"I’ll try to convince you that none of our fancy modern machinery would have helped at all. Both then and now, there are no formal paths to discovery."
By "fancy machinery" are you referring specifically to NHST? If so, I'd probably agree with you. But considering the best that statistics has to offer--which in, my opinion, is careful probabilistic modeling of the data-generating process--this claim feels way too strong.
Like a couple other commenters I'm skeptical but looking forward to seeing where you go with this.
Most or all philosoph of science texts agree, inasmuch as they distinguish the context of justification from the contex of discovery. They discuss and contend about the logic of explanation, but they agree that there's no necessary logic of discovery. That said, however, it seems that most of this discussion was about the logic of justification/explanation--then suddenly applying that discussion to the (non)logic of discovery.
A classic STS reference on a certain notion of reproducibility—specifically, witnessability by fellow gentlemen—being crucial to the development of modern science: "Leviathan and the Air-Pump: Hobbes, Boyle, and the Experimental Life" by Steven Shapin and Simon Schaffer: https://press.princeton.edu/books/paperback/9780691178165/leviathan-and-the-air-pump
(Definitely a different notion than those being referenced in the replication wars!)
"The discovery of vitamins and deficiency diseases shows how a chaotic lack of reproducibility might be core to science itself"
Is this a descriptive or normative claim? If descriptive, then sure, nobody disagrees with your title "science has always been in crisis". But if it's normative, then no argument has been given, and hence, the second part of the title "This is fine" has been entirely ignored in substance. You've offered a single example of a successful discovery made pre-formal methods--what of the thousands of failures? To be sure, this post seems merely suggestive so maybe I am being too harsh. Looking forward to your substantive arguments in future posts then.
I recently read Carlos Chavez’s line that “the credibility revolution succeeded by developing research designs where the identifying assumptions were embedded in the structure of the data rather than imposed by the researcher.” (https://carloschavezp29.substack.com/p/why-macro-never-had-a-credibility).
To me, that framing captures the wrong turn that keeps the social sciences in permanent crisis. The identifying assumptions never vanish, they just move upstream into measurement, classification, sampling, and the choice of what even counts as "treatment," "unit," and "outcome." But many applied social scientists still convince themselves that data are somehow self-interpreting facts rather than acknowledge that they become evidence only through theory, and are already theory-laden categories. If the theory is thin or wrong, the categories can be wrong, and the statistics will deliver precise answers to a misconceived question. The fix is not to pretend we can design away judgment, it is to make judgment explicit, contestable, and honestly limited in scope.
"I’ll try to convince you that none of our fancy modern machinery would have helped at all. Both then and now, there are no formal paths to discovery."
By "fancy machinery" are you referring specifically to NHST? If so, I'd probably agree with you. But considering the best that statistics has to offer--which in, my opinion, is careful probabilistic modeling of the data-generating process--this claim feels way too strong.
Like a couple other commenters I'm skeptical but looking forward to seeing where you go with this.
I just bought one. Thanks !
"there are no formal paths to discovery."
Most or all philosoph of science texts agree, inasmuch as they distinguish the context of justification from the contex of discovery. They discuss and contend about the logic of explanation, but they agree that there's no necessary logic of discovery. That said, however, it seems that most of this discussion was about the logic of justification/explanation--then suddenly applying that discussion to the (non)logic of discovery.
A classic STS reference on a certain notion of reproducibility—specifically, witnessability by fellow gentlemen—being crucial to the development of modern science: "Leviathan and the Air-Pump: Hobbes, Boyle, and the Experimental Life" by Steven Shapin and Simon Schaffer: https://press.princeton.edu/books/paperback/9780691178165/leviathan-and-the-air-pump
(Definitely a different notion than those being referenced in the replication wars!)
The hard sciences are fine. The danger, as I write here, is that bad social science threatens higher ed and has made universities particularly vulnerable to AI disruption. https://hollisrobbinsanecdotal.substack.com/p/attention-is-all-you-need-to-bankrupt