This debate revealed a lot about how information is handled by the medical system (LOL, the line about medical doctors and the medical establishment). For this particular technology, it may be right that it will not lead to better outcomes, but I contend that there is a lot of good information that is not obtained because of the way that information gets co-opted by the medical-industrial-legal complex, creating testing cascades, etc. To correct this, healthcare needs to move beyond Bayes rule toward decision theory. The other parties - industry, legal, etc - need to change as well.
Kind of feels like the same crowd that convinced itself that token-maxing was the right way to use AI is now trying to convince everyone that scan-maxing is healthcare.
The hope is always that data has positive network effects rather than diminishing marginal returns and the log relationship between training data and capability for LLMs sure seems like it should've settled this for even the most enthusiastic boosters of this line of thought, but noooo
I've got a more optimistic take which I will give (and then promptly duck as I expect this probably isn't the vibe of the conversation). Midjourney as a company was profitable enough and the founder retained enough control (he has no VC investors!) that the founder was able to take a big chance and build a first version of something that could actually be beneficial. Now we can all see together whether it was a good or bad idea...
The problem, in my opinion, is that doctors focus on curing patients. Any intervention that does not help with that is going to be ignored at best, actively opposed at worst. But gathering data, even if it leads to little in the way of useful clinical interventions on current patients, is valuable per se. You can use a million of those scans to train a foundation model for medicine. Maybe this will be helpful for *future* patients, even if it’s useless or even harmful for current patients.
The biggest issue with the bitter lesson fromy perspective is that it is premised on a misunderstanding of capitalism or at least of the dynamics of competitive markets. Even if you leave aside the question of whether more data or compute will improve your algorithm, the fundamental problem is that what matters is not whether more compute/data will solve your problem. The real problem is whether more compute/data will decrease the cost to be below having a human do the thing (taking into account the added cost of that added compute and data). Famously, both David Graber and Peter Theil both made this exact point about why there are "no flying cars like in the Jetsons" because workers wages are too low to incentivize new techmology development in the pages of the Baffler in like 2012.
I would think a lot of the techno boosters are (mis)led by a belief that everything to do with compute/data exhibits emergent phenomenon with scale. Is this how people in Silicon Valley interpret The Bitter Lesson *jazz hands*?
But it's good to know that the radiologists are still around.
This debate revealed a lot about how information is handled by the medical system (LOL, the line about medical doctors and the medical establishment). For this particular technology, it may be right that it will not lead to better outcomes, but I contend that there is a lot of good information that is not obtained because of the way that information gets co-opted by the medical-industrial-legal complex, creating testing cascades, etc. To correct this, healthcare needs to move beyond Bayes rule toward decision theory. The other parties - industry, legal, etc - need to change as well.
Kind of feels like the same crowd that convinced itself that token-maxing was the right way to use AI is now trying to convince everyone that scan-maxing is healthcare.
The hope is always that data has positive network effects rather than diminishing marginal returns and the log relationship between training data and capability for LLMs sure seems like it should've settled this for even the most enthusiastic boosters of this line of thought, but noooo
I've got a more optimistic take which I will give (and then promptly duck as I expect this probably isn't the vibe of the conversation). Midjourney as a company was profitable enough and the founder retained enough control (he has no VC investors!) that the founder was able to take a big chance and build a first version of something that could actually be beneficial. Now we can all see together whether it was a good or bad idea...
The problem, in my opinion, is that doctors focus on curing patients. Any intervention that does not help with that is going to be ignored at best, actively opposed at worst. But gathering data, even if it leads to little in the way of useful clinical interventions on current patients, is valuable per se. You can use a million of those scans to train a foundation model for medicine. Maybe this will be helpful for *future* patients, even if it’s useless or even harmful for current patients.
"More information can make you sicker." You've articulated the real bitter lesson, for this century at least.
The biggest issue with the bitter lesson fromy perspective is that it is premised on a misunderstanding of capitalism or at least of the dynamics of competitive markets. Even if you leave aside the question of whether more data or compute will improve your algorithm, the fundamental problem is that what matters is not whether more compute/data will solve your problem. The real problem is whether more compute/data will decrease the cost to be below having a human do the thing (taking into account the added cost of that added compute and data). Famously, both David Graber and Peter Theil both made this exact point about why there are "no flying cars like in the Jetsons" because workers wages are too low to incentivize new techmology development in the pages of the Baffler in like 2012.
I would think a lot of the techno boosters are (mis)led by a belief that everything to do with compute/data exhibits emergent phenomenon with scale. Is this how people in Silicon Valley interpret The Bitter Lesson *jazz hands*?
But it's good to know that the radiologists are still around.