I hope this is written tongue-in-cheek, because otherwise, it's one of those cases where someone manages to instantly shatter their credibility. Going forward with this blog would require not so much Gell-Mann amnesia, but a Gell-Mann lobotomy.
I'm very tempted to go into full rant-mode, but it's late so I'll limit myself to the following:
1. While it was discovered at a single collider, the signal for the Higgs was found by two separate experiments (i.e. independently designed detectors) at different locations along the LHC ring. Sure, systematics can be correlated within the same location but it would be quite unlikely for the same outdoor rumble to show up in this particular signal and in no other search.
2. The LHC experiments did not stop their work on Higgs related physics after its discovery -- even now, more precise measurements of its mass and decay width are being made, see https://arxiv.org/abs/2409.13663 .
3. The Higgs field gives rise to the mass of elementary particles, and thus contributes to things like the stability of the proton, and rate of radioactive decay (and hence why the sun is able to shine continuously). The fact that the Higgs field couples to known particles in proportion to their masses may mean it couples to otherwise undetectable massive particles as well, which may be one way of ascertaining what dark matter is.
4. The properties of the Higgs boson and field as measured are to say the least, not ideal in terms of theory. In some sense, it would have been nicer if it hadn't been found at all! (It might yet be that these properties are explained by the Higgs being a composite particle itself.)
To put it another way, there is a tremendous incentive for particle physicists to similarly produce the discovery of SUSY via an LSP candidate, or indeed any new signal post-2012. Have the bureaucrats been too busy with congratulatory backslapping to cook something up?
I'm astounded by the small-mindedness of it all by someone professing to be so learned. We've been clever enough to discover things that are far removed from our every day experience. But there remoteness does not preclude them from ever being useful in our lives, though it may well take time for the seeds of discovery to bear the fruit of utility.
In particular I really don't think the comparison to "QRPs" / p-hacking makes any sense. High energy physics certainly has a lot of problems right now, but its problems are very distinct from the replication crisis of social psychology. The standards created to try and deal with the issues in psych just don't apply in a naive way.
Impressive levels of irreverence here. Probably more useful to re-direct this energy and momentum to pick apart big pharma RCTs, where neither data nor analyses are out in the open. Particle physics receives a lot of sunlight, so to speak, and other uses of statistics, with arguably more direct (and negative) consequences, ought to be scrutinized with this level of motivation more often.
Yes, 100% agree. I'll certainly be back to grumbling about the lack of transparency in drug trials before too long.
I brought up the Higgs because it's the canonical example of a discovery that wouldn't have been possible without null hypothesis testing. This post highlights how even here, it's incredibly tricky to draw conclusions from frequentist standards. I like the way David Chapman (@meaningness) puts it here: https://substack.com/@meaningness/note/c-73906654?utm_source=activity_item
I'm struggling to see how this argument doesn't apply to any subatomic particle, in particular electrons, or even silver atoms. Or is this just a complaint about scale?
My argument is conjuring the ghost of Ernst Mach who was famously skeptical of the existence of atoms. But the story of how we decided atoms existed is informative in this context.
Perrin provided evidence from 13 diversely different arguments. And then further study using the atomic model led to new discoveries *and* new engineering applications. My polemic here is just trying to highlight how it is often rather tricky about where you draw the discovery line.
Good to know my reading wasn’t that far off! It’s a reasonable commentary on the scale of the project too (not to mention Academia, broadly). The arguments tallied by Perrin might quality as “indirect” evidence, which particle physicists have reams of. Collider experiments, cosmic ray experiments (terrestrial, ballon-borne, satellite), neutrino experiments from nuclear reactors, the sun, etc all test the Standard Model collectively. And there are a lot of observables, from basic kinematics to branching ratios, to the scaling of various parameters with energy scale. Given the important role that the Higgs plays in the electroweak theory, wouldn’t that volume of evidence play a similar, indirect role?
The discovery of gravitational waves by LIGO also involved a fair amount of statistical analysis: I believe that they looked for a chirp close to one in a bank of 250K chirps generated by a two-parameter model of black hole collisions, then considered whether that match was robust to random noise under some assumed models of noise.
One thing LIGO has over the particle accelerators is their promise of early detection of astrophysical events. Here's one such event that was first picked up on LIGO and then found on optical telescopes in the next few hours: https://www.ligo.org/detections/GW170817.php
Ben is an exception on most counts, but since the CS bros essentially run the economy now, they're metaphorically killing the father discipline here.
"Hey this whole complicated scientific edifice that is intimidating and threatens your intellectual legacy -- it was just a bunch of over-credentialed bureaucrats validating their scientific priors."
"With 10 billion dollars, they confirmed a model of particles that connects the crowning achievements of 20th-century physics. Through a massive collaborative government, they achieved a democratic consensus about the fundamental building blocks of physical theory, a united nations of physics. The Higgs Discovery is a celebration of modern bureaucracy, not a revelation about material reality."
A whole bunch of money, bureaucracy, "faith", etc. And you can't possibly begin to understand this stuff because it's super duper complicated.... Ugh. That's science?! No thanks. (I am somewhat tongue-in-cheek here but it's not the kind of science I want to do or have an impact on.)
Ian Hacking phrased his qualified entity realism by saying "if you can spray it, it's real." He was referring to a proposed experiment to confirm the existence of quarks, where a very cold ball of niobium is sprayed with either positrons to increase the charge or with electrons to decrease the charge. What is even the analogue of this in the Higgs experiment? What can you spray, so to speak?
The experiment convinced Hacking that electrons and positrons exist (not the fractional charges the experiment was looking for). The direct analogy in the LHC would be protons. The LHCs "sprays" protons on protons, so it would convince Hacking that protons exist (not the Higgs).
Right, I'd argue that explosions and inference don't count as spraying for existence. But then where does this leave astrophysics? Can you spray for supernovae?
I think the philosophy of science and the philosophy of technology diverges at this point. Discovering truth in the scientific realm is just not the same as probing what we can create given the constraints of reality. This spraying analogy seems more grounded along the lines of: “I cant use this scientific model for any technological thing until I can see it works in front of my eyes on a real world tangible example”. A lot of scientific models don’t seem to need that second part. You could just believe that that’s just how it works but the scientific community in astrophysics just demands a lot more rigor than say… people who believe in the efficacy of 3000 year old chinese medicines or whatever.
There is the REAL issue with funding pressure, self government & experimental complexity that no one person fully understands. This might be another bigger graveyard than the myelin-theory Alzheimer with Masliah and his decade if misdirected research funding.
Very thought-provoking observations, Ben! I am also totally following your argument about 5σ and p<0.05 being purely conventions which on their own are not ideal paragons of truth. As any measure which holds any importance, they are particularly subject to Goodhart's Law. However, I am stumped why you use the potential drawback and failure modes of these measures, as well as the potential of CERN scientists making false assumptions and mistakes to make the claim that the Higgs discovery has no relation to physical reality.
Sure, such a complex apparatus (and I don't just mean the accelerator) filters reality and makes potentially false assumptions. But is this really different to the way that each of us individually filters reality? I hope you see the Popplerian idea at which I am getting at. At the end of the day, if we follow your line of reasoning, we can never claim with any certainty that we are discovering anything of true physical reality. The best thing we can do is to try to falsify our assumptions. And isn't that precisely the utility of the measures such as 5σ? Sure, the results from CERN do not guarantee with 100% certainty that there is physical Higgs boson, but as you say so yourself, that what we understand of physics at these scales is so mind-boggling that it is hard to conceive as being an actual physical reality and not mathematical flights of fancy. But what their finding does do is provide sufficient evidence such that my (and many other physicist's) credence of such an object existing is substantiated.
And yes, at the end of the way it is about belief. But not purely on the bureaucracy of CERN, but the inescapable bureaucracy of the systems we set in place to define scientific findings. As we continue to probe regions which are increasingly removed from our physical perception of reality, what other hopes do we have?
I hope this is written tongue-in-cheek, because otherwise, it's one of those cases where someone manages to instantly shatter their credibility. Going forward with this blog would require not so much Gell-Mann amnesia, but a Gell-Mann lobotomy.
I'm very tempted to go into full rant-mode, but it's late so I'll limit myself to the following:
1. While it was discovered at a single collider, the signal for the Higgs was found by two separate experiments (i.e. independently designed detectors) at different locations along the LHC ring. Sure, systematics can be correlated within the same location but it would be quite unlikely for the same outdoor rumble to show up in this particular signal and in no other search.
2. The LHC experiments did not stop their work on Higgs related physics after its discovery -- even now, more precise measurements of its mass and decay width are being made, see https://arxiv.org/abs/2409.13663 .
3. The Higgs field gives rise to the mass of elementary particles, and thus contributes to things like the stability of the proton, and rate of radioactive decay (and hence why the sun is able to shine continuously). The fact that the Higgs field couples to known particles in proportion to their masses may mean it couples to otherwise undetectable massive particles as well, which may be one way of ascertaining what dark matter is.
4. The properties of the Higgs boson and field as measured are to say the least, not ideal in terms of theory. In some sense, it would have been nicer if it hadn't been found at all! (It might yet be that these properties are explained by the Higgs being a composite particle itself.)
To put it another way, there is a tremendous incentive for particle physicists to similarly produce the discovery of SUSY via an LSP candidate, or indeed any new signal post-2012. Have the bureaucrats been too busy with congratulatory backslapping to cook something up?
I'm astounded by the small-mindedness of it all by someone professing to be so learned. We've been clever enough to discover things that are far removed from our every day experience. But there remoteness does not preclude them from ever being useful in our lives, though it may well take time for the seeds of discovery to bear the fruit of utility.
In particular I really don't think the comparison to "QRPs" / p-hacking makes any sense. High energy physics certainly has a lot of problems right now, but its problems are very distinct from the replication crisis of social psychology. The standards created to try and deal with the issues in psych just don't apply in a naive way.
Impressive levels of irreverence here. Probably more useful to re-direct this energy and momentum to pick apart big pharma RCTs, where neither data nor analyses are out in the open. Particle physics receives a lot of sunlight, so to speak, and other uses of statistics, with arguably more direct (and negative) consequences, ought to be scrutinized with this level of motivation more often.
Yes, 100% agree. I'll certainly be back to grumbling about the lack of transparency in drug trials before too long.
I brought up the Higgs because it's the canonical example of a discovery that wouldn't have been possible without null hypothesis testing. This post highlights how even here, it's incredibly tricky to draw conclusions from frequentist standards. I like the way David Chapman (@meaningness) puts it here: https://substack.com/@meaningness/note/c-73906654?utm_source=activity_item
I'm struggling to see how this argument doesn't apply to any subatomic particle, in particular electrons, or even silver atoms. Or is this just a complaint about scale?
My argument is conjuring the ghost of Ernst Mach who was famously skeptical of the existence of atoms. But the story of how we decided atoms existed is informative in this context.
https://www.argmin.net/p/les-atomes
Perrin provided evidence from 13 diversely different arguments. And then further study using the atomic model led to new discoveries *and* new engineering applications. My polemic here is just trying to highlight how it is often rather tricky about where you draw the discovery line.
Good to know my reading wasn’t that far off! It’s a reasonable commentary on the scale of the project too (not to mention Academia, broadly). The arguments tallied by Perrin might quality as “indirect” evidence, which particle physicists have reams of. Collider experiments, cosmic ray experiments (terrestrial, ballon-borne, satellite), neutrino experiments from nuclear reactors, the sun, etc all test the Standard Model collectively. And there are a lot of observables, from basic kinematics to branching ratios, to the scaling of various parameters with energy scale. Given the important role that the Higgs plays in the electroweak theory, wouldn’t that volume of evidence play a similar, indirect role?
The discovery of gravitational waves by LIGO also involved a fair amount of statistical analysis: I believe that they looked for a chirp close to one in a bank of 250K chirps generated by a two-parameter model of black hole collisions, then considered whether that match was robust to random noise under some assumed models of noise.
One thing LIGO has over the particle accelerators is their promise of early detection of astrophysical events. Here's one such event that was first picked up on LIGO and then found on optical telescopes in the next few hours: https://www.ligo.org/detections/GW170817.php
Ah, that inspires more confidence than the paper discussing the 2015 observation.
Beautiful analysis (and utterly damning of experimental high energy physics)! Or, rather, confirmed many things I thought about the area.
how is it damning?
Ben is an exception on most counts, but since the CS bros essentially run the economy now, they're metaphorically killing the father discipline here.
"Hey this whole complicated scientific edifice that is intimidating and threatens your intellectual legacy -- it was just a bunch of over-credentialed bureaucrats validating their scientific priors."
"With 10 billion dollars, they confirmed a model of particles that connects the crowning achievements of 20th-century physics. Through a massive collaborative government, they achieved a democratic consensus about the fundamental building blocks of physical theory, a united nations of physics. The Higgs Discovery is a celebration of modern bureaucracy, not a revelation about material reality."
A whole bunch of money, bureaucracy, "faith", etc. And you can't possibly begin to understand this stuff because it's super duper complicated.... Ugh. That's science?! No thanks. (I am somewhat tongue-in-cheek here but it's not the kind of science I want to do or have an impact on.)
that's fine do something else Anna!
Ian Hacking phrased his qualified entity realism by saying "if you can spray it, it's real." He was referring to a proposed experiment to confirm the existence of quarks, where a very cold ball of niobium is sprayed with either positrons to increase the charge or with electrons to decrease the charge. What is even the analogue of this in the Higgs experiment? What can you spray, so to speak?
The experiment convinced Hacking that electrons and positrons exist (not the fractional charges the experiment was looking for). The direct analogy in the LHC would be protons. The LHCs "sprays" protons on protons, so it would convince Hacking that protons exist (not the Higgs).
Yes, that was exactly my point.
Right, I'd argue that explosions and inference don't count as spraying for existence. But then where does this leave astrophysics? Can you spray for supernovae?
I think the philosophy of science and the philosophy of technology diverges at this point. Discovering truth in the scientific realm is just not the same as probing what we can create given the constraints of reality. This spraying analogy seems more grounded along the lines of: “I cant use this scientific model for any technological thing until I can see it works in front of my eyes on a real world tangible example”. A lot of scientific models don’t seem to need that second part. You could just believe that that’s just how it works but the scientific community in astrophysics just demands a lot more rigor than say… people who believe in the efficacy of 3000 year old chinese medicines or whatever.
Or we could agree that science is about corroborating or falsifying theoretical claims, leaving “existence” to metaphysics?
There is the REAL issue with funding pressure, self government & experimental complexity that no one person fully understands. This might be another bigger graveyard than the myelin-theory Alzheimer with Masliah and his decade if misdirected research funding.
the argument of - "i don't understand it and its really complicated so therefore its not true" is not particularly convincing to anyone.
https://www.researchgate.net/publication/323897272/figure/fig1/AS:606467224788992@1521604462526/The-expected-and-observed-four-lepton-invariant-mass-distribution-for-the-selected-Higgs.png
https://cms-results.web.cern.ch/cms-results/public-results/publications/HIG-22-001/CMS-HIG-22-001_Figure_0B3-b.png
the Higgs observation is now at maybe 20-30 sigma now and is not controversial , except for internet trolls
Are you familiar with how CERN blinds their analyses? https://cmsexperiment.web.cern.ch/physics/cms-higgs-search/blinding-and-unblinding-analyses
“There has to be something we can do with substantive causal theories for them to be real.” I guess this rules out astrophysics as “real”?
Very thought-provoking observations, Ben! I am also totally following your argument about 5σ and p<0.05 being purely conventions which on their own are not ideal paragons of truth. As any measure which holds any importance, they are particularly subject to Goodhart's Law. However, I am stumped why you use the potential drawback and failure modes of these measures, as well as the potential of CERN scientists making false assumptions and mistakes to make the claim that the Higgs discovery has no relation to physical reality.
Sure, such a complex apparatus (and I don't just mean the accelerator) filters reality and makes potentially false assumptions. But is this really different to the way that each of us individually filters reality? I hope you see the Popplerian idea at which I am getting at. At the end of the day, if we follow your line of reasoning, we can never claim with any certainty that we are discovering anything of true physical reality. The best thing we can do is to try to falsify our assumptions. And isn't that precisely the utility of the measures such as 5σ? Sure, the results from CERN do not guarantee with 100% certainty that there is physical Higgs boson, but as you say so yourself, that what we understand of physics at these scales is so mind-boggling that it is hard to conceive as being an actual physical reality and not mathematical flights of fancy. But what their finding does do is provide sufficient evidence such that my (and many other physicist's) credence of such an object existing is substantiated.
And yes, at the end of the way it is about belief. But not purely on the bureaucracy of CERN, but the inescapable bureaucracy of the systems we set in place to define scientific findings. As we continue to probe regions which are increasingly removed from our physical perception of reality, what other hopes do we have?
Excuse my ramblings!