That's the title of a very interesting article at Science News, arguing that bad statistics is the dirty secret of science. I believe it is.
Though there is an entire field of physics called "statistical mechanics", the statistics there don't go beyond the 19th century. To this day, physics undergraduate and graduate programmes cover statistics minimally, or not at all. Perhaps it seems unimportant to theorists, but it is crucially important in testing hypotheses, which is what experiments claim to do. Or perhaps hypotheses in physics are sufficiently clear-cut, and experimental data sufficiently clean, that sophisticated hypothesis testing is not necessary.
In other fields, hypotheses are murky and plentiful, data are noisy and ambiguous, but the practitioners are still ignorant of statistics. When the field is medicine, and the question is of new drugs or therapies, it is a crucial matter.
Famously, Sir Roy Meadow -- creator of the discredited "Munchhausen syndrome by proxy" hypothesis -- sent several mothers to jail with his expert evidence based on bogus statistics. The consequences of bad statistics may not always be equally bad, but if the medical literature is as riddled with them as recent articles suggest, the cumulative effect may be worse than anything Meadow did.
But bad statistics in the medical literature is just the starting point: there are problems throughout the practice of standard medicine. This is why, though I respect mainstream medicine and regard much "alternative medicine" as fraudulent and the rest as of very limited (and unvalidated) applicability, I was sufficiently annoyed by this post by Orac to leave this comment. (See also other comments there on dubious practices in the health industry.) I think Orac does, in theory, a great service by pointing out peddlers of pseudoscience and exposing their ignorance and, often, fraudulence. In practice, he preaches to the converted and, I suspect, antagonises nearly everyone else.
3 comments:
You will be pleased to note that a basic course on Probability and Statistics is part of the Core that is done by all students at IISER.
This at least introduces them to the possibility of methods affecting the outcome of a measurement (Bertrand's paradox). Whether they will recognise it when they see it during their research is another story; after all even statisticians and probability theorists can sometimes be fooled!
I remember reading up a bit on Munchausen by proxy a long while ago when there was a spam flood for "rescue Aditya Chandran" on the web and from that site and the best of my memory it sounded a bit iffy by itself; and especially doubtful applied in that case which sounded like a bad divorce going hellishly wrong for the lady concerned.
thanks,
Jai
Kapil -- I am currently reading Jaynes' "Probability theory: the logic of science", where he basically trashes "orthodox statistics" (it's an unsystematic collection of ad-hockisms, a hodgepodge of the feuding schools of Fisher and Neyman-Pearson, according to him) and argues that physicists and astronomers have been better off not using statistical hypothesis tests because their own intuition is much better at assessing significance. He argues strongly for Bayesian methods as outlined by him; the methods go back decades and he gives huge credit to Harold Jeffreys and his 1939 book on probability and statistics.
Today the worst of the polemics between the "frequentist" and "Bayesian" camps are over and Bayesian methods are widely accepted because they work, but I suspect the deeper points that Jaynes makes are still not widely accepted, perhaps because so many people would be personally hurt by it. For example, he says the chi-square test is often wrong by orders of magnitude because it makes incorrect assumptions about the tails of normally-distributed data. There are several other examples in each chapter.
I am very far from finishing the book, much less absorbing it, but will blog about it one of these days.
Jai -- yes, that case seemed extremely odd, and like you, it seemed to me that the lady was being victimised.
Post a Comment