MESSAGE FROM THE MCMS PRESIDENT
KELLY R. O'KEEFE, MD President, Monterey County Medical Society
Beads on a Wire By Kelly R. O’Keefe, MD President, Monterey County Medical Society Recent events across a range of disciplines have rekindled a long-standing interest of mine in uncertainty. Not the uncertainty of Heisenberg that operates most obviously at quantum scales, but the everyday uncertainty that plagues so many of our daily decisions. Bear with me, while I share some of the events, and then what they have to do with beads on a wire. Two of the events are from deeply basic sciences: cosmology and particle physics. The first event was the announcement of the discovery of a new elementary particle in the summer of 2012 that was subsequently confirmed to have the characteristics of the previously theorized Higgs boson. The second was the announcement, a few days ago, of the discovery of inflationary gravitational waves in the Cosmic Background Radiation. As important as they now seem, each of these discoveries may turn out to be wrong. We have a pretty good estimate of how likely that is. Here is a quote from the abstract related to the former, “This observation, which has a significance of 5.9 standard deviations … is compatible with the production and decay of the Standard Model Higgs boson.” And from the later, “We find an excess of B-mode power… inconsistent with the null hypothesis at a significance of > 5σ.” In medicine, we may have had such seminal discoveries announced recently, but in any case, we have had many high-impact activities implemented that sometimes affect patients by the dissemination of new medical knowledge about diagnosis and treatment, and other times by delivery system changes. What does the medical literature have to say about each of these mechanisms? As I noted a couple of months ago, PubMed is adding almost two-thirds of a million new citations each year, many of them describing new diagnostic or therapeutic approaches, and many of those including statistics that we use to estimate the likelihood that the results are “real.” A p-value <0.05 is often treated as if it separates important from unimportant. Further, the care approaches added to our knowledge base by these papers underpin our technical approach to the patient. John P. A. Ioannidis, MD, C. F. Rehnborg Professor in Disease Prevention in the School of Medicine at the Stanford University School of Medicine, is concerned about whether our reliance on published studies may lead us astray. The title of one of his published papers doesn’t pull any punches, “Why Most Published Research Findings Are False.” Mayo Clinic Proceedings recently described medical reversal, where new trials contradict current practice, “A decade of reversal: an analysis of 146 contradicted medical practices.” Vinay Prasad, MD, and Adam Cifu, MD, state in an abstract from a paper on medical reversal, “Famous examples include the class 1C anti-arrhythmics post-myocardial infarction (contradicted by the CAST trial) or routine stenting for stable coronary disease (contradicted by the COURAGE trial),” and note, “… importantly, it creates a loss of faith in the medical system by physicians and patients.” Also, along the lines of loss of faith in the medical system are recent
findings related to delivery system changes. Some of these findings are from the so-called Oregon Medicaid Experiment, where a Medicaid lottery allowed a randomized, controlled experiment of insuring previously uninsured adults. While this study found some positive effects, it also found that emergency department use by the insured increased 40% in comparison to the control group, health expenditures increased, and overall physical health was not improved. These findings were in stark contrast to those expected from the change in insured status. A recent New England Journal of Medicine article dashed a similar dose of cold water on the patient-centered medical home concept with an article entitled, “Almost No Benefit of Medical Home Interventions in Community-based Practices.” I’m not quite sure if this is a medical reversal or the initial, uncontrolled, study, but these types of findings are especially troubling because system delivery changes affecting millions of people and costing billions of dollars are routinely implemented with even less evidence of likely positive effects than practices subject to medical reversals. About 35 years ago, I read an essay by British astronomer R. A. Lyttleton in The Encyclopedia of Ignorance in which he proposed a mental model to be adopted in evaluating any hypothesis, which includes all of the above situations. He suggested that one think of their belief in a hypothesis as a wire with a bead on it, with one end of the wire representing certainty that the hypothesis is true (1) and the other end representing certainty that the hypothesis is false (0). His advice is, “Never let your bead ever quite reach the position 0 or 1.” He says that is when the bead falls into the emotional pit of pride and prejudice and it cannot be recovered. Our Bayesian friends would put it more formally and quantitatively, but I take their point to be the same. When we become too certain of our ideas, we fall into a frame of mind that does not do justice to the fundamental uncertainties of our ability to understand the world. Somehow, it seems to me that that degree of skepticism would be very helpful in avoiding some of the errors that clinging too tightly to positions informed by p<0.05 evidence, or worse yet, no evidence at all, brings us. Our physicist colleagues seem to be able to make do with waiting for p<0.0000003 results to declare adequate certainty. Although human variability and a need to bring diagnoses and treatments to patients in a timely fashion, undoubtedly, make waiting for five sigma significance unrealistic, could we strike a better balance? Is this part of the meaning of, “First of all, do no harm.” That seems to me to be a sensible interpretation. But I have to admit, I am not certain.
Kelly R. O’Keefe, MD, is the 2013-2014 president of the Monterey County Medical Society. He is a board certified pathologist and is currently CEO of Adaptive Clinical Solutions, Inc. MARCH/APRIL 2014 | THE BULLETIN | 5