Scientistic Interpretations of Medical Evidence: How Patients Lose Options

Scientism: where a scientist places too much faith in quantitative evidence to the point of being unable to recognize its limitations.

Many articles describe existing evidence on how to enhance adaptation to PAP therapy. Authors frequently recommend the application of heated humidifiers, more coaching by phone or in-person appointments, and pressure relief systems to resolve expiratory pressure intolerance. A more detailed analysis of some of these steps often suggests the results are mixed at best. For example, many studies have compared CPAP devices to either auto-adjusting CPAP (APAP) or dual pressure devices (BPAP) but have not consistently shown benefits of great magnitude. In fact, if we were to ask many sleep specialists whether or not they frequently consider APAP or BPAP options in their patients, it is my understanding that many sleep physicians do not employ these options. If asked the rationale for rejecting these options, I surmise they would declare evidence does not reliably show sufficient benefit to offer this alternative to a patient who may be failing CPAP. 

In research terminology, when something meets statistical significance standards it only means one thing: the experiment conducted appears to be reliable in its finding. Thus, if a research protocol testing CPAP vs APAP or BPAP does not manifest a statistically significant difference (i.e., the research produces essentially the same results for either type of PAP mode), then it suggests there is no benefit to prescribing one device over another. “Mixed results” of course means that perhaps a few studies showed some significant advantage with APAP or BPAP, but overall, across the full breadth of studies the most common finding would be no advantage to one PAP mode over another.

With this backdrop, I would like you to imagine a situation where a sleep doctor recommends a patient to continue using CPAP even though he or she is repeatedly failing it. The doctor’s advice is to work with CPAP more gradually or with lower starting pressures, until such time as the individual adapts to the device and uses consistently throughout the night every night. Is this point of view accurate? Is it based on sound scientific evidence? Is it in line with best practices?

Most physicians who read the scientific literature would probably embrace affirmative responses to all three questions. In this post, I want to clarify how sometimes the acceptance of this point of view is more accurately called a scientistic perspective (as opposed to scientific)—a term with a few contradictory definitions. The definition to be used here is the pejorative one in which the scientist is placing too much faith in science to a point of being unable to recognize its limitations. The reason a scientistic perspective is problematic in clinical care is that it may prevent a physician, literally, from using common sense to help a patient overcome a specific barrier in the attempt to use a PAP device.

Let’s look at the example of heated humidification because it’s the easiest one to explain the problem of scientistic thinking. Suppose a research study conducted in Miami Beach discovers patients do not benefit from heated humidifiers. You and the researcher would presume the high humidity in Florida (~70% humidity) obviated the need for heated humidification instead of the use of pass over humidification (simply adding cold water to the PAP chamber without heating it), and this assumption is probably correct. Now, suppose no one published a paper on the use of heated humidifiers in Albuquerque, where the air is much drier (~30% humidity), should the prescribing physician in New Mexico base his or her judgment on the paper published in Florida (scientific evidence) or go with common sense? Indeed, in New Mexico, nearly all sleep medicine specialists prescribe heated humidifiers and were probably doing so long before others in the field caught onto this innovative technology. 

Why did this local custom rapidly emerge on the medical landscape in our region? Because everyone knows our climate is incredibly dry and that patients constantly complain of dry mouth while using PAP. Therefore, pragmatic clinical experience and common sense dictated that nearly all sleep doctors in NM learn more rapidly about the benefits of heated humidification, even before any scientific evidence was published on this specific topic. Were these doctors acting scientifically? Yes, in terms of relying on anecdotal cases and realizing the obvious benefit. But, this level of evidence (anecdotal) is extremely low in the eyes of the scientific community. However the information was incredibly accurate with respect to New Mexico.

Should NM docs have waited to use heated humidifiers until investigators researched the problem and published their findings to prove the point? Or, was it reasonable to move forward and prescribe heated humidifiers? This example is a great one, because it is so obvious that physicians responded to an obvious local barrier to PAP therapy adaptation instead of sticking with a scientistic viewpoint that would insist the issue has not been appropriately studied. By the way, my memory on the topic is that there were a very small number of sleep physicians who chose not to move in this direction until much later than those of us who were reacting to anecdotal evidence.

Thinking about this example will help you understand more ambiguous cases where circumstances and particulars may seem more cloudy than clear. As such, the following examples tend to involve the conundrum of mixed results in the scientific literature. And, therefore, it would be important to find out:

  • Why the results were mixed?
  • Was the research not designed properly?
  • Were the wrong types of patients selected for the research?
  • What was the motivation of the research team who conducted the study?

Returning to the first example of when and how a physician decides to switch the patient from CPAP to another device, it is not unusual for the sleep doctor to give scant attention to this possibility at all. The doctor may have examined several studies on this topic and come away with mixed feelings matching the mixed results. But, where this perspective descends into scientistic thinking is the failure to ask the questions about who was actually studied in the research.

What if a CPAP attempter suffers from claustrophobia, anxiety, or PTSD—the doctor should ask: were any studies testing samples of patients with these conditions to observe how they fared with CPAP vs APAP or BPAP? In general, there are rare if any randomized controlled studies (highest level of evidence) on this topic. Therefore, the application of the scientistic perspective would promote this message: “there’s no research to prove a benefit from APAP or BPAP, so why switch?” Yet common sense should dictate that someone with anxiety is probably going to show more difficulty breathing out against pressurized air coming in. You might be surprised how many sleep physicians do not employ this common sense approach to lead them towards consideration of a dual pressure device with expiratory relief.

In related examples, what if a CPAP user also suffers from insomnia? Again, there are few if any randomized controlled trials looking at single pressure (CPAP) vs dual pressure (BPAP) systems in the treatment of patients with co-morbid insomnia and sleep apnea. If the sleep physician chooses the scientistic pathway, then the opinion emerges that without sufficient evidence there is no clear reason to deviate from CPAP. Common sense could have entered the equation, and a discussion with the insomniac might have generated a testable, anecdotal hypothesis: if the problem with CPAP is one of discomfort, is there a device that might prove more comfortable?

The last example involves weight. Surprisingly, many sleep professionals in our field do not recognize or otherwise attend to the fact that an enormous proportion of sleep apnea patients are not overweight. Many in fact are underweight. Yet, nearly every single study that ever looked at single vs dual pressure PAP modes only examined overweight patients, usually with mean BMIs above 30 (obesity) or above 35 (morbid obesity). But, what if a CPAP patient who reports difficulty adapting is not overweight? Are there distinct features in a thin, sleep apnea patient that create barriers to PAP use and would necessitate a switch to another device such as BPAP? Could the thin sleep apnea patient prove less tolerant of pressurized air in general; could this patient be more prone to insomnia? If there is no consistent research or few studies in general on thin sleep apnea patients, the doctor would be wise to employ a critical decision-making process when confronted with someone intolerant of CPAP. 

Two final points. First, bear in mind in many research studies showing mixed results, there is usually a proportion of patients who reported benefits by trying out something different such as APAP or BPAP, but because the sample as a whole did not reach statistical significance on improved outcomes, the findings are said be mixed or inconclusive. Yet, nothing was inconclusive for the people in the study who reported benefits from APAP and BPAP. In all likelihood, the new device was not providing them with a placebo response. Rather, the new device was providing some critical element to their specific case that was highly relevant to those who responded favorably, but the research study may not have been evaluating the factors that would have explained these enhanced results. The design of a research protocol may misdirect physicians into thinking a certain advanced technology will not be effective (scientistic thinking) instead of combing through the types of patients studied to discern who and why some people benefit.

Finally, from this type of discussion, you might imagine that I am advocating greater use of anecdotal evidence, which on the one hand is arguably correct, but on the other hand is simply indicative of my belief in a personalized sleep medicine approach. As a result of this approach instead of a scientistic process, we have been able to observe that many complex patients do indeed respond better to dual pressure symptoms. We have seen no less than a thousand patients since 2005, seeking second opinions while on CPAP, who eventually reported a greater capacity to use PAP therapy with a BPAP or other dual pressure devices.

This thought process often reminds me of Loren Eiseley’s famous story, “The Star Thrower:”

A man was walking on the beach one day and noticed a boy who was reaching down, picking up a starfish and throwing it in the ocean. As he approached, he called out, “Hello! What are you doing?” The boy looked up and said, “I’m throwing starfish into the ocean”. “Why are you throwing starfish into the ocean?” asked the man. “The tide stranded them. If I don’t throw them in the water before the sun comes up, they’ll die” came the answer. “Surely you realize that there are miles of beach, and thousands of starfish. You’ll never throw them all back, there are too many. You can’t possibly make a difference.” The boy listened politely, then picked up another starfish. As he threw it back into the sea, he said, “It made a difference for that one.”

Barry Krakow MD


Dr Krakow’s 27 years of sleep research have focused on the complex relationship between physiological and psychological sleep disorders. Dr Krakow currently operates private sleep medical center, Maimonides Sleep Arts & Sciences, Ltd., and serves as Classic SleepCare’s paid Medical Director.

Leave a comment

Comments have to be approved before showing up.