Healthy Skepticism

We can become so enamored of new technology in medicine that we sometimes forget it needs to be tested to prove it can accomplish what it seems to promise.

Recently, for example, many health-care systems have embraced a range of remote telemonitoring products to track the health of patients at home. Some patients use such monitoring apps and gadgets on their own initiative (see “The Patient of the Future”). It seems logical that more data for doctors and patients is a good thing. In medicine, however, what matters is whether the innovation helps people live longer, with better health and quality of life. We have not yet assessed whether health-tracking apps actually do this.

In reality, few new technologies are subjected to rigorous evaluation. Unlike new drugs, most can legally be put into practice without formal testing. Moreover, the testing that is done is often carried out by the people who developed the technology, so study designs are susceptible to subtle and often unintended bias.

With colleagues, I recently published the results of a large NIH-funded randomized trial of a highly touted telemonitoring system for patients with heart failure. Patients called an automated phone system every day to answer a series of questions about their symptoms. Certain responses triggered alerts to doctors. It seemed like a simple, effective technology destined to transform medicine and was promoted to hospitals as highly effective. Unfortunately, our study failed to find any benefit. These results were confirmed within months by another large independent evaluation of a similar product.

The lesson: we cannot assume on the basis of mere common sense that a promising technology will work. We need to have our assumptions confirmed by independent studies. This principle applies to a host of medical technologies, including other self-monitoring and reporting tools and electronic health records.

There are many reasons why a technology might not realize its promise. The trickiest have to do with implementation: even a technology that works perfectly and is capable of great things may be used in a way that makes for little or no impact on patients.

Medical technologies are not as simple as pills that a patient takes or does not take. Existing practices may not be able to accommodate a novel approach. Information generated by monitoring systems may be misused, misunderstood, or ignored. For whatever reason, “can’t miss” technologies often do.

Developers of new technologies need to prototype and test with those complexities in mind. At the same time, independent tests of medical technologies need to become faster, cheaper, and more routine. Ultimately, the promotion and implementation of technologies like home monitoring must be based on the results that matter most: the extent to which we have made the patient feel better and live longer.

Harlan Krumholz is the Harold H. Hines Jr. professor of medicine, epidemiology, and public health at the Yale University School of Medicine.