Computers still aren’t close to replacing doctors for diagnosing disease

From record store clerks to phone operators and taxi drivers, computers are swiftly proving they can do many jobs at least as well as — or better than — humans.

And there’s certainly reason to think that machines could one day overtake humans when it comes to diagnosing disease. Doctors, after all, get medical diagnoses wrong about 15 percent of the time. And the price of those misdiagnoses is human suffering and death.

But just how soon will computers replace flesh-and-blood MDs?

In the first such head-to-head study, researchers from Harvard Medical School and Brigham and Women’s Hospital in Boston decided to put humans and machines to the test, comparing the diagnostic accuracy of physicians with that of 23 different medical diagnostic software tools.

These tools, known as symptom checkers, can be found on apps or sites like WebMD; they allow users to type in their various symptoms and answer a series of questions, and then spit out a list of “probable” diagnoses generated by computer algorithms.

In the study, both the doctors and machines were given “clinical vignettes” — or made-up stories about patients — and asked to make three possible diagnoses and rank them.

“Physicians vastly outperformed computer algorithms in diagnostic accuracy,” the researchers wrote. While doctors listed the correct diagnosis in their top three picks 84 percent of the time, the computers did the same only a little more than half of the time. In other words, humans made a correct diagnosis more than twice as often as machines, and the computer algorithms basically performed only slightly better than a coin toss.

This may seem like a win for human doctors, but it doesn’t mean that computers couldn’t one day outperform them.
“While the computer programs were clearly inferior to physicians in terms of diagnostic accuracy, it will be critical to study future generations of computer programs that may be more accurate,” said senior investigator Ateev Mehrotra, an associate professor of health care policy at Harvard Medical School, in a press release. As computer algorithms get more and more accurate, Mehrotra suggests, doctors better watch out.