What do I need to fulfill (requirements, certification, tests, ...) in order to get an AI-based medical application's diagnosis to be accepted as "as hard" as a professional's?

I have learned about the usage of AI in giving diagnosis, but it seems that there's still a strong belief towards human professionals making "final calls".

OTOH I've also learned about misdiagnosis rate by professionals (sometimes even up to 50%), suggesting that even professionals are not 100% accurate.

  • It's not realistic to envision one person doing this. You have years of work, millions of dollars, and the whole regulatory approval process ahead of you, which is going to require you to hire a number of people who know how to go about all this.
    – Carey Gregory
    Dec 22, 2021 at 15:23

1 Answer 1


You would probably need to do a trial where you directly compare AI diagnosis to professional diagnosis in a real-world setting.

To be convincing, you'd need to consider that diagnosis in the clinic is not a simple consideration of information present at one time, but is an iterative process that often involves cycles of different tests in different conditions, attempts at treatment for simple causes, and may involve consultation with multiple professionals with different expertise.

You'd need to consider that "accuracy" can be a useless and misleading statistic. For example, you can achieve 99.999% accuracy if you simply never diagnose a rare condition that occurs in <1/100000 cases. This is neither impressive nor meaningful. There is highly variable risk in different types of misdiagnosis - in some cases, a (false-positive) misdiagnosis can be costly in the area of additional testing and treatment, yet those costs are reasonable in the face of the alternative of not making that diagnosis when it is true (false-negative). So, your machine (and your trial) needs to consider the weighted risk of some diagnoses over others.

You also will suffer from lack of a "gold standard" for many diagnoses. If there was a better method for diagnosing, it would be used in the clinic, so you may not be able to consider your estimated ground truth as an actual ground truth without making your study artificial and not particularly generalizable.

This may sound daunting. It is. You're describing an entire field of research:


There is never a simple answer in research, all the simple answers and low-hanging fruit have been found very long ago. What remains are hard questions, difficult study designs, costly experiments. People spend their entire careers on these sorts of things without making substantial progress.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.