Medical Sciences Stack Exchange is a question and answer site for professionals in medical and allied health fields, students of those professions, related academics, and others with a sound understanding of medicine and healthcare-related sciences. It only takes a minute to sign up.
Sign up to join this community
What do I need to fulfill (requirements, certification, tests, ...) in order to get an AI-based medical application's diagnosis to be accepted as "as hard" as a professional's?
I have learned about the usage of AI in giving diagnosis, but it seems that there's still a strong belief towards human professionals making "final calls".
OTOH I've also learned about misdiagnosis rate by professionals (sometimes even up to 50%), suggesting that even professionals are not 100% accurate.
You would probably need to do a trial where you directly compare AI diagnosis to professional diagnosis in a real-world setting.
To be convincing, you'd need to consider that diagnosis in the clinic is not a simple consideration of information present at one time, but is an iterative process that often involves cycles of different tests in different conditions, attempts at treatment for simple causes, and may involve consultation with multiple professionals with different expertise.
You'd need to consider that "accuracy" can be a useless and misleading statistic. For example, you can achieve 99.999% accuracy if you simply never diagnose a rare condition that occurs in <1/100000 cases. This is neither impressive nor meaningful. There is highly variable risk in different types of misdiagnosis - in some cases, a (false-positive) misdiagnosis can be costly in the area of additional testing and treatment, yet those costs are reasonable in the face of the alternative of not making that diagnosis when it is true (false-negative). So, your machine (and your trial) needs to consider the weighted risk of some diagnoses over others.
You also will suffer from lack of a "gold standard" for many diagnoses. If there was a better method for diagnosing, it would be used in the clinic, so you may not be able to consider your estimated ground truth as an actual ground truth without making your study artificial and not particularly generalizable.
This may sound daunting. It is. You're describing an entire field of research:
https://scholar.google.com/scholar?hl=en&as_sdt=0%2C50&q=AI+diagnosis&btnG=
There is never a simple answer in research, all the simple answers and low-hanging fruit have been found very long ago. What remains are hard questions, difficult study designs, costly experiments. People spend their entire careers on these sorts of things without making substantial progress.
