Letter from the Editor
/
“ChatGPT crushes real doctors in answering patient questions”
I often read and enjoy Vinay Prasad’s “Sensible Medicine”, and this was the title of his recent summary of an article that appeared in the Journal of the American Medical Association which described the results of a “man vs machine” study comparing the quality of the answers to patient questions provided by physicians vs ChatGPT. The machine “won”.
In the news and throughout the social media we find ourselves suddenly beset by an ever-increasing clamor over the potential good versus malevolence of “artificial intelligence” (AI). Not that this clamor -this debate - is inappropriate; anyone who doubts AI will convey a tremendous change in our culture, economy and very behavior need only consider the staggering societal impact of the Internet-connected “smart phone”.
But no matter whether you perceive AI to be the salvation of mankind or its destruction, it seems intuitive that even the most sophisticated AI program will require ongoing input generated from human experience and creativity in order to avoid eventually exhausting the supply of what it can so efficiently store, analyze, retrieve and regurgitate. But what exactly does this have to do with migraine?
To perform effectively as a neurologist one must become adept at data input, analysis and processing. A good neurologist learns how to ask the appropriate questions, to prioritize the answers received (separating the wheat from the chaff) and, combining these verbal data with what can be gleaned from the physical examination, then to provide an accurate diagnosis. This is no less true for migraine, cluster and other primary headache disorders than it is for stroke, multiple sclerosis, and Parkinson’s disease.
Equally important to the effective practice of clinical neurology is acquiring the art of observation. Repetitive observation leads to pattern recognition, a key ingredient to maximizing the likelihood of accurate diagnosis, and with experience comes improved pattern recognition. The very best clinicians amongst us approach each new patient as a new source of data and, an opportunity to improve pattern recognition…and thus diagnostic acumen. For example, while no two migraineurs are identical, when it comes to clinical presentation, there are relatively few variations on the migraine theme amongst the many millions afflicted by the disorder. That said, outliers do exist, and if the pattern of the clinical presentation does not “fit” with a diagnosis of migraine based on one’s skill at using pattern recognition, it may be time to consider another cause for the patient’s symptoms.
My new headache clinic patients who provide a very typical history for migraine and whose physical/neurologic examinations are normal often ask how I can make the diagnosis of migraine and propose a treatment strategy without first obtaining a brain CT or MRI scan. The simple answer: in that clinical setting – given those data/that pattern - the yield of brain imaging is essentially nil. By the process of informed pattern recognition they have migraine, and to reflexively obtain a brain MRI scan costing thousands of dollars is pointless, wasteful and even potentially harmful. On the other hand, if something is not quite right - if there is something in the history or on exam that does not conform to the pattern of migraine - it may be that brain imaging is very much indicated.
Data input and data analysis blended with pattern recognition to produce accurate and useful output – sounds tailor-made for AI. With continued new data input might a “machine” (AI) achieve such a nuanced level of pattern recognition that it could duplicate or exceed the performance of an experienced headache medicine clinician? Perhaps at the quantitative level, but after having spent these past few decades observing in many thousands of patients the sometimes subtle qualitative differences that distinguish one patient from another, in this particular case my money is on man…and not the machine.
