AI in Healthcare: Adopting the Bias We Aimed to Eliminate
Written by Louisa Kirk
Edited by Matthew Lane
In recent years, workplaces have begun to integrate Artificial Intelligence tools into workplace practices to streamline tasks and be more efficient. One major workplace application of AI is healthcare: a field where AI can help analyze medical images, predict outcomes, and complete repetitive tasks.
Despite what may seem like a recent surge in AI technology in media and workplaces, AI has been present in healthcare fields, in some capacity, for decades. In the 1960s, researchers began exploring its medical applications, and by the 1970s, Stanford scientists had developed software to help diagnose bacterial infections [1]. Today, Artificial Intelligence and healthcare companies have cooperated to expand these applications. The CEO of Microsoft Satya Nadella said that “AI is perhaps the most transformational technology of our time, and healthcare is perhaps AI’s most pressing application” [2].
Radiology, in particular, has become one of the largest areas of AI research in medicine. In a global review of more than 7,000 clinical AI articles, radiology was the most represented specialty (by about 40%) [7]. Given this, radiology is an important place to examine the potential drawbacks of AI in medicine.
Medical AI models are trained using existing patient data, in a process called machine learning. As of 2018, 63% of AI companies in the United States used machine learning, which analyzes data to identify patterns and make predictions about new data such as medical images [1]. In healthcare, traditional machine learning uses collections of previous patient data to inform care decisions. In radiology, these collections include large databases full of X-rays, CT scans, MRIs, and mammograms. The AI model picks up on patterns within these images to make predictions about new patient scans.
However, this is where bias can arise: the data that manufactures feed the AI is not always representative of the patient population as a whole. A review of published AI articles between 2015 and 2019 found that imaging AI data from the United States came primarily from only California, Massachusetts, and New York, and that AI databases entirely omitted data from 34 states [8]. As a result, many datasets do not have adequate breadth of information about race, ethnicity, insurance, or other ‘human’ factors that radiologists account for when analyzing scans.
AI can also “go rouge” by learning patterns manufacturers do not want it to apply to patient scans. For example, in 2018, researchers tested pneumonia-detection AI using 158,323 chest X-rays from the NIH Clinical Center, Mount Sinai Hospital, and Indiana University. In the study, researchers found that the AI model could almost perfectly identify which hospital an X-ray came from, with accuracy of 99.95% for NIH images and 99.98% for Mount Sinai images [9].
This matters because pneumonia rates differed significantly between the hospitals. At Mount Sinai, 34.2% of the X-rays in the dataset were of patients with pneumonia and only about 1% of X-rays at both NIH and Indiana University showed pneumonia. That caused the AI model to predict pneumonia based on which hospital it came from rather than making diagnoses based only on signs of pneumonia. This is known as shortcut learning: the algorithm relies on patterns in its training data rather than the actual disease characteristics researchers want it to identify.
Despite its apparent drawbacks, Artificial Intelligence has strong potential in radiology. Notably in South Korea, researchers found that AI diagnosed breast cancer 12% more accurately than human radiologists [6]. Findings like these illustrate the advantages AI can offer to medicine. However, there is no question that the bias regarding patient population and harmful shortcuts need to be addressed before AI continues to become wholly implemented in hospitals.
Louisa Kirk ‘30 is in the College of Human Ecology. She can be reached at ljk222@cornell.edu.
