The healthcare industry has spent years exploring how artificial intelligence can improve medical imaging, but most radiology AI tools have been designed to perform narrowly defined tasks. A recent seminar presentation from the Medical Imaging and Data Resource Center (MIDRC) explored a more ambitious vision: creating a radiology foundation model capable of supporting a wide range of imaging applications.

During the webinar, "Toward a Radiology Foundation Model," part of MIDRC's ongoing seminar series, moderator Maryellen Giger, Ph.D., A.N. Pritzker Distinguished Service Professor of Radiology at the University of Chicago, welcomed Curtis Langlotz, M.D., Ph.D., professor of radiology, medicine, and biomedical data science at Stanford University and one of MIDRC's principal investigators.

Langlotz discussed recent advances in self-supervised learning, synthetic data and large-scale model training that could help bring foundation models to radiology. Such models, he argued, have the potential to transform how imaging AI is developed and deployed across healthcare.

Moving Beyond Traditional Medical Imaging AI

For much of the past decade, medical imaging AI has relied heavily on supervised learning, requiring radiologists to manually label images before models can be trained.

"During 2012 to 2020, we were using labeled data, supervised learning, to train medical imaging models," Langlotz explained. While those datasets were larger than previous generations, they were limited by the cost and effort required to generate expert annotations.

Outside healthcare, AI development has shifted dramatically toward scale. Large language models such as GPT, Gemini and Claude have demonstrated that increasing data and computing power can significantly improve performance. Medicine, however, has not yet benefited from the same degree of scale.

"Because of privacy and difficulties in aggregating data from different institutions, we're still back down in the lower part of this curve," Langlotz said. "We have a lot of opportunity to use that scale."

Rather than relying on manually labeled datasets, foundation models use self-supervised learning. The approach allows algorithms to learn from massive collections of data by identifying patterns and relationships without requiring extensive human annotation.

"The reason that we can scale training datasets so large now is that we don't need labels," Langlotz noted.

Building the Foundations

At Stanford, researchers have been developing large multimodal models designed specifically for radiology.

One project, called CheXone, was trained on millions of chest X-rays, radiology reports, question-and-answer pairs, and reasoning traces. The model uses both imaging and language data to learn clinical relationships and generate interpretations.

According to Langlotz, the model has demonstrated strong performance across multiple evaluation tasks, including identifying rare diseases and supporting differential diagnosis. Researchers have also developed similar approaches for cross-sectional imaging such as CT scans and MRIs. Together, these efforts represent what Langlotz described as the foundation upon which larger radiology models can be built.

Teaching AI How Radiologists Think

One of the most intriguing aspects of the work involves capturing radiologists' reasoning processes. Langlotz highlighted a project involving more than 400 radiologists and trainees from 70 countries who interpreted over 50,000 chest X-rays. Researchers collected not only the final interpretations but also detailed "chains of thought" showing how radiologists arrived at their conclusions.

"We now have over 100,000 chains of thought reasoning traces," Langlotz said. Participants clicked on image regions as they examined them, creating a rich dataset that links visual attention with clinical reasoning.

According to Langlotz, incorporating these reasoning traces into model training has already shown promise for improving diagnostic performance. The work reflects a broader trend in AI research toward teaching models not only what experts conclude, but how they arrive at those conclusions.

Making Large Models More Efficient

While larger datasets can improve performance, they also require enormous computing resources. A significant portion of Stanford's research has therefore focused on improving efficiency.

One technique identifies redundant imaging studies and reduces their representation during training while emphasizing more unusual or clinically challenging cases. Using this approach, researchers achieved similar performance while reducing training data requirements by roughly two-thirds.

"We achieve the same accuracy as the full dataset with about one-third of the amount of data," Langlotz said.

Other projects have focused on improving contrastive learning techniques, compressing large medical images without sacrificing diagnostic information and reducing the influence of misleading correlations in training data.

For example, models sometimes learn shortcuts that can undermine clinical performance. A pneumothorax detection model may learn to recognize chest tubes rather than the untreated pneumothorax itself.

"We'd like to remove that spurious correlation from our training method," Langlotz said. Researchers are developing methods to identify and mitigate these biases before they affect downstream performance.

The Role of Synthetic Data

Another area of investigation involves synthetic medical images generated by AI.

General-purpose image generation models often struggle to produce realistic radiology images. To address that challenge, Stanford researchers retrained open-source diffusion models using large collections of chest X-rays and radiology reports.

The resulting system can generate realistic chest radiographs with specific findings, demographics and clinical characteristics. Researchers found that synthetic data alone is not sufficient.

"Training on just synthetic data really isn't nearly as good as training on real data," Langlotz explained. However, when used strategically alongside real-world data, synthetic images can improve model performance and reduce the amount of real data needed for training.

The most effective approach, researchers found, involved pretraining models on synthetic data before fine-tuning them with real clinical data.

Why Foundation Models Matter

Beyond improving performance on common imaging tasks, Langlotz believes foundation models could be especially valuable for rare diseases. Traditionally, rare disease AI development has been limited by a lack of training examples. Foundation models may help overcome that challenge by providing a stronger starting point.

"You're going to get better accuracy and you're going to require less labeled data to train that model," he said.

The concept mirrors developments in other AI domains, where large pretrained models can be adapted for specialized tasks with relatively little additional data. Langlotz suggested that radiology foundation models could eventually serve as a universal platform for developing a wide variety of imaging applications.

Looking Ahead

Stanford is now preparing to train what may become one of the largest radiology foundation models developed to date. The effort involves approximately 1.8 petabytes of imaging data spanning numerous modalities and clinical applications.

The project will incorporate the various efficiency improvements discussed during the seminar, including chain-of-thought reasoning, synthetic data, data filtering techniques and improved contrastive learning methods.

Langlotz said researchers hope to present initial results at the annual meeting of the Radiological Society of North America (RSNA) later this year. "We expect to have some results by the RSNA meeting this year in November," he said.

The first generation may initially focus on 2D imaging, followed by expansion into 3D studies. Within the next 12 to 18 months, Langlotz said the team hopes to release an open-source version for non-commercial research use.

For radiologists concerned about the technology's impact on the profession, Langlotz offered a reassuring perspective. "AI is not going to cause any problems for the radiology workforce," he said. "We have way too much work to do."

Instead, he argued, the future belongs to clinicians who learn how to work effectively with AI tools.

As he summarized near the close of the seminar: "Radiologists who use AI will replace radiologists who don't."