FHIR TL;DW: Using LLMs to Process Longitudinal Health Records

In our FHIR TL;DW (Too Long; Didn’t Watch) series, we share bite-sized summaries and insights from key HL7 FHIR DevDays presentations.

How can large language models (LLMs) be used to process longitudinal health records, and what use cases provide the most value? In his DevDays talk, “Garbage In, Good Out? The Potential of Ingesting Multimodal Health Records Directly into LLMs”, Google’s Will Rosenfeld explored what’s possible now and what’s on the horizon. 

I’ve outlined key insights from the talk below.

Finding the best solution: Deterministic, generative, and hybrid approaches 

Processing longitudinal health records—a collection of a patient’s health data over time—is a complex task. The question isn’t just how to process this data, but how to do so effectively in ways that benefit both clinicians and patients. 

In his talk, Will Rosenfeld highlights three primary approaches for using LLMs in healthcare: 

• Deterministic: Involves upfront costs for capturing and structuring data, making it highly reliable for system-to-system use cases like data exchange.
• Generative: Skips the preprocessing phase, focusing instead on creating responses during queries. While flexible, this approach comes with risks like latency and hallucination.
• Hybrid: Combines the strengths of deterministic and generative approaches to balance reliability and adaptability. 

Google argues that deterministic methods are still the gold standard for most use cases, particularly those requiring high accuracy. However, generative approaches still have their place, offering new possibilities where deterministic methods fall short. 

LLM use cases in healthcare 

Google identifies four key use cases for LLM prompting: 

1. Search: Simple search functions for locating information in medical records. 
2. Summarize: Creating encounter summaries or key points from patient data.
3. Synthesize: Populating forms or templates based on patient records. 
4. Multi-Turn: Engaging in interactive conversations with electronic health records (EHRs) to retrieve and contextualize data. 

Key takeaways 

1. Today, LLMs are most effective for “professionals-in-the-loop” scenarios, where clinicians remain actively involved in the process to minimize risks like hallucination and inaccuracies. It’s currently most suitable for proof-of-concept implementations of the Synthesize and Multi-Turn use cases, because these are much more difficult to accomplish using deterministic approaches.
2. For FHIR implementers, Rosenfeld emphasized the importance of including all available data in FHIR resources, even if it’s not immediately needed for interoperability. This could include text files, PDFs, or legacy formats. Such data might become valuable in the future as LLM-driven use cases evolve. 

Learn more from Firely

At Firely, we’re committed to sharing best practices and insights for everything FHIR. For more in-depth insights and discussions on navigating FHIR challenges, explore our blog and FHIR training courses.

Interested in more talks like this? Join us and HL7 at the next FHIR DevDays, the largest FHIR-only event in the world where you can learn all about FHIR and hone your FHIR expertise. There’s plenty of content for both developers and non-coders, for FHIR starters and FHIR experts alike.

To watch the full length video of Will Rosenfeld’s talk, you can find that here. For all videos from FHIR DevDays 2024, head over to the Devdays 2024 playlist on Youtube.