Language models should write your FHIR mappings, not run them 

Let’s say you want to use FHIR (Fast Healthcare Interoperability Resources), but your data isn’t in FHIR. You’ll need to transform it. And to transform it, you’ll need a data mapping.

It’s tempting to use an LLM (Large Language Model). Give it your data and get back FHIR. Unfortunately, LLMs may hallucinate and they may not always give consistent answers. This doesn’t scale efficiently for large data sets, and it can be expensive, too.

To get the best of both worlds, we can ask an LLM to write a structured data mapping that will always give consistent, explainable answers.

And we can verify that the mappings are correct using rules about FHIR data, which are also written in FHIR data. I’ll talk about how to create data mappings this way using a test-driven approach and other best practices.

Structure data mappings 

Structured data mappings, whether written in the FHIR Mapping Language or a general-purpose programming language, are deterministic and explainable. They’re also slow to write, especially when you’re dealing with a data model or rules you’ve never seen before. The feedback loop is long: write some mapping code, run a transformation, look at the output, find a problem, go back.

LLMs can help. They’re fast at bridging unfamiliar formats, translating natural language into code, and adapting when requirements shift. But they’re also usually non-deterministic, expensive to use for transforming a lot of data, and often confidently wrong. Especially for specific FHIR data rules found deep within StructureDefinitions and ValueSets, rather than in data the LLM was trained on. Asking an LLM to remember the cardinality constraints on a FHIR ExplanationOfBenefit profile is exactly what you should not use an LLM for.

A hybrid approach is the answer: LLM as the author of the mapping, and the mapping as the source for data transformation. You get the flexibility of an LLM where it’s helpful (authoring, iteration, interpretation), and the reliability of deterministic code where it’s critical (accuracy, production transformation).

How do you know a mapping is correct? 

Transformed data produced by a mapping should be both valid and preserve the original meaning of the data. So, we need transformation tests.

First: Is the transformed data valid? Does it obey FHIR’s data rules? If a constraint says a Patient can have at most one birth date, and you’ve transformed an OMOP Person into a FHIR Patient, that Patient had better conform. Correctness there is specified by the data model. At minimum, the transformed data should be valid: it should correspond to its FHIR data model — a StructureDefinition or profile.  

Second: Does the transformed data preserve the meaning of the original data? An OMOP gender code 8532 should look like a FHIR code for “female”. Examples like this serve as a partial specification of what a correct mapping is. This part is subtle: examples of expected, transformed data don’t give you a mapping.

They help define what mapping should do. Either a human or an LLM can create examples of what the expected data should look like. Ideally, the example data should be valid, anyway. The goal is to make an implicit, idea of a mapping into an explicit one that can run transformations. 

What do you need for the mapping process? 

We want to create data mappings that can be tested, updated based on the test results, tested again, then repeat. Data models and example data for both FHIR and your non-FHIR data are needed to start this process.

First: Example input data means a fully anonymous sample, like a CPCDS CSV row or an OMOP Person record or whatever your source data looks like, and a corresponding expected output in the target format, like FHIR JSON. This is what the LLM will test against. It doesn’t need to be extensive. One good example per resource type is enough to start the feedback loop. 

Example output data could be generated by an LLM, but this should only be done once to bootstrap the process. Again, you should check that it’s valid and preserves the right meaning. 

Second: Data models are what mappings are made of. A mapping like “All OMOP Person records are FHIR Patient records” describes whole sets of data, not individual records. Data models give an LLM context: what models the mappings will be written in and what rules are encoded in those models. For FHIR, this means models like StructureDefinitions and ValueSets, allowed codes. LLMs should not attempt to “remember” these but instead simply look them up. 

The mapping loop 

The process is simple: 

1. Run the transformation tests 

2. Update the mapping based on the results 

3. Repeat until tests pass 

The transformation tests should ensure that the actual, transformed data is valid using a deterministic validator. The tests should also ideally match the actual, transformed output data to the expected output data. If the expected output data is already valid, and the actual output matches the expected, then you know it’s valid, too. 

Validating FHIR data means checking that the data conforms to the rules encoded in a data model like a FHIR StructureDefinition and/or ValueSet, which, conveniently, are just other kinds of FHIR data.

The prompt needed for this should be simple, like: “Create a mapping between these two data models. Run the transformation tests. Update the mappings based on the test results. Repeat until the tests pass.” FHIR data models provide most of the context needed, so we want to minimize the amount of speculation an LLM engages in. 

Contrary to popular belief, we don’t always want a large prompt (or “skill”). We want deterministic tests to do most of the work. And we don’t always want a large amount of context, either: LLM instruction-following generally decreases when context increases. The key here is to prevent the LLM from guessing what the correct mapping is. Use the right tools for the right job. LLMs are made for translating languages. Data models are made for describing data and enforcing rules about data.

What could go wrong? 

The most common mistake is context overload. An LLM that has been given too much information that isn’t relevant to a particular mapping, like an entire FHIR Implementation Guide, is going to get lost in the details. Instead, use “progressive disclosure”: the LLM agent should be able to find and expand context as needed, rather than being given everything up-front. Aim for quality over quantity. You wouldn’t trust yourself to remember specific details about some data model. Why would you trust an LLM to do the same?

Another mistake is too much “reasoning” in the sense of an LLM writing an internal monologue to itself and expanding its own context. Usually this improves the quality of answers. But in our case, the data models and deterministic mappings and tests should provide almost all of the answers needed. Stop confidently guessing and just run the tests.  

Finally, you might ask: “Won’t this whole approach become obsolete as super-genius LLMs become smarter?” But this confuses the problems we’re trying to solve. We want to know exactly how and why our data is transformed by using deterministic, structured data mappings. We want a fast, run-time transformation in the form of a deduction, not a probabilistic prediction.

A smarter LLM could write mappings faster or more accurately. But the beauty of FHIR data models is that they already give almost all the  answers needed to create mappings. Those answers simply need to be looked up and applied by translating between data models — which are really just different languages.

Bonus: Can we use local LLMs? 

The best part about this architecture is that it enables smaller LLMs to write data mappings. That means LLMs running on a computer or server you have access to, instead of in a massive data center run by a large corporation. The benefits compound over time: Privacy, sustainability, efficiency, and affordability. And you don’t need a genius to carefully read and apply data rules.  

Language models are already “black boxes”: they predict each, next piece of language based on the entire, previous context of words or code. Running a proprietary language model on someone else’s computer makes this situation even more opaque. What extra, hidden processes and dependencies are involved? These are nice things to know for sane engineering, human or not.

Getting started 

If you want to try this, the setup is simple: 

• A test harness that runs transformations and checks output against expected data 
• One or two example source/target pairs per FHIR resource type  
• A large language model — perhaps even a less large, local model 
• A simple prompt that tells the agent where to write the mappings, how to run the tests, and how to update the mappings until the tests pass 
• A mapping language like C# with the Firely .NET SDK