Where quality measurement stands after twenty years

For most of the last twenty years, implementing a quality measure meant waiting for a specification to arrive as a document written in English and then having people translate it into a proprietary database language, like SQL, which for an update to your HEDIS or CMS measures could take six to nine months.

Then you ran it, and since it was unlikely you got what you were looking for, you massaged the data and went around again until the numbers looked plausible, without ever knowing whether your result matched the next vendor, the facility next door or the other payer.

I have been working on quality measure implementation for twenty years this year, and I think we’ve made it.

How do you know the answer is right?

That is the question that actually changed. Implementers now sit together in connectathons and in the Digital Quality Implementers Community (DQIC) with their engines running side by side, unit testing the little stuff, does five plus five equal ten, then taking huge patient decks from HEDIS and CMS and checking that all the measures line up.

No one is asserting parity. We are testing for it, and what we are testing toward is the same data, same measures and value sets, same outcome every time, which we are now very close to.

The pieces that make that possible are real and in the open. Clinical Quality Language (CQL) has been published since 2015, digital HEDIS exists with its measures written in CQL rather than inside an 1,100 page document, and the open source .NET CQL engine we maintain with NCQA is tested against HEDIS and CMS measures.

HEDIS certification is set to be FHIR only by 2030, and in June CMS said it will take FHIR-based measures this year, prefer them next year and only accept FHIR for new measures the year after.

This doesn’t mean the standards work is finished; CQL 2.0 is in ballot and the quality data floor is still being profiled and vetted. However, the work has shifted: measure logic can now be expressed, shared, executed, and tested without every organization translating the specification into its own implementation by hand.

What we look at instead

So much has shifted away from the specification that the questions I spend my time on now are different ones. Is FHIR data is rich enough? Does it contain everything the measure needs? An entire patient history is a different problem from a test deck at a connectathon. We spent years stuck on validation, and being stuck on analytics instead is a much better problem to have.

The underlying problem the community keeps hitting is coding, since roughly half of the clinical FHIR resources in real-world data still use local or proprietary value sets, and a measure can be perfectly specified and deterministically executed and still return nonsense if the data below it is coded in a private vocabulary.

What is encouraging is what happens when measures meet production data, and breast cancer screening results turning up in DiagnosticReport when the measure logic went looking in Observation is the example everyone cites now, because it got found, it got ticketed and the specification was updated in two or three months rather than sitting in a queue for years.

The output format is a big one

QRDA (Quality Reporting Document Architecture) is going away, and it is worth being fair to it on the way out. It was a structured document you built per measure to report to CMS, and while it had its problems (especially when it was asked to do interoperability as well), it did the job for a long time.

What replaces it is a FHIR MeasureReport, and the important part is that it is not specifically a quality reporting document. A measure report can be built for many reasons and shaped for whatever program or workflow needs it.

That is what I mean when I say this was never a reporting pipeline, it is just a query system, which is why prior authorization is already using it and care gaps run on it. With a proprietary database you could maybe run a measure overnight or next week, too late to matter, whereas now you could run a longitudinal measure against one patient at the click of a button and inform the provider what else to do before that visit ends. It’s pretty awesome, honestly.

What’s still unsolved

Exclusions, first. Today we often ask providers to enter more data to “prove” that they didn’t do something, and I don’t like that. The conversation that needs to happen is not at the FHIR level yet, it is about what data capture looks like in the first place.

I remember coding things where the answer from the floor was “we’re never going to do that” and “use this data instead”, which was not in the spirit of the spec.

Hopefully this eventually changes. Maybe sometimes 87% is the right answer for now, because we did not do something clinically, and we do not need another field in the record explaining why. That should continue to be a clinical and measurement-design conversation, not something we solve by piling more documentation onto the provider.

Second, and this is the awkward part of my own argument, it can feel like a black box, since it sits in the server and the engine and just happens, which is a fair reaction from teams who used to read every line themselves. The answer is not asking anyone to take it on faith, it is education and good debugging tools.

Where this goes next

HL7 is forming a FHIR Quality Improvement Accelerator, and they picked the word improvement on purpose, not measurement, because the ambition is to change care while it’s happening, not just report on it afterward.

Meanwhile, the Digital Quality Implementers Community keeps growing, and it’s telling that some of the newer members came to it from prior authorization rather than quality. That is the same infrastructure doing more than one job.

After twenty years, the hard part is no longer proving the logic can run. The hard part is data quality, modernizing measure specifications, and bringing together enough organizations and providers to work through these problems together.

Our digital quality measures engine, Firely dQM, runs on Firely Server and lets you measure continuously through the year instead of once at the end of it, so if you want to talk through what your first digital measure would involve, get in touch.