A FHIR Server Comparison using “Quantity” Searches

How well implemented are common search parameters by different FHIR server providers?

This is a question I started asking a number of months ago, when I encountered surprising failures using some managed FHIR servers.

I was looking for a structured way to compare the abilities of FHIR servers. Something I could run repeatedly and document. I started with search parameters, as this was where I first noticed differences.

The results were unexpected.

Observation is the most commonly used FHIR resource type. And quantity is one of the most commonly used value types in Observation. This seemed like a good place to start.

The Queries

I began by coming up with four test queries, each one testing a server’s ability to handle a different variation of quantity searching. I used only two search parameters: “value-quantity” and “code-value-quantity”.

Here’s the FHIR documentation for the quantity search parameter: https://hl7.org/fhir/R4/search.html#quantity

And here’s the documentation for the Observation search parameters where you’ll find the two parameters I used: https://hl7.org/fhir/R4/observation.html#search

Query 1

Give me all “body weight” Observations where the value is above 80.

Observation?code=29463-7&value-quantity=gt80

I would expect all servers to run this query without issue. But the query itself has a problem that limits its value: it doesn’t specify a unit of measure. This means it will return results where the body weight is over 80kg, over 80Ibs and even over 80 grams. This last might seem surprising but I have encountered new-born baby weights being measured in grams.

Query 2

Give me all “body weight” Observations where the value is above 80kg.

Observation?code=29463-7&value-quantity=gt80|http://unitsofmeasure.org|kg

Here I clearly specify the units of measurement as kilograms. Again, I would expect all servers to handle this without difficulty, as not doing so severely limits the ability to properly search on quantity.

Query 3

Give me all “body weight” Observations with a value of approximately 80kg.

Observation?code=29463-7&value-quantity=ap80|http://unitsofmeasure.org|kg

The “ap” search prefix is one that I do not expect all servers to manage. It should work by delivering results where the value is within 10% of the specified amount.

Query 4

Using a single composite search parameter, give me all “body weight” Observations where the value is above 80kg.

Observation?code-value-quantity=29463-7$gt80|http://unitsofmeasure.org|kg

Essentially, this is the same query as Query 2, but rolled up into a single composite or “combined” search parameter. Support for composites is sporadic but is important as some searches can only be properly written using composite parameters. Specific blood pressure queries are an example.

The Server Providers

I ran each of these queries against 8 FHIR servers from different providers, each of whom sell “out of the box” FHIR servers.

• Microsoft Azure
• Google FHIR Store
• AWS Healthlake
• Firely
• HAPI
• Aidbox
• Medplum
• Oystehr

I pre-populated 6 of these servers with synthea test data and relied on existing data already in the public Firely and HAPI servers. Locally installed Firely and HAPI servers may behave differently, so bear this in mind when interpreting results.

Quick Results

All test queries were run on January 12th, 2025, then re-run on January 18th.

OperationOutcomes & Notes

[1] invalid_query(further diagnostics: invalid numeric value ap80 for term value-quantity)
[2] The search comparator ap is not supported.
[3] Approx on Quantity is not yet implemented.
[4] Search type “quantity” with unit specified is not implemented.
[5] Query ran without error, but no results were returned.
[6] Query ran without error, but results do not match Query 2.
[7] InternalServerError. Search type of composite component wasn’t found.
[8] Unrecognized search parameter type: composite.

Results Breakdown

Query 1
As expected, all servers successfully ran this query.

Query 2
One server failed to run this query on the first pass, but succeeded a few days later once the deficiency was rectified.

Query 3
Four servers successfully ran the “approximate” query and four failed. This was not surprising as it’s not a commonly used search prefix.

Query 4
I had high hopes here, as being able to run composite search parameters on Observation is important and does say a lot about how the underlying server and database is constructed. Of the 8 servers tested, only Azure, AWS and HAPI ran the query successfully. Oystehr made some changes to their servers and successfully ran the query a few days later.

Google ran the query without error but returned no results. It successfully returned 12 Observations for Query 2. It should have returned the same 12 results here but did not

The Firely server also ran the query without error, but the results came with questions. The query returned 1,018 results, whereas Query 2 returned 1,059 results. They should be the same.

Summary

Bear in mind that for this comparison I chose only a single resource type — Observation. And I tested only two search parameters — “value-quantity” and “code-value-quantity”. A very small sample.

Despite this, the servers responded very differently. Of the eight servers tested, only three successfully ran all four test queries without issues.

• Microsoft Azure, HAPI and Oystehr

All the other servers either did not implement parts of the Quantity search, or did not handle the search parameter itself. In some cases they appeared to implement the parameter but results indicated that there were problems with the implementation.

I’m left wondering what I’d find if I tested all search parameters for all 145 resource types. An exercise for another day.

---

Download my “FHIR Architecture Decisions” book

FHIR Weekly

Join 1,100+ business and technical leaders.