A year ago I tested Claude and ChatGPT side-by-side to see how well they could generate FHIR data. Results were mixed.
Today I tried again with a lighter prompt, leaving them room to improvise.
Here’s the prompt.
I asked for a complete transaction Bundle for a patient admitted for a day procedure. I did not specify any resource types. Everything was left up to them.
The output differed more than I expected.
Claude returned 22 resources across 17 resource types.
ChatGPT returned 61 resources across 23 resource types.
Claude left out Coverage, AllergyIntolerance, Questionnaire, QuestionnaireResponse, ClinicalImpression and Medication. All reasonable resources for the scenario I presented.
I specified a Dublin hospital on purpose, to see if each would introduce financial resource types that are not used in Irish healthcare workflows. Both passed this test and excluded CoverageEligibilityRequest and CoverageEligibilityResponse, used by US payers.
Then I posted each Bundle to an Aidbox FHIR server – chosen because Aidbox enforces referential integrity and rejects any Bundle that contains incorrect resource references.
Both failed on the first attempt.
ChatGPT added an Organization to the Appointment.participant.actor element, which is not allowed.
Claude failed to add a policy or policyRule element to the Consent resource. Neither is mandatory, but one of them must be there – easy to miss.
Both Bundles were otherwise strong, with most elements populated, but Claude performed far better when it came to coded values. ChatGPT used text values for medications and even for the procedure itself. Claude went looking and sometimes found the correct SNOMED or LOINC codes.
Both worked out that a laparoscopic hernia repair uses mesh and created a Device resource to capture it.
And both produced a Bundle that a FHIR server rejected until a human stepped in.
ChatGPT’s Bundle: https://vanyalabs.com/files/ChatGPT-Bundle.json
Claude’s Bundle: https://vanyalabs.com/files/Claude-Bundle.json
---
Download my “FHIR Architecture Decisions” book
FHIR Weekly
Join 1,100+ business and technical leaders.