Harmonizing Patient Medical Encounters against MRI Phantoms -- Simulating Patient Medical Continuum?

[TheCedarPrince]

Hi folks,

Apologies for the rather lengthy title but I have been tinkering on an idea for the past two or so years and just got all the conceptual pieces in place for the idea to take action. As @cncastillo and folks in JuliaHealth here most likely are aware, I tend to work within health economics and the observational health research space. Historically, it has been rather onerous to simulate observational health data (i.e. electronic health records and patient medical claims) at a massive scale as well as convert it into a computable format. In fact, it has mostly been impossible to do so.

As of today, that has changed thanks to OHDSI/dbt-synthea#91 (comment). I was able to, for the first time ever, generate 1 million synthetic but statistically accurate patients using the Synthea data generator with 1 year patient look back and using dbt-synthea, convert that observational health data into a common computable format being the OMOP Common Data Model. Synthea simulates, for a wide array of diseases, patient encounters with healthcare systems within the USA. These encounters range from visiting a clinic (outpatient, inpatient) and drug events (prescriptions, refills, etc.) to diagnoses and procedures (biopsies, radiology procedures, ambulatory evaluations).

To my understanding, if my conceptual model is now correct here, I will be able to:

• Generate a patient's interaction with a healthcare provider
• Per each MRI procedure, simulate a phantom using KomaMRI.jl
• Harmonize an entire patient's medical history with phantoms simulated using KomaMRI.jl

All for a variety of diseases supported by Synthea (such as Alzheimer's, Depression, PTSD, and more). Patient populations could even have simulated comorbidities such as Type 2 Diabetes and other conditions alongside neurobiological/neuropsychological conditions.

I want to know from you all, given what I propose, does the above now seem possible to harmonize a simulated patient's medical history alongside simulated MRI readings for given patient populations? Resulting in datasets that can more wholly describe a patient?

I wanted to write this down to make sure I had the idea down as well as being a bit fueled by excitement. What makes me even more excited is that Synthea generates health economics information as well as social determinants of health data per patient. It feels like there may be something here as not even PhysioNet Databases seem to have something like what I am describing above.

Thanks folks!

~ tcp 🌳

P.S. Weren't you all also exploring cardiac MRI simulations as well?
P.P.S. Linking this issue for reference: OHDSI/dbt-synthea#91

[aTrotier]

Hi,
Let me rephrase to see if I understand what you want to do at the MRI levels.

You have a simulated "patient" with specific morbidities, you want to generate the MR datasets in that case.
For example, you need to simulate one patient with Alzheimer's and you need to create images corresponding to what will be done by the radiologies.

Protocol example :

Alzheimer :

• T1-weighted (3D MPRAGE or SPGR or IR-FSPGR)
  High-resolution anatomical imaging
  Detects medial temporal atrophy (especially hippocampal volume loss)
  Used for automated volumetric analysis (e.g., FreeSurfer, NeuroQuant)
• T2-weighted
  Identifies vascular changes, white matter hyperintensities (WMHs), and other non-Alzheimer pathologies
• FLAIR (Fluid-Attenuated Inversion Recovery)
  Detects white matter hyperintensities (WMHs), which may indicate vascular contributions to cognitive decline

Stroke

• Diffusion-Weighted Imaging (DWI)
  Most sensitive for acute ischemic stroke (detects within minutes)
  Shows cytotoxic edema (bright on DWI, dark on ADC map)
  Helps differentiate acute vs. chronic infarcts

• Apparent Diffusion Coefficient (ADC) Map
  Confirms DWI findings; acute infarcts appear dark on ADC
  Chronic infarcts show normalization or increased ADC

• T2-weighted / Susceptibility-Weighted Imaging (SWI)
  Detects hemorrhage, microbleeds, and thrombus (e.g., susceptibility vessel sign in MCA occlusion)

• Fluid-Attenuated Inversion Recovery (FLAIR)
  Helps age the stroke
  Hyperintense infarct on DWI but isointense on FLAIR = hyperacute (<6 hours)
  Hyperintense on both DWI & FLAIR = >6 hours

• Time-of-Flight (TOF) MR Angiography (MRA)
  Evaluates large vessel occlusion (LVO) (MCA, ICA, basilar artery)
  Identifies stenosis, dissection, or embolic occlusions

---

On principles KomaMRI can be used to simulate that but you will confronted to multiple problems :

• It is computationnaly intensive to generate real acquisitions (generally we used low-resolution, mono-slice, 2D etc)
• The "phantom" used by KomaMRI is an healthy patient with standard relaxometry parameters.
• Some of the physical effects are not simulated like Time Of Flight

[TheCedarPrince]

Hey @aTrotier, thanks so much for taking the time to respond here! Let me share some follow-up thoughts:

Looking at your protocol example, that is basically spot on to what I was imagining. Put another way, I was envisioning defining a patient population of interest (could be based on a particular disease, symptoms, etc.), generate the continuum of care for this patient population using the synthetic data generator I have, and then use KomaMRI.jl to simulate MRI data for prescribed MRI procedures within these patients.

On principles KomaMRI can be used to simulate that but you will confronted to multiple problems :

Ah OK, I wasn't aware of these limitations. To each of your points:

• Regarding computational complexity, I am not too worried at the moment on this personally as I do have compute available beyond my personal computer that could hopefully handle simulation -- would have to ultimately test at some point.
• Ah! Thanks for describing more about the "phantom".
• For now, I am not too worried about every single physical effect being simulated but this is great to know.

---

In short, thank you again for taking the time to share thoughts. As you can gauge, I am not much of a medical imaging expert but am trying to get a sense of what could be possible. I appreciate you letting me bounce ideas around here. 😄

[cncastillo]

The "phantom" used by KomaMRI is an healthy patient with standard relaxometry parameters.

Yeah I agree. I am not completely sure how you would use the simulated medical data to get a phantom that could later be used to simulate MRI acquisitions. It could be an interesting application of generative AI.

It is computationnaly intensive to generate real acquisitions (generally we used low-resolution, mono-slice, 2D etc)

I wanted to say that @rkierulf made some efforts in making simulations scale better with the number of spins, which is relevant for 3D acquisitions: #537. These preliminary results already show simulations performance improvements of 10x (or more) for large phantoms.

Moreover, implementing a EPG-like simulation method could be of great help. I haven't done it yet because I wanted to have a fairly optimized Bloch simulator, which I believe we are very close to.

Some of the physical effects are not simulated like Time Of Flight

@pvillacorta has simulated ToF, diffusion, flow, etc. Obviously these simulations are more costly, but it could be done. Some preliminary results (not including PR's 537 improvements):

ToF:

Phase contrast:

Tagging:

We can also use Distributed.jl to run distributed simulations:

So I believe we could do some cool stuff, even for 3D, but not sure how to get the "phantom" definition.

[aTrotier]

Whoo impressive works :)

Moreover, implementing a EPG-like simulation method could be of great help. I haven't done it yet because I wanted to have a fairly optimized Bloch simulator, which I believe we are very close to

Yeah, it will be particularly efficient (especially for gradient echo images which requires a lof of supsampling), we should definitely try to make an efficient GPU compatbile) standalone EPG package (ideally with the possibility to use AD like https://github.com/MagneticResonanceImaging/EPGsim.jl) that Koma should rely on.

---

About the initial topic

At the end, what do you expect to do with the simulated images ?

It could be an interesting application of generative AI.

I think it will be easier to implement something like that -> using MR scans (T1 weighted, T2 flair etc) of patient databases (from https://openneuro.org/ or https://osf.io/ ...) annotated with comorbidities and other metadata (age / sex) and train generative AI to create a specific scan for the input.
Alternatively, you could also start a collaboration with physicien to do this prospectively in order to get as much metadata as you can.

Take a look at https://doi.org/10.1007/978-3-031-53767-7_13 it seems they generate multi-contrast tumor MR images.

[TheCedarPrince]

Hey @cncastillo and @aTrotier,

Thanks for taking the time to share thoughts here! Picking up on this question:

At the end, what do you expect to do with the simulated images ?

So, ideally what I want to do is to simulate, as best as possible, the full patient continuum of care for patients afflicted with neurological or mental disorders that exhibit a prodromal period (schizophrenia, alzheimer's, etc.). As there are many psychosocial factors at play in the development of such disorders, having the entire continuum of care available for patients from a health systems point of view, as well as a social determinants of health perspective, would be beneficial alongside actual findings from MRI procedures. The hope I had would be to investigate potential underlying latent variables across the entire spectrum of a patient's care continuum that could give further insight into the development of prodromal periods and potential early detection markers.

patient databases (from openneuro.org or osf.io ...) annotated with comorbidities and other metadata (age / sex) and train generative AI to create a specific scan for the input.

You are absolutely right that these databases would have images I could use. The only problem I had with these kinds of datasets is that they are not as rich as what I would want to investigate in terms of a patient's medical history -- rather, the metadata you suggest is quite coarse and rudimentary in my experience. Now, that is an interesting idea however; I could simulate patient medical histories alongside actual patient images based off these coarse metadata information. In effect, cutting out any direct simulation from KomaMRI and instead use data from these sources as the images. The more I think about it, this might actually be a really fruitful path to take (with having to make some significant assumptions).

The generative approach might be an interesting approach too -- would need to tinker further with potentially harmonizing against datasets from OpenNeuro or OSF. Don't have too much of an intuition about how this would work just yet, but

Alternatively, you could also start a collaboration with physicien to do this prospectively in order to get as much metadata as you can.

Yes I could but am sadly not in a position to do that at the moment -- thus the curiosity of looking at simulations.

[aTrotier]

You can also take a look at thus AI tools :
https://monai.io/model-zoo.html

some of them can be used to generate images