Dr. Maryam Nabavi Nouri talks with Dr. Konrad Wagstyl about the MELD Project, an open-science consortium using deep learning principles to develop automated lesion detection of clinical MRI data.
Sharp Waves episodes are meant for educational purposes only, and not as medical or clinical advice.
Listen below or download the episode.
Find Sharp Waves on Spotify, Apple Podcasts, Google Podcasts, Amazon Music, iHeart Radio, and Stitcher.
Or listen to all the episodes and read transcripts on the ILAE website.
Podcast Transcript
Dr. Nabavi Nouri: We are joined today by Dr Konrad Wagstyl, one of the leads of the MELD Project, and MELD stands for Multi-centre Epilepsy Lesion Detection project. He joins us from the Wellcome Center of Neuroimaging in London, UK. This project was published last year in Epilepsia, and it really brings together a large neuroimaging cohort of patients, including MRI lesion maps and demographic, clinical, and surgical variables using open science practice.
Dr. Wagstyl: Hi, yeah, I’m Konrad. I’m a senior research fellow at the Wellcome Center in UCL in London and I work on methods to automatically analyze MRI scans, particularly in children and adults with focal cortical dysplasia. The biggest part of this role is the multicenter epilepsy lesion detection project.
Dr. Nabavi Nouri: Can you tell us more about how this project came together and some background on this topic and really why this study was done? Those are big questions but really the main idea behind the project.
Dr. Wagstyl: Sure, absolutely. We work on focal cortical dysplasia (FCD), which is the leading cause of drug resistant epilepsy in children and the/a leading cause in adults. The aim of the projects is to, so these lesions are very difficult to diagnose on an MRI can, but if you can diagnose them, you can offer surgery, and this can be a cure for patients and seizures.
The co-lead of the project, who is Dr. Sophie Adler, was doing her PhD at Great Ormond Street Hospital where they see a lot of these patients, and I was doing my PhD at Cambridge University, and we decided this was an ideal candidate for these computation tools we’d been developing to identify these lesions automatically. So we identified a small cohort from Great Ormond Street of patients with FCDs and created software that did a reasonable job of auto identifying them. This was now 5 or 7 years ago, and the key step I think, the real novelty we introduced was sharing all our code. So sites around the world started to use the code themselves, got back in touch with us saying how it was working, and based on all of this positive feedback we decided it’d be a great idea to join these sites together and create this multicenter project.
Our aim was to create a really large neuroimaging cohort of patients with FCD and together that made this large MRI cohort with all this rich clinical data and that’s where we are today with the multicenter epilepsy lesion detection project, MELD as we call it.
So we have two publications. The first one is atlasing. So we have this rich dataset and we really thought it was our responsibility to make the most of it – learn things about FCD from this large cohort that couldn’t’ be found with smaller data. And the second paper, which is just out in Brain, is this automatic lesion detection tool, which we offer as an open tool for sites around the world to use.
Dr. Nabavi Nouri: This is fantastic work, and it must be the work of many, as I see in this paper, many ideas and a lot of data. How was the data compiled and also addressing the difficulties that come with the heterogeneity of the data of this kind and source and magnitude?
Dr. Wagstyl: That’s a great question. I think it’s important to say the MELD project, we have 22 sites, 70 or 80 collaborators, and they are all really fantastic – everyone has put a lot of work into this, because they think the problem is important. They give us time creating the data set, feedback on all the work that we’re doing, and help writing the manuscripts and sharing the work, and I think it wouldn’t be possible without all that buy in from around the world. Each site processes the data locally, the MRI data, and makes it as anonymous as we could do. We only receive these relatively de-identified features rather than the raw MRI scans, and this is what we get sent. We then have to go about harmonizing it, as you said. So every site has a different scanner, different sequences, and it’s a really big challenge. It’s a challenge not just for epilepsy but for all of imaging. So again we have to reach out to collaborators who are experts in this harmonization, and we’ve developed tools for bringing the MRIs together, making them look like they came from the same scanner, so we can treat it as a single very large cohort of patients to do our analysis.
Dr. Nabavi Nouri: If you could tell us a little bit about the masking, the masking technique that was used, and then would each center identify the FCD, or all they all being masked and transferred to your center and being analyzed there?
Dr. Wagstyl: It’s a key thing we think about now is how to get consistent masking, so in the current study radiologists or expert neurologists at each site would do the masking on the MRI scan in 3D, and that would be sent to us, and this is what we use for our analyses. As a separate small side project within our own site, we got 3 very experienced radiologists to mask the same 10 patients, as we have 3 examples of different people masking them. They of course all found the lesion in roughly the same place, but they’re not the same masks. So everything we then find, you have to bear in mind there’s human variability in how these are being masked. It’s kind of an unavoidable issue but motivates the need for these automated tools to get objective diagnoses of where the abnormality is rather than these intervariable, the radiologist-dependent masks that we’re currently dependent on.
What’s really nice is where you look at the three maskers look and you look at where this new MELD algorithm identified, it agrees very nicely with the three maskers, as if it was a fourth masker, so there’s a bit of variability there.
Dr. Nabavi Nouri: Can you tell us about the main findings of the study?
Dr. Wagstyl: Great. So this first atlases of lesion location study, we put together all these lesion masks, we have 580 lesion masks, and the first thing we wanted to look at is where these lesions are located. Because FCDS happen in the cortex and in theory they can happen anywhere, and that’s what we find – you do find them everywhere in the cortex. But there’s a really striking non-uniform pattern. Certain parts of the brain were much more affected than others. So in the temporal pole, superior frontal sulcus and frontal pole, we found many more lesions, and this is particularly true for the FCD IIs. This is a striking finding and also very informative. If someone has a suspicion of the lesion and they think it’s in the superior frontal sulcus, that’s quite likely to be the case, so that seems like a very useful tool.
The second interesting finding that I would like to focus on is age of epilepsy onset. You can ask whether having a lesion in a particular brain area is associated with having an older or younger age of onset. We found lesions in the occipital cortex or the sensory cortex tended to have an earlier age of onset.
And then I think the last one that I think is particularly of clinical relevance was that we looked at whether where your lesion is impacts the likelihood of seizure freedom. Lesions that are around eloquent cortex – visual cortex, motor cortex, and the language areas, were associated with much lower likelihoods of seizure freedom. What we think here is that because the lesion is next to eloquent cortex the surgeons are being deliberately cautious—you don’t want to take out eloquent cortex—and therefore there’s an increased likelihood that some of the lesion is left behind, or some of the epileptogenic tissue is left behind ad continues to cause seizures.
Dr. Nabavi Nouri: I know you had included pediatrics and adults – I wonder if this seizure freedom rates differed in pediatrics versus adult patients, because sometimes in the pediatric populations the surgeons, because of the neuroplasticity in age, might be a bit more so to say generous with the resection, compared to adults where they have less room for plasticity.
Dr. Wagstyl: We didn’t look directly at that, but one other interesting thing that we did see is common with some of the other findings in the literature – there is a small effect of duration. So if you have the surgery earlier, younger, you’re more likely to have good seizure freedom.
Dr. Nabavi Nouri: What were the main implications of the findings, or the results, as you’d like to put it?
Dr. Wagstyl: So I think the first thing is that one thing we see is there’s quite a large duration of epilepsy in our cohort. Typically around 10 years. Given the current view that longer duration is less good for outcome and less good for cognitive outcomes, there’s a motivation to move things earlier. One way we can go about doing that is by getting diagnoses earlier. FCD is really difficult to diagnose, and one reason for this delay may be the delay in being diagnosed on an MRI scan and being sent to a surgical center. So one of the key takeaways for us was tools for automated diagnosis at first presentation could be beneficial in reducing the duration of epilepsy.
The second one is this outcome relating to eloquent cortex. Again we think that this really motivates the development of automated tools for identifying exactly how large the lesion is. Because providing that to the surgeons can make a much better surgical plan, a much more data driven decision about whether you’re going to leave some of that lesion, rather than doing it by accident because you’re worried about leaving eloquent cortex.
And the last thing is this lesion map, which is I think a really good guide for, if you can’t find the lesion, look harder in these particular areas.
Dr. Nabavi Nouri: Was there a specific focus on patients that had been identified as MRI negative and what were the detection rates using your method specifically in patients that were, clinically went through with the surgery having had a negative MRI.
Dr. Wagstyl: That’s a great question because the MRI negative is one of the key groups we’re looking to help diagnose. So these are patients whose scan is reviewed by a radiologist and at some point on first viewing they can’t see an abnormality, so they call it MRI negative. Roughly a third of our patients are MRI negative, so that’s kind of in keeping with the general background literature.
This was for us the really interesting group to test our algorithm on. So we trained an algorithm based on these MRI features to learn what lesions look like and predict them on new patients. We’re picking up about 70% of all lesions in general, but for MRI negative patients we’re picking up 62%. So that seems to us like a really big group we could be helping with. These are patients who were missed at some point – if you’d had this report of where the algorithm thinks the lesion is, you could have a second look at that area and say actually there’s something subtle going on there.
Dr. Nabavi Nouri: That’s wonderful. They pose the biggest challenge as you pointed out, patients who are MRI negative.
You said the tool is available for everybody to use. My question is how can people with clinical background with no computational background be using this tool, these are sort of fast paced, day to day physicians that also may not have a lot of time to spare reviewing images. So how can they incorporate it in their clinical practice, given that it can be very impactful and change outcomes.
Dr. Wagstyl: Key to everything we do in the MELD project is open science. We’ve made all of our code available at our GitHub – a website where you can download the code. We have video workshops of how to run the code on your own machine. Clinicians at some hospitals have downloaded it and started using it. In addition, we are working with some partner hospitals to run the algorithm for them. So at Great Ormond Street, if they suspect an FCD, they tell us that this is, uh, likely candidate. This is also true at Queen Square, so these are 2 hospitals at UCL we’re working with. They send us the MRI scan, we run it and we put it back in. I think making this scalable so that anyone who wants to run this on their local MRIs is a real challenge for us. This is what we’re actively working with people to improve.
As you say, this is mostly software that requires a little bit of computer expertise to use. The key thing to say is that if you’re interested in running this on your patients, at your hospital, please get in touch with us. We’re keen to get people set up with this. We’ve seen it helping. We’re delighted when a site, a hospital sends us an example of when they run it and it works – that’s kind of why we did all of this. So do get in touch with us and we’d love to help set you up running this on your own patients.
Dr. Nabavi Nouri: Thank you. So and then I want to ask if you can briefly discuss the strengths and limitations of the study.
Dr. Wagstyl: Yes. So one of the things that this multicenter approach enabled us to do is see a whole variety of patients, different histological subtypes, patients scanned on different scanners, different ages, got lots of children and adults in the study from around the world, and all of this heterogeneity strengthens the algorithm – seeing 600 patients, it’s much more likely to understand what it’ll look like in the 601st patient. I think that was a key development for us, because showing it works at one hospital is one thing, but showing it works in 20 is a completely different challenge.
At the moment, there’s always ways to improve algorithms and this is definitely something we’re working on. I’d say the biggest drawback at the moment is that we still find false positives. Even if you put normal patients through the scanner, maybe 40 % of them the algorithm will think something looks a bit funny. So it’s definitely not ready for use without someone checking and confirming with other modalities. To that end, we’re still working on the MELD data set to improve the algorithm and we’re starting at second MELD study to get a much larger data set with much more diverse histologies, so including other causes of focal epilepsy, and the aim here is to make a much more robust tool – new version, better performance, better reliability.
Dr. Nabavi Nouri: For the younger investigators that are listening to our podcast and are interested in the work that you’re doing, how can they be involved? And this could really either involve physicians that are on the receiving end, where they want to use your tool to help their patients, or whether these are investigators that would love to be involved in more of the research aspect of this.
Dr. Wagstyl: So I think there’s three ways we’re really moving forward with this, and this impacts how people can get involved. The first is to figure out how and where it can be used in a clinical scenario. So we have a clinal trial using it as targeting for SEEGs, we’re attempting to use it as being presented in an MDT study, and we are encouraging sites around the world to use it locally and give us feedback on it and how it is to use. All of this will help us understand where it’s best and in which scenarios it’s used best.
So if you’re interested in running the code, it’d be incredibly helpful for us to have feedback on that. That’s one really valuable step. The second is that we’re running a second study now – MELD focal epilepsies, and we’re at this stage inviting sites to join. This includes, definitely includes young researchers who would like to be part of the multicenter study, are interested in open science. As I said all our code is openly available, we run collaborative workshops so it’s a great way of kind of getting involved in a large-scale study with lots of senior researchers but also people who are new and energized. We value all the contributions that we receive. If you’re interested in joining, please email us at MELD.study@gmail.com – this will be either if you’re interested in the technical aspects of lesion detection or if you have one, two, doesn’t matter how small your patient cohort is, please get int ouch we’d love to make our cohort as large and diverse as possible.
We run teleconferences roughly every 2 to 3 months. We don’t want to burn people with too many meetings but those are open to anyone interested in the MELD projects. Those are completely open, so we do share what we’re working on at the time, but it’s also for getting a feel of the project, asking what it takes to become involved or just assessing whether you’d like to join.
Dr. Nabavi Nouri: Physicians – if they run the code, you still, from my understanding, is that your team would still like to review the results, given the fact that the algorithm is still being perfected and there are some false positives?
Dr. Wagstyl: That’s a really good point to make and I definitely want to emphasize it’s not a licensed diagnostic device, it’s still what we call a research tool. So it’s definitely to be used with the expertise of a radiologist and a clinical team. From our point of view, we don’t have control over who’s using it. You can download it; you can use it as you like. What we’d really like to hear is how you’re finding using it. Getting us feedback. “It’s working really well for these patients,” or even we just like seeing a screenshot every so often of saying, “This was a child with MRI negative epilepsy and look, this really helped.” That really makes our day.
Dr. Nabavi Nouri: I can imagine. And as we conclude, I’d like to hear sort of your final points that you’d like to make – anything we didn’t talk about that you’d like to point out, essentially take-home messages for our listeners.
Dr. Wagstyl: I think I’d like to say that we’re really grateful to the whole epilepsy community for kind of embracing MELD and this open science approach. We have 22 sites who are giving up their research time or clinical time to collect the data, share it with us, share their expertise, and give us advice. We’re all making this tool together and everyone has the motivation of providing patients with better diagnostic tools. And I think that’s a really nice atmosphere within the epilepsy research community that has enabled this to happen.
RESOURCES
Twitter: @meld_project
Website: MELD Project
Publications:
Atlas of lesion locations and postsurgical seizure freedom in focal cortical dysplasia: A MELD study (2021, Epilepsia)
Interpretable surface-based detection of focal cortical dysplasias: a Multi-centre Epilepsy Lesion Detection study (2022, Brain)
Networks Underlie Temporal Onset of Dysplasia-Related Epilepsy: A MELD Study (2022, Annals of Neurology)
##
Contact us with feedback or episode ideas at podcast@ilae.org.
Founded in 1909, the International League Against Epilepsy (ILAE) is a global organization with more than 125 national chapters.
Through promoting research, education and training to improve the diagnosis, treatment and prevention of the disease, ILAE is working toward a world where no person’s life is limited by epilepsy.
Website | Facebook | Instagram | YouTube
Twitter: English French Japanese Portuguese Spanish
