By Combining Neuroimaging and AI, Researchers Aim To Save Military Lives

TROY, N.Y — Triage care on the battlefield requires split-second decision-making and proficiency in providing first aid. Correctly applying a tourniquet or inserting an intubation tube before a patient is evacuated to a hospital could save lives. To improve training for the field medics responsible for this level of care, researchers at Rensselaer Polytechnic Institute will team up with researchers from the United States Army Research Laboratory in Orlando, Florida, to better understand skill acquisition and to standardize the prolonged field care (PFC) certification process.

With the support of a $3.1 million grant from the U.S. Army Combat Capabilities Development Command Soldier Center, the team will combine neuroimaging, computer vision, eye-tracking, and artificial intelligence methods to make PFC certification faster, more objective, and scalable for all soldiers.

“All of these are highly challenging problems,” said Suvranu De, the co-director of the Center for Modeling and Simulation in Medicine (CeMSIM) at Rensselaer. “If we can make it work, this would be a game-changer for the Army.”

De and his fellow CeMSIM co-director, Xavier Intes, are co-principal investigators on the grant. Their first step will be to create an AI-enabled system for automatically and accurately evaluating performance. The current certification process relies on a human scorer and is, therefore, more subjective and requires a significant amount of personnel time.

The team will determine what the gold standard of PFC mastery looks like, providing a benchmark against which all field medics and soldiers can be objectively scored. To develop this performance scale, the research team will gather multimodal information from participants as they complete simulation-based training. Data that includes hand motions captured by cameras, eye movement captured by gaze tracking, and brain activity captured by neuroimaging will be integrated into AI models that automatically classify performance.

You can watch De discuss this research program here.

“We’re going to see, if we couple all the information in deep learning environments, can we look at a learner and predict whether or not that person is learning?” De said. “This is going to be a completely automated system. It will provide a ‘go/no-go’ gauge for whether they are allowed to go into the battlefield or not.”

The research program will also investigate whether skill acquisition can be accelerated and retained through the use of non-invasive neuromodulation.

“For psychomotor tasks, there’s a lot of knowledge that we’ve gathered about the brain for the last few decades, but that research always requires very simple scenarios,” Intes said. “This will be very challenging because there is a lot of activity in the brain that is still not known. It’s still at the cutting edge of what we are trying.”

The collaboration between De and Intes exemplifies The New Polytechnic, the multidisciplinary and innovative model for research at Rensselaer. De’s expertise in medical simulation, surgical skill assessment, and patient safety is critically supported by Intes’ expertise in bioimaging and artificial intelligence deep neural networks. This pair recently won a $2.2 million grant from the U.S. Army Medical Research and Development Command of the U.S. Department of Defense, received through a Medical Technology Enterprise Consortium award, to combine neuroimaging, neuromodulation, and artificial intelligence to better understand and measure surgical skill acquisition.

The Rensselaer team will also include Qiang Ji, a professor of electrical, computer, and systems engineering, Uwe Kruger, a professor of biomedical engineering, and Rahul, a senior research scientist in CeMSIM. The Rensselaer team will collaborate with a team of human factors experts, neuroscientists, and clinicians at the University at Buffalo’s Jacobs School of Medicine and Biomedical Sciences, including Dr. Steven Schwaitzberg, chair of the Department of Surgery, Lora Cavuoto, an associate professor of industrial and systems engineering, and Anirban Dutta, a research associate professor of biomedical engineering and surgery.

What researchers uncover at the intersection of neuroscience and artificial intelligence could lay the foundation for further brain research, as well as improved outcomes for patients — military and civilian alike.

About Rensselaer Polytechnic Institute

Founded in 1824, Rensselaer Polytechnic Institute is America’s first technological research university. Rensselaer encompasses five schools, 32 research centers, more than 145 academic programs, and a dynamic community made up of more than 7,600 students and over 100,000 living alumni. Rensselaer faculty and alumni include more than 145 National Academy members, six members of the National Inventors Hall of Fame, six National Medal of Technology winners, five National Medal of Science winners, and a Nobel Prize winner in Physics. With nearly 200 years of experience advancing scientific and technological knowledge, Rensselaer remains focused on addressing global challenges with a spirit of ingenuity and collaboration. To learn more, please visit www.rpi.edu.