Innovative model improves Army human-agent teaming

ADELPHI, Md. — Army researchers developed a novel computational model for gathering cognitive data that may be a game changer in the fields of neuroscience and econometrics, and has broad relevance to networked and multi-agent systems.

At the U.S. Army Combat Capabilities Development Command’s Army Research Laboratory and the University of Maryland, College Park, researchers developed what is known as the Autoregressive Linear Mixture, or ALM, a novel model for analyzing time-series data, or how things change over time.

For the Army, this comes in to play to assess the cognitive states of Soldiers to allow an intelligent adaptive system to balance the workload of the crew when completing challenging critical missions.

Employing recent advancements in optimization for nonconvex problems, the researchers adapted a proximal gradient algorithm and validated the proposed model and algorithm on an open-source electroencephalography, or EEG, dataset.

This work, recently featured in

IEEE Xplore

, innovates two fundamental models in signal processing, dictionary learning and multivariate autoregressive, or MVAR, models.

“These models are so popular due to their simplicity and explanatory power,” said CCDC ARL researcher Dr. Addison Bohannon. “As generative models for data, dictionary learning and MVAR models allow researchers and engineers to encode their understanding of the underlying physical process into a concise model. Accordingly, they find applications in neuroscience, economics and image processing, where we have a rich understanding of the underlying science. However, the simplicity of these models also limits their effectiveness in describing complex, real-world signals like those that will be observed outside of laboratory experiments that the Army will require.”

On the other hand, he said, complex models do not necessarily lead to greater insight, as they pose problems of reliability and interpretability. This work grapples with these issues head-on.

The researchers hypothesize that cognitive processes can be composed of independent neurological processes that can be observed in EEG recordings. Although they do not directly test this hypothesis in the paper, their validation results show that they can reliably identify independent neurological processes in EEG recordings of individuals while sleeping.

“Our results suggest that the EEG activity associated with different sleep stages can be composed of different combinations of these independent neurological processes,” Bohannon said. “That is to say that these neurological processes are fundamental in some way. Moreover, these fundamental processes that we discover correspond to known neurological phenomena used in sleep stage analysis.”

This work is inspired by the incredibly successful thread of research on functional connectivity in neuroscience.

Functional connectivity explains cognitive processes in the brain through functional relationships (activity correlation) as opposed to structural relationships (connective tracts).

The first step in estimating functional connectivity often requires fitting an MVAR model to observed data, Bohannon said. By various means, researchers can derive networks from the estimated parameters of the MVAR model.

Through experimentation, he said, these networks can be associated with various cognitive processes. However, this process implicitly assumes that a single brain network contributes to a cognitive process since the MVAR model only models a single generative process for the data.

“The ALM innovates on this approach by explicitly modeling data as the output of simultaneous and independent processes,” Bohannon said. “Each of these processes can be interpreted as separate functional brain networks. Through the combination of these separate functional brain networks, we get high-level cognitive functions.Although this contribution is primarily to propose a new analysis technique, it could enable fundamental discovery in network neuroscience. In fact, we do find evidence for multiple functional networks coming together to yield different sleep states in our validation results.”

According to Bohannon, this effort will support the Next Generation Combat Vehicle via the Human-Autonomy Teaming, or HAT, Essential Research Program.

One project under this effort is currently developing techniques for estimating cognitive states of Soldiers from biophysical and other sensors.

“This project supports the broader vision to provide an adaptive and interactive crew station where autonomy within the crew station is able to adapt to the state of the crew,” Bohannon said. “The cognitive state of the individual crew members is one of the most informative driving factors of that adaptation.”

For instance, a user who is focused on a cognitively demanding task, such as route planning, may need the interface to eliminate potential distractions, Bohannon said.

Additionally the cognitive state of the individual crew members can inform dynamic tasking, as a vehicle commander is not going to ask a cognitively fatigued crew member to analyze newly arriving sensor data from an unmanned aerial system. Rather, this would allow an intelligent adaptive system to balance the workload of the crew, he said.

“One of the major challenges in developing systems for human-autonomy teaming is reliably estimating the cognitive state of individuals,” said Army researcher Dr. Nicholas Waytowich. “We believe that our approach can be used as a tool for improving cognitive state estimation that will enable the design of autonomous systems that adapt to the user, instead of requiring the user to adapt to the system.”

The researchers believe that their approach also has potential to elucidate the relationship between underlying cognitive states of individuals and emergent team behaviors in teams of humans and autonomous systems, said Army researcher Dr. Vernon Lawhern.

“For example, identifying individuals’ cognitive states that are likely to produce effective teams will be critical to enabling successful team missions,” Lawhern said.

According to Bohannon, it’s very important that this research impacts the Army of the future.

“The nature of basic research-long-term, high-risk-makes it difficult to predict whether this, or any similar effort, will directly impact the Army,” Bohannon said. “That makes it especially important to spend time understanding the challenges that the future Army will face and use that understanding to inform the work that we do.”

The next step is to transition this knowledge into an applied research project such as individual state estimation within the Human-Autonomy Teaming ERP, he said.

###