Abstract STEM (Science, Technology, Engineering, Math) concepts are often conveyed visually. Intricate graphics of mathematical data trends and interactive simulations of molecules and electricity help students visualize and understand these concepts in a more concrete way.
Unfortunately, many of these visual-based learning techniques omit the multi-sensorial way students learn and create an accessibility issue for students, especially those with visual impairments.
It’s a problem Jenna Gorlewicz, Ph.D., associate professor of mechanical engineering at Saint Louis University, and her team strives to solve.
“STEM graphics like bar charts, geometric figures, and line or scatter plots, can be very complex,” Gorlewicz said. “We are investigating how to layer the data in meaningful, multimodal ways so that critical themes can be uncovered quickly.”
Through collaborations with PhET Interactive Simulations, Gorlewicz’s team is developing software that incorporates touch feedback into interactive simulations. Interactive simulations have moving elements that change with user input, making nonvisual access challenging.
The exploration of interactive content in collaboration with PhET is the next phase of Gorlewicz’s National Science Foundation-funded CAREER research. Other project components include the design of more complex graphics and simple hardware add-ons to use in conjunction with touchscreens to provide an enhanced touch dimension.
The team is also looking at downsampling and information layering in the creation of multimodal images, or interactive geometric figures, and how this can be accomplished in an automated or semi-automated way.
“Our previous research illustrates the feasibility of doing this for simple graphics,” Gorlewicz said.
Gorlewicz’s research is funded through a 2019 NSF CAREER award.
A Human-Centered Approach to Technology Development
Gorlewicz’s team collaborates regularly with adults and students with visual impairments, teachers and practitioners of the visually impaired, and individuals of all abilities – garnering ideas and feedback from multiple perspectives. This has driven the design of the technology into its current form.
“It’s not just about building something and then testing it with stakeholders,” Gorlewicz said. “It’s an iterative process that takes a human-centered approach from the onset.”
Gorlewicz is passionate about the potential impact of her work, as well as its impact on the training of future engineers.
“This is challenging from an engineering perspective because the solution to this problem is deeply multidisciplinary. We cannot just approach this with a traditional engineering perspective,” said Jennifer Tennison, a graduate student doing her dissertation work on the project. “We have to incorporate psychology, cognition, computer science, statistics, and education. It’s a challenge that has broadened my interpretation of what it means to be an engineer.”
“Bridging the Digital Accessibility Gap in STEM Using Multisensory Haptic Platforms” targets K-12 education. The grant number is NSF 1845490.
The SLU team is partnering on the project with the Special School District of St. Louis County, Missouri School for the Blind, Francis Howell School District, Illinois School for the Visually Impaired, Hollinger Consulting, and the Lighthouse for the Blind – St. Louis.
Founded in 1818, Saint Louis University is one of the nation’s oldest and most prestigious Catholic institutions. Rooted in Jesuit values and its pioneering history as the first university west of the Mississippi River, SLU offers more than 12,000 students a rigorous, transformative education of the whole person. At the core of the University’s diverse community of scholars is SLU’s service-focused mission, which challenges and prepares students to make the world a better, more just place.
