Improving Cognition During Ambulatory Augmented Reality

Augmented reality (AR) via the Integrated Visual Augmentation System (IVAS) has the ability to optimize training and operations by presenting Soldiers with accurate, real-time mission information. The challenge is that the human cognitive system is limited in processing capacity and any additional information provided in AR has the potential to compromise situational awareness, increase task difficulty for the user, and reduce performance, rather than augmenting performance.

This project has three main objectives: 1) identify the cognitive impacts of ambulatory AR on task performance, 2) identify metrics that predict changes in cognition during ambulatory AR, and 3) identify strategies that will improve cognition during ambulatory AR. To achieve these objectives we will fuse data acquired from multiple sensors that can inform the AR system about the user’s cognitive state, with an emphasis on tracking attention, situational awareness and performance while engaged in a real-world, Soldier-relevant task. The selection of sensors is based on evidence in the literature, as well as our previous work. These sensors include those that may be fieldable in near-term training and operational contexts, including eye tracking, accelerometers for gait and head pose.

The results of this project will provide key insights into the impact on attention and situational awareness in particular and how potential negative impacts can be mitigated. In addition, the inclusion of the ambient lighting conditions and virtual/physical objects will set important boundary conditions for deploying AR in realistic training scenarios.