Physiological Measurements of Cognitive Activity

The leading premise of the Digitop Project is that the work involved in digital manufacturing has evolved away from the physical work that is well understood in ergonomics and human factors, towards cognitive work for which we don’t have clear industry standards and safety guidelines, and where we lack knowledge of the impact of increasingly difficult cognitive effort on long term well being. Although models of cognitive workload have been well validated in research and industry, actually estimating these cognitive concepts, like Mental Workload (or cognitive effort or mental effort), Attention, Working Memory, etc, is still an open challenge.

Traditionally, we have relied on either subjective techniques – like NASA TLX (retrospective) or Instantaneous Self Assessment (concurrent) – or secondary task indicators, where ability to cope with a secondary task indicates available mental resources. Each of these are fine for measurement instruments, but they can also affect work or indeed increase the mental effort that we are trying to measure (and probably reduce through research).

Actually estimating Mental Workload is a challenge that many people, including those involved in our proposal, are taking different approaches to, where research has shown that many cognitive activities results in different physiological changes. From an off-body observational approach, high mental workload typically affects the posture of workers as they put effort into concentrating. Eye pupils also typically dilate, in fixed lighting, when mental workload increases. Using thermal cameras, research has also shown that nose temperature typically reduces during periods of high mental workload.

On-body, there are also many measurable changes. Galvanic Skin response and heart changes can be detected from e.g. wearable devices on the wrist, if not from more advanced chest straps that measure heart rate and breathing. My own team has been using functional Near Infra-Red Spectroscopy (fNIRS) for estimating Mental Workload, where increase blood oxygenation (or rather the reduction in de-oxygenated haemoglobin) in the prefrontal cortex is affected by high workload tasks. This approach is more work-tolerant than EEG (which has also been shown to change to Mental Workload), and studies similar physiological changes to an MRI but whilst being more mobile.

Unfortunately, Mental Workload changes are not the only things to cause these physiological changes. Many papers report stress, or emotion, or other related concepts also creating the same physiological changes. This would sound like a problem (or rather a research challenge ^_^ ), but this is largely because these things that we are trying to estimate are concepts, rather than objective physiological changes, that themselves haven’t been formally differentiated or integrated in literature – a recent panel at the Human Factors and Ergonomics Society 2016 Annual Meeting discussed this issue directly and our own work showed that a tool for manipulating stress also manipulates subjective mental workload reports from participants. Further, these physiological changes are affected by environmental factors that create systemic changes in the body that fluctuate over longer time periods, where the body may become more alert over the period of the morning, and have a lower level of arousal after lunch, for example.

A good opportunity for research, however, is to triangulate estimations of these concepts from multiple measures of physiological change, in relation to measurements of the environment and the state of the system the person is using. It is this breakthrough that is the focus of WP3, to significantly develop the way that we estimate cognitive work through integrated physiological measurements. We aim towards a stage where we can allow systems to make reliable judgements on the current capacity of a user, to avoid overloading them with demand, whilst adapting to their current time-of-day, state-of-health, varying capabilities.

With the mix of people involved in the digitop project, we have a great opportunity to investigate and work with different techniques. Further, we get to work with research partners like Bristol Robotics Lab to develop dynamic manufacturing prototypes that respond to these measurements, and to collaborate with industry partners that are providing motivating scenarios for the context of these prototypes.

Dr Max L. Wilson
University of Nottingham

Leave a Reply