"Application of Machine Learning Models for Tracking Participant Skills in Cognitive Training"

Assistant Project Scientist Anja Pahor (left) and Associate Professor Susanne Jaeggi (right) published an article on methods to estimate user skill level and predict appropriate task challenges in Frontiers in Science. The title of their article is Application of Machine Learning Models for Tracking Participant Skills in Cognitive Training.

Co-authors are Sanjana Sandeep (first author), Christian R. Shelton, and Aaron R. Seitz, all from the University of California,  Riverside.

Pahor a member of the UC Irvine Working Memory and Plasticity (WMP) Lab and the UC Riverside Brain Game Center for Mental Fitness and Well-being. Her research interests involve understanding the mechanisms underlying higher-level cognitive functions, particularly memory and reasoning. She has a Ph.D. in Psychology and Cognitive Neuroscience from Univerza v Mariboru in Slovenia, and research experience from Slovenia, The Netherlands, and the United Kingdom. 

​Jaeggi researches training and transfer, individual differences in working memory capacity and executive control, as well as the nature of working memory limitations across the lifespan. She directs UCI's Working Memory and Plasticity Lab. Jaeggi is a  fellow of the Center for the Neurobiology of Learning and Memory and holds a courtesy appointment in the Department of Cognitive Sciences in UCI's School of Social Sciences.

Abstract

A key need in cognitive training interventions is to personalize task difficulty to each user and to adapt this difficulty to continually apply appropriate challenges as users improve their skill to perform the tasks. Here we examine how Bayesian filtering approaches, such as hidden Markov models and Kalman filters, and deep-learning approaches, such as the long short-term memory (LSTM) model, may be useful methods to estimate user skill level and predict appropriate task challenges. A possible advantage of these models over commonly used adaptive methods, such as staircases or blockwise adjustment methods that are based only upon recent performance, is that Bayesian filtering and deep learning approaches can model the trajectory of user performance across multiple sessions and incorporate data from multiple users to optimize local estimates. As a proof of concept, we fit data from two large cohorts of undergraduate students performing WM training using an N-back task. Results show that all three models predict appropriate challenges for different users. However, the hidden Markov models were most accurate in predicting participants' performances as a function of provided challenges, and thus, they placed participants at appropriate future challenges. These data provide good support for the potential of machine learning approaches as appropriate methods to personalize task performance to users in tasks that require adaptively determined challenges.
​