Research-practitioner partnership gives boost to California's first online public college
By Christine Byrd
January 16, 2024 California created a unique educational model when it launched Calbright Community College: a fully online public college designed for working adults. Unlike traditional colleges where credit is earned through course hours, Calbright is entirely competency-based, meaning students can move on as soon as they demonstrate mastery of a certain skill. While this innovative educational model holds the potential to expand higher education opportunities among thousands of adult learners, it also presents challenges for educators and students alike. Recognizing this, Calbright forged an applied research partnership with the UCI School of Education, the nonprofit organization ideas42, and researchers from the University of Virginia and Cornell University to measure and enhance student success. |
“Our partnership is a proactive collaboration between various teams at Calbright and experts in data and measurement, online learning design, and innovative behavioral science to support Calbright’s efforts to identify specific strategies to better support adult learners under a competency-based model,” said Di Xu, principal investigator of the project and associate professor of education at UCI. “As more higher education institutions embrace online programs for adult learners, we aspire to share our findings widely, with the hope that they may offer guidance to others in their endeavors.”
A national team
UCI’s partnership with Calbright began in 2021, when the college was just three years old. One aspect where UCI was well positioned to provide expertise was in tracking and measuring student success. Leveraging insights from the UCI Measuring Undergraduate Success Trajectories (UCI-MUST) project that he leads, Professor of Sociology and Education Richard Arum is helping Calbright develop a data management system that effectively tracks student success, including data on demographics, academic performance and progress, as well as student wellbeing and belonging. Tracing students can be especially tricky because Calbright’s courses are free, fully online and self-paced, but robust data will help the college be more accountable to its students, its board and the state.
“Higher education institutions are often haphazard at best in how they track students,” said Arum. “At UCI, we’ve developed the best measurement system in the world to track undergraduate student trajectories and outcomes, so we’re ideally positioned to help Calbright improve their efforts to measure and track their students.”
In addition to Arum and Xu – who is a national expert in both community colleges and online instructional delivery – the research team includes Jutta Heckhausen, Distinguished Professor of Psychological Science in UCI’s School of Social Ecology, who is designing surveys to measure student wellbeing, and developing some of the experiments and interventions the partnership is utilizing to improve student outcomes. Ben Castleman, associate professor of public policy and education at the University of Virginia, is lending his expertise in “nudges” – small interventions such as periodic digital reminders that help keep students on track – and working with a team of behavioral scientists at ideas42. Cornell University’s René Kizilcec, assistant professor of information science, is a national leader in MOOCs (massive open online courses), and helping design rigorous research experiments to test the effectiveness of various interventions in Calbright programs.
A national team
UCI’s partnership with Calbright began in 2021, when the college was just three years old. One aspect where UCI was well positioned to provide expertise was in tracking and measuring student success. Leveraging insights from the UCI Measuring Undergraduate Success Trajectories (UCI-MUST) project that he leads, Professor of Sociology and Education Richard Arum is helping Calbright develop a data management system that effectively tracks student success, including data on demographics, academic performance and progress, as well as student wellbeing and belonging. Tracing students can be especially tricky because Calbright’s courses are free, fully online and self-paced, but robust data will help the college be more accountable to its students, its board and the state.
“Higher education institutions are often haphazard at best in how they track students,” said Arum. “At UCI, we’ve developed the best measurement system in the world to track undergraduate student trajectories and outcomes, so we’re ideally positioned to help Calbright improve their efforts to measure and track their students.”
In addition to Arum and Xu – who is a national expert in both community colleges and online instructional delivery – the research team includes Jutta Heckhausen, Distinguished Professor of Psychological Science in UCI’s School of Social Ecology, who is designing surveys to measure student wellbeing, and developing some of the experiments and interventions the partnership is utilizing to improve student outcomes. Ben Castleman, associate professor of public policy and education at the University of Virginia, is lending his expertise in “nudges” – small interventions such as periodic digital reminders that help keep students on track – and working with a team of behavioral scientists at ideas42. Cornell University’s René Kizilcec, assistant professor of information science, is a national leader in MOOCs (massive open online courses), and helping design rigorous research experiments to test the effectiveness of various interventions in Calbright programs.
Addressing the challenges
The partnership has already made an impact for Calbright students. One appealing aspect of Calbright’s model is that working adults can move through courses as quickly or as slowly as they like, while balancing work and families. But that open-endedness can also make it hard to see a course through to completion. One of the first solutions, collaboratively designed and piloted by Calbright teams and the behavioral scientists at ideas42, was “Pace and Progress Timelines” which have proved effective at helping students finish their programs. Now, when beginning a new course, Calbright students are matched to a timeline based on how soon they want to earn their certificate, the amount of time they can devote to the program, and their past experience with the subject matter. Once on a timeline, they then receive reminders and special outreach indexed to their progress relative to their planned timeline.
“When we initially piloted the timelines, we saw that students started moving at a meaningfully faster pace relative to a control group, about 50 percent faster,” said Tom Tasche, associate managing director at ideas42. “Removing deadlines may sound like a good way to give working adults the flexibility they need, but it can create different kinds of barriers to completion. What we’re trying to do with the timelines is find the right balance between flexibility and structure to help students stay on track toward their goals.”
The partnership has already made an impact for Calbright students. One appealing aspect of Calbright’s model is that working adults can move through courses as quickly or as slowly as they like, while balancing work and families. But that open-endedness can also make it hard to see a course through to completion. One of the first solutions, collaboratively designed and piloted by Calbright teams and the behavioral scientists at ideas42, was “Pace and Progress Timelines” which have proved effective at helping students finish their programs. Now, when beginning a new course, Calbright students are matched to a timeline based on how soon they want to earn their certificate, the amount of time they can devote to the program, and their past experience with the subject matter. Once on a timeline, they then receive reminders and special outreach indexed to their progress relative to their planned timeline.
“When we initially piloted the timelines, we saw that students started moving at a meaningfully faster pace relative to a control group, about 50 percent faster,” said Tom Tasche, associate managing director at ideas42. “Removing deadlines may sound like a good way to give working adults the flexibility they need, but it can create different kinds of barriers to completion. What we’re trying to do with the timelines is find the right balance between flexibility and structure to help students stay on track toward their goals.”
Another pilot intervention the team has been working on is learning community. Previous research has found that a sense of belonging and connection can fuel students’ persistence. For example, in a study, Xu and colleagues found that structured learning communities improved UCI biology students’ academic performance and a sense of belonging, and contributed to them sticking with their major. Xu and the UCI researchers have been working with cross-functional teams at Calbright to develop and test the impact of offering students opportunities to join live sessions with their instructor, advisor or study groups with other learners, to see if that also improves students’ sense of wellbeing and improves program completion.
|
Meanwhile, Kizilcec and Heckhausen are leading experiments to gauge the effectiveness of psychological approaches to improving performance, such as having students set goals and articulate their reasons for enrolling in the course in the first place. Then, email or text nudges can remind students of their larger purpose and encourage them to keep working toward their goals.
The interdisciplinary team of researchers are driven by the desire to help Calbright's innovative approach achieve its potential to address the evolving needs of California's workforce. With emerging technologies like artificial intelligence reshaping California’s economy, more and more adults may start turning to Calbright or other alternatives to traditional higher education to get the skills they need for in-demand jobs like IT support or database management.
“Calbright is engaged in a series of capacity-building efforts to become the leading edge of competence-based education,” said Xu. “We are excited to be engaged in this partnership work that is aimed at transforming higher education to meet the changing needs of society.”
The interdisciplinary team of researchers are driven by the desire to help Calbright's innovative approach achieve its potential to address the evolving needs of California's workforce. With emerging technologies like artificial intelligence reshaping California’s economy, more and more adults may start turning to Calbright or other alternatives to traditional higher education to get the skills they need for in-demand jobs like IT support or database management.
“Calbright is engaged in a series of capacity-building efforts to become the leading edge of competence-based education,” said Xu. “We are excited to be engaged in this partnership work that is aimed at transforming higher education to meet the changing needs of society.”