""Transforming everyday information into practical analytics with crowdsourced assessment tasks

Associate Professor June Ahn is lead author, and third-year doctoral student Ha Nguyen second author, of an article in the Proceedings of the 11th International Learning Analytics and Knowledge Conference – LAK21.
 
The title of the article is “Transforming everyday information into practical analytics with crowdsourced assessment tasks.”
 
Co-authors are doctoral scholar Fabio Campos (New York University) and William Young (Competitive Intelligence Solutions LLC).

​Ahn’s research interests include learning technologies, research-practice partnerships, human-computer interaction, educational technology, and data use and analytics. 

June Ahn

Ha Nguyen

His core research interest is understanding how technology, information, and co-designing solutions with the community, can enhance the way we learn and deliver education. Ahn co-designs technology with community partners for diverse learning contexts. He has engaged in research-practice partnerships around emerging technologies including social media, alternate reality games, and data visualization platforms. He serves as faculty director of Orange County Educational Advancement Network (OCEAN) and a member of the Connected Learning Lab (DLL) at UCI. Currently, he is editor of Educational Researcher.

​Nguyen is specializing in Teaching, Learning, and Educational Improvement (TLEI) for her doctoral work. Her research interests include design of STEM learning experiences and multimodal assessment to study collaboration and conceptual understanding. She is advised by Professors Mark Warschauer and Rossella Santagata.    
 
Abstract
 
Educators use a wide variety of data to inform their practices. Examples of these data include forms of information that are commonplace in schools, such as student work and paper-based artifacts. One limitation in these situations is that there are less efficient ways to process such everyday varieties of information into analytics that are more usable and practical for educators. To explore how to address this constraint, we describe two sets of design experiments that utilize crowdsourced tasks for scoring open-ended assessments. Developing crowdsourced systems and their resulting analytics introduced a variety of challenges, such as attending to the expertise and learning of the crowd. In this paper, we describe the potential efficacy of design decisions such as screening the crowd, providing multimedia instruction, and asking the crowd to explain their answers. We also explore the potential of crowdsourcing as a learning opportunity for those participating in the collective tasks. Our work offers key design implications for leveraging crowdsourcing to process educational data in ways that are relevant to educators, while offering learning experiences for the crowd.